Cross-sea bridge health monitoring system and method thereof

Abstract

The invention provides a cross-sea bridge health monitoring system and method. The system comprises a monitoring control middle table, an inspection route planning unit, an unmanned aerial vehicle control unit, an image processing unit, a structural feature extraction unit and a health monitoring unit. The inspection route planning unit plans a target inspection route of the unmanned aerial vehicle through a geographic information system based on structural characteristics and structural parameters of the to-be-monitored cross-sea bridge in combination with weather information at the current time; the unmanned aerial vehicle control unit controls the unmanned aerial vehicle to carry out image acquisition on the cross-sea bridge by utilizing visual shooting equipment according to the target inspection route to obtain an initial inspection image; the image processing unit removes the invalid image to obtain a target inspection image; a structural feature extraction unit performs feature extraction on the target inspection image to obtain key structural features; and the health monitoring unit generates a health monitoring report of the cross-sea bridge based on the key structure features. The health condition of the cross-sea bridge is accurately mastered, and safe operation of the cross-sea bridge is guaranteed.

Description

Technical Field

[0001] The present invention relates to the technical field of unmanned aerial vehicles, and in particular to a sea-crossing bridge health monitoring system and method thereof. Background Art

[0002] At present, the common health monitoring methods of cross-sea bridges mostly rely on the deployment of a large number of sensors, such as strain gauges, accelerometers, etc., at key parts of the bridge structure. These sensors are connected to the data acquisition system by wire or wireless means to collect physical parameters such as stress and vibration of the bridge in real time. However, this method has a major disadvantage, that is, the deployment range of sensors is limited and it is difficult to cover the entire area of ​​the bridge, especially some difficult-to-reach parts, such as the suspended area under the bridge, complex node parts, etc. Due to the limitations of the number and location of sensors, diseases or damage in some local areas may be missed, resulting in the inability to fully and timely grasp the health status of the cross-sea bridge. Summary of the invention

[0003] The present invention provides a cross-sea bridge health monitoring system and method thereof, aiming to accurately grasp the health status of the cross-sea bridge and ensure the safe operation of the cross-sea bridge.

[0004] In a first aspect, the present invention provides a cross-sea bridge health monitoring system, including a monitoring and control center, an inspection route planning unit, an unmanned aerial vehicle control unit, an image processing unit, a structural feature extraction unit, and a health monitoring unit; the monitoring and control center is respectively connected to the inspection route planning unit, the unmanned aerial vehicle control unit, the image processing unit, the structural feature extraction unit, and the health monitoring unit to manage each unit;

[0005] The inspection route planning unit is used to plan the target inspection route of the UAV through the geographic information system based on the structural characteristics and structural parameters of the cross-sea bridge to be monitored and the weather information at the current time;

[0006] A UAV control unit is used to control the UAV to collect images of the surface, structural joints and piers of the cross-sea bridge using a visual shooting device according to the target inspection route to obtain an initial inspection image;

[0007] An image processing unit, used for removing invalid images containing light reflections and light shadows in the initial inspection image based on an image feature recognition algorithm, to obtain a target inspection image;

[0008] A structural feature extraction unit, used to extract features from the target inspection image to obtain key structural features;

[0009] A health monitoring unit is used to generate a health monitoring report of the cross-sea bridge based on the key structural features; the health monitoring report includes the corresponding abnormal parts of the cross-sea bridge and the location, type and severity of the abnormal parts.

[0010] In a second aspect, the present invention further provides a cross-sea bridge health monitoring method, which is implemented based on the cross-sea bridge health monitoring system described in the first aspect, and the cross-sea bridge health monitoring method comprises:

[0011] Based on the structural characteristics and structural parameters of the cross-sea bridge to be monitored and the current weather information, the target inspection route of the UAV is planned through the geographic information system;

[0012] Controlling the UAV to collect images of the surface, structural joints and piers of the cross-sea bridge using visual shooting equipment according to the target inspection route to obtain an initial inspection image;

[0013] Based on an image feature recognition algorithm, invalid images containing light reflections and light shadows are removed from the initial inspection image to obtain a target inspection image;

[0014] Extracting features from the target inspection image to obtain key structural features;

[0015] A health monitoring report of the cross-sea bridge is generated based on the key structural features; the health monitoring report includes the corresponding abnormal parts of the cross-sea bridge and the location, type and severity of the abnormal parts.

[0016] In a third aspect, the present invention further provides an electronic device, comprising: a memory for storing a computer software program; a processor for reading and executing the computer software program, thereby implementing any of the above-mentioned methods for monitoring the health of a cross-sea bridge.

[0017] In a fourth aspect, the present invention further provides a non-transitory computer-readable storage medium, wherein the storage medium stores a computer software program, and when the computer software program is executed by a processor, the method for monitoring the health of a cross-sea bridge as described above is implemented.

[0018] In a fifth aspect, the present invention further provides a computer program product, comprising a computer program, which, when executed by a processor, implements any of the above-described methods for monitoring the health of a cross-sea bridge.

[0019] The cross-sea bridge health monitoring system provided by the embodiment of the present invention can comprehensively cover all key parts of the bridge, including areas that are difficult to reach with existing sensors, by planning routes using GIS. The surface of the bridge and complex structural parts can be photographed without blind spots through the drone flight shooting process to obtain comprehensive image data. Through the processing, feature extraction and comparative judgment of the image data, the diseases and damages of various parts can be found. It is no longer limited by the arrangement position and number of sensors, and the health status of the cross-sea bridge can be comprehensively detected, which effectively solves the shortcoming of incomplete monitoring areas in the existing methods, thereby accurately grasping the health status of the cross-sea bridge and ensuring the safe operation of the cross-sea bridge. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 It is a structural schematic diagram of the health monitoring system for a cross-sea bridge provided by the present invention;

[0021] Figure 2 It is a schematic diagram of the flow of the cross-sea bridge health monitoring method provided by the present invention;

[0022] Figure 3 An embodiment diagram of an electronic device provided by an embodiment of the present invention;

[0023] Figure 4 An embodiment diagram of a computer-readable storage medium provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0025] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0026] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or explanation". Any embodiment described as "for example" in the present invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present invention.

[0027] Optional, see Figure 1 As shown, Figure 1 It is a structural schematic diagram of the cross-sea bridge health monitoring system provided by the present invention. The cross-sea bridge health monitoring system includes a monitoring and control center, an inspection route planning unit, an unmanned aerial vehicle control unit, an image processing unit, a structural feature extraction unit and a health monitoring unit; the monitoring and control center is respectively connected to the inspection route planning unit, the unmanned aerial vehicle control unit, the image processing unit, the structural feature extraction unit and the health monitoring unit to manage each unit.

[0028] Optionally, the inspection route planning unit comprehensively collects the structural characteristics and structural parameters of the cross-sea bridge to be monitored, where the structural characteristics include the length, width, distribution of piers, and bridge structure type (beam bridge, cable-stayed bridge, suspension bridge, etc.). Structural parameters include the size and material of each part of the bridge. At the same time, obtain the weather information of the current time, such as wind speed, wind direction, visibility, rainfall, etc. Geographic Information System (GIS) has powerful spatial analysis and visualization capabilities.

[0029] Furthermore, the inspection route planning unit inputs the structural data and weather data of the bridge into GIS, and through its spatial analysis tools, comprehensively considers the flight performance of the drone (such as endurance, maximum flight speed, wind resistance, etc.), and plans a target inspection route that can fully cover the bridge surface, structural connections and piers that need to be monitored.

[0030] Optionally, the drone control unit establishes a connection with the drone through a wireless communication module and transmits the target inspection route data to the drone. The drone is equipped with a high-precision visual shooting device, such as a high-definition camera with a resolution of 4K, and starts flying according to the received route instructions. During the flight, the drone reaches a preset shooting point, such as at certain intervals (such as 10 meters) on the surface of the bridge, at key monitoring locations of structural joints and piers, triggering the visual shooting device to collect images. The collected images are transmitted back to the health monitoring system in real time and stored in the system's large-capacity data storage device to form an initial inspection image set.

[0031] Optionally, the image processing unit calls the image feature recognition algorithm library to analyze the stored initial inspection images one by one. Light reflection and light shadow will interfere with the accurate judgment of the bridge structure, making it difficult to identify key structural features in the image. The algorithm identifies abnormal brightness areas in the image. For example, light reflection will cause the local area to be too bright and the pixel value will exceed the normal range; light shadow will cause the pixel value of some areas to be too low. For images with abnormal brightness areas exceeding a certain proportion (such as 20%), they are judged as invalid images and removed from the initial inspection image set, and finally a clear and effective target inspection image set is obtained.

[0032] Optionally, the structural feature extraction unit applies a special image feature extraction algorithm to the target inspection image to obtain key structural features, wherein the key structural features include crack edge features, structural deformation contour features, and structural joint displacement features. For crack edge feature extraction, an edge detection algorithm, such as the Canny algorithm, is used. This algorithm calculates the image gradient and finds edges with obvious gradient changes. It can accurately identify the edge contour of cracks on the surface of the bridge body and extract its shape, length, width and other characteristic parameters. For structural deformation contour features, image matching technology is used to compare the current inspection image with the reference image of the bridge in a normal state, calculate the displacement, distortion and other deformation information of each part of the image, and extract the contour features of the structural deformation. For structural joint displacement features, by analyzing the relative position changes of different structural components in the joint area image, a feature point matching algorithm, such as the SIFT algorithm, is used to identify whether there is displacement at the structural joint, and obtain key information such as the direction and distance of the displacement.

[0033] Optionally, the health monitoring unit conducts a comprehensive analysis and judgment based on the extracted key structural features. For the identified cracks, the severity is judged based on the length, width and development trend of the cracks. For example, cracks with a length of more than 10 cm and a width of more than 2 mm are judged to be more serious cracks. For structural deformation, the severity is assessed based on the degree of deformation and the impact on structural stability. For displacement at structural joints, the severity is determined based on the displacement distance and the impact on the mechanical properties of the connection parts. The system organizes the location (determined by the image shooting location combined with the route information), type (cracks, deformation, displacement, etc.) and severity of all abnormal parts into a table form, and combines it with a text description to generate a health monitoring report for the cross-sea bridge. The report is stored in PDF or Excel format for subsequent reference and analysis.

[0034] The embodiment of the present invention utilizes GIS to plan routes to fully cover all key parts of the bridge, including areas that are difficult to reach with existing sensors. The surface of the bridge and complex structural parts can be photographed without blind spots through the drone flight shooting process to obtain comprehensive image data. Through the processing, feature extraction and comparative judgment of the image data, the diseases and damages of various parts can be discovered. It is no longer limited by the arrangement position and number of sensors, and the health status of the cross-sea bridge can be fully detected, which effectively solves the shortcoming of incomplete monitoring areas, thereby accurately grasping the health status of the cross-sea bridge and ensuring the safe operation of the cross-sea bridge.

[0035] Optional, see Figure 2 , Figure 2 : is a flow chart of the cross-sea bridge health monitoring method provided by the present invention. The execution subject of the cross-sea bridge health monitoring method in the embodiment of the present invention is a health monitoring system, therefore, the cross-sea bridge health monitoring method includes:

[0036] Step 10, based on the structural characteristics and structural parameters of the cross-sea bridge to be monitored and the weather information at the current time, the target inspection route of the UAV is planned through the geographic information system.

[0037] Optionally, the health monitoring system comprehensively collects the structural characteristics and structural parameters of the cross-sea bridge to be monitored, among which the structural characteristics include the length, width, distribution of piers, and bridge structure type (beam bridge, cable-stayed bridge, suspension bridge, etc.). Structural parameters include the size and material of each part of the bridge. At the same time, obtain the current weather information, such as wind speed, wind direction, visibility, rainfall, etc. The Geographic Information System (GIS) has powerful spatial analysis and visualization capabilities.

[0038] Furthermore, the health monitoring system inputs the structural data and weather data of the bridge into GIS, and through its spatial analysis tools, comprehensively considers the flight performance of the drone (such as endurance, maximum flight speed, wind resistance, etc.), and plans a target inspection route that can fully cover the bridge surface, structural connections and piers that need to be monitored, as described in steps 101 to 104.

[0039] In one embodiment, a cable-stayed bridge with a length of 5 kilometers is monitored, the main bridge spans 1 kilometer, and there are 20 piers evenly distributed on the approach bridges on both sides of the main bridge. The current weather is light wind (wind speed 3-5m / s) and visibility is good. The health monitoring system obtains the CAD drawings of the bridge, converts them into a format recognizable by GIS, and enters the bridge structure parameters. In the GIS software, according to the performance of the drone's endurance of 1 hour, maximum flight speed of 50km / h, and wind resistance of 8m / s, the flight rules are set: the drone maintains a safe distance of 50 meters from the surface and structural parts of the bridge for circumferential shooting. Through the path planning function of GIS, the drone is planned to start from one end of the bridge, fly longitudinally along the bridge body, circle the piers at each pier position to shoot, and then continue to fly to the other end of the bridge. The entire route planning ensures that the inspection is completed within the drone's endurance, and considering the breeze, the flight direction is as consistent as possible with the wind direction to reduce energy consumption. Step 20, controlling the UAV to collect images of the surface, structural joints and piers of the cross-sea bridge using visual shooting equipment according to the target inspection route to obtain an initial inspection image.

[0040] Furthermore, the health monitoring system establishes a connection with the drone through a wireless communication module and transmits the target inspection route data to the drone. The drone is equipped with a high-precision visual shooting device, such as a high-definition camera with a resolution of 4K, and starts flying according to the received route instructions. During the flight, the drone reaches the preset shooting points, such as at certain intervals (such as 10 meters) on the bridge surface, at key monitoring locations of structural joints and piers, triggering the visual shooting device to collect images. The collected images are transmitted back to the health monitoring system in real time and stored in the system's large-capacity data storage device to form an initial inspection image set.

[0041] In one embodiment, the drone flies to the connection between the main bridge and the approach bridge of the cable-stayed bridge according to the planned route, which is the key part of the structural monitoring. The health monitoring system sends a shooting command, and the drone hovers at a distance of 50 meters from the connection, adjusts the camera angle, and shoots the steel beam connection part, concrete bonding surface, etc. at the connection from multiple angles, and collects 10 high-definition images from different sides. When flying along the surface of the bridge body, the drone automatically takes an image of the bridge body surface every 10 meters to record whether there are cracks, peeling, etc. on the surface. When reaching the pier position, the drone flies around the pier, shoots the four sides of the pier and the connection part with the bridge body, and collects 20 images for each pier. The drone continues to fly for about 40 minutes, collects a total of about 1,000 initial inspection images, and transmits them back to the health monitoring system in real time.

[0042] Step 30: Based on an image feature recognition algorithm, invalid images with light reflections and light shadows in the initial inspection image are removed to obtain a target inspection image.

[0043] Furthermore, the health monitoring system calls the image feature recognition algorithm library to analyze the stored initial inspection images one by one. Light reflection and light shadow will interfere with the accurate judgment of the bridge structure, making it difficult to identify key structural features in the image. The algorithm identifies abnormal brightness areas in the image. For example, light reflection will cause the local area to be too bright and the pixel value will exceed the normal range; light shadow will cause the pixel value of some areas to be too low. For images with abnormal brightness areas exceeding a certain proportion (such as 20%), they are judged as invalid images and removed from the initial inspection image set, and finally a clear and effective target inspection image set is obtained.

[0044] In one embodiment, in the initial inspection image set, there is an image of the bridge surface. Due to the reflection of the sun, large white spots appear in about 30% of the area in the image, and the surface condition of the bridge cannot be seen clearly. Using an image feature recognition algorithm based on histogram analysis and threshold segmentation, the brightness histogram of the image is calculated, and it is found that the proportion of pixels with too high brightness exceeds the set 20% threshold. The system automatically marks the image as an invalid image and deletes it from the image set. After the algorithm processes 1,000 initial inspection images, a total of 150 invalid images caused by light reflection and shadows are removed, and 850 target inspection images are obtained.

[0045] Step 40: extract features from the target inspection image to obtain key structural features.

[0046] Furthermore, the health monitoring system uses a special image feature extraction algorithm for the target inspection image to obtain key structural features, where the key structural features include crack edge features, structural deformation contour features, and structural joint displacement features, as specifically described in steps 401 to 406.

[0047] For crack edge feature extraction, edge detection algorithms such as the Canny algorithm are used. This algorithm calculates the image gradient and finds edges with obvious gradient changes. It can accurately identify the edge contours of cracks on the bridge surface and extract its characteristic parameters such as shape, length, and width. For structural deformation contour features, image matching technology is used to compare the current inspection image with the reference image of the bridge in a normal state, calculate the displacement, distortion and other deformation information of each part of the image, and extract the contour features of structural deformation. For structural joint displacement features, by analyzing the relative position changes of different structural components in the joint area image, feature point matching algorithms such as the SIFT algorithm are used to identify whether there is displacement at the structural joint and obtain key information such as the direction and distance of the displacement.

[0048] In one embodiment, in a target inspection image, a certain area on the surface of the bridge body is photographed. The health monitoring system uses the Canny algorithm to process the image and detects a crack edge with a length of 5 cm and a width of about 1 mm. The system extracts characteristic information such as the starting point and end point coordinates of the crack and the direction of the crack. For an image of a structural connection, it is matched with the reference image by SIFT feature point matching, and it is found that the steel beam at the connection has a relative displacement of 3 mm. The system records the displacement direction and distance and other structural connection displacement features. By extracting features from 850 target inspection images, a total of 50 crack edge features, 10 structural deformation contour features, and 5 structural connection displacement features were identified.

[0049] Step 50, generating a health monitoring report of the cross-sea bridge based on the key structural features.

[0050] Furthermore, the health monitoring system conducts a comprehensive analysis and judgment based on the extracted key structural features. For the identified cracks, the severity is judged based on the length, width and development trend of the cracks. For example, cracks with a length of more than 10 cm and a width of more than 2 mm are judged to be more serious cracks. For structural deformation, the severity is assessed based on the degree of deformation and the impact on structural stability. For displacement at structural connections, the severity is determined based on the displacement distance and the impact on the mechanical properties of the connection. The system organizes the location (determined by the image shooting location combined with the route information), type (cracks, deformation, displacement, etc.) and severity of all abnormal parts into a table form, and combines it with a text description to generate a health monitoring report for the cross-sea bridge. The report is stored in PDF or Excel format for subsequent reference and analysis, as described in steps 501 to 503.

[0051] In one embodiment, after analysis, the health monitoring system points out in the report that: at the connection between the fifth pier and the bridge body of the cable-stayed bridge approach, the structural connection was found to be displaced, with a displacement distance of 5 mm, which was determined to be moderately serious, and the location coordinates were (based on GIS positioning information) [x1, y1]. A crack was found on the surface of the main bridge body, with a length of 8 cm and a width of 1.5 mm, which was determined to be slightly serious, and the location coordinates were [x2, y2]. The report is presented in the form of an Excel table, including columns such as abnormal part, location, type, and severity. At the same time, at the beginning of the report, the main abnormalities found in this monitoring and the preliminary assessment of the overall health of the bridge are summarized in text, providing a strong basis for subsequent maintenance decisions.

[0052] The embodiment of the present invention utilizes GIS to plan routes to fully cover all key parts of the bridge, including areas that are difficult to reach with existing sensors. The surface of the bridge and complex structural parts can be photographed without blind spots through the drone flight shooting process to obtain comprehensive image data. Through the processing, feature extraction and comparative judgment of the image data, the diseases and damages of various parts can be discovered. It is no longer limited by the arrangement position and number of sensors, and the health status of the cross-sea bridge can be fully detected, which effectively solves the shortcoming of incomplete monitoring areas, thereby accurately grasping the health status of the cross-sea bridge and ensuring the safe operation of the cross-sea bridge.

[0053] In one embodiment, the description of steps 101 to 104 is as follows:

[0054] Step 101, based on the three-dimensional modeling function of the geographic information system and the structural parameters, a three-dimensional model of the cross-sea bridge is constructed in the geographic information system.

[0055] Optionally, the health monitoring system calls the three-dimensional modeling tool of the geographic information system (GIS) and inputs the obtained cross-sea bridge structural parameters, such as the dimensions of each part of the bridge body (beam length, width, height, pier diameter, height, etc.), material properties and other information. GIS uses these precise data to gradually build a three-dimensional model corresponding to the actual cross-sea bridge in its virtual environment by spatial coordinate positioning and geometric figure construction. The model can intuitively present the overall appearance of the bridge, the spatial position relationship of each structural component and the detailed structure. In one embodiment, for a cross-sea bridge with a main bridge of a double-tower double-cable-plane cable-stayed bridge structure, the main bridge has a span of 800 meters, a beam width of 30 meters, a main tower height of 200 meters, and 20 piers with a diameter of 5 meters and a height of 30 meters distributed in the approach bridge part. The health monitoring system organizes these structural parameters into a data file in a specific format and imports them into the GIS three-dimensional modeling module. According to the parameters, GIS first determines the position coordinates of the bridge body in the geographic space, and establishes a coordinate system with the starting point of the bridge as the origin. Next, a rectangular model was constructed based on the size of the beam to represent the beam, and a cylindrical model was constructed to represent the bridge pier. By adjusting the position and angle of the model, it accurately simulated the connection between the bridge and the pier in reality. For the cable-stayed cable, a line model was used to connect the main tower and the corresponding position of the beam, and finally a three-dimensional model of the cross-sea bridge was successfully constructed. The model clearly shows the structural details of each part of the bridge and their layout in space.

[0056] Step 102, dividing the three-dimensional model based on the structural characteristics of the cross-sea bridge to obtain different inspection areas in the three-dimensional model.

[0057] Furthermore, the health monitoring system divides the constructed three-dimensional model in detail according to the different structural characteristics of the cross-sea bridge. For parts with complex structures and prone to problems, such as the connection between the bridge body and the piers, the cable-beam anchorage area of ​​the cable-stayed bridge, etc., they are separately divided into key inspection areas; for relatively regular and single-structured bridge surfaces, they are divided into several general inspection areas based on factors such as length or area. In the division process, the shooting angle, flight convenience and monitoring accuracy requirements of subsequent drone inspections are fully considered to ensure that each inspection area can be efficiently and comprehensively covered, and there is no omission or overlap between areas. In the above three-dimensional model of the cable-stayed bridge, the health monitoring system divides the 20 connections between the bridge body and the piers into a key inspection area, because these parts are subject to greater stress and are key nodes for structural safety. For the surface of the main bridge body, it is divided into 8 general inspection areas with every 100 meters as a section; the surface of the approach bridge body is divided into several general inspection areas with every 100 meters as a section according to the length. The cable-beam anchorage area of ​​the cable-stayed bridge is a complex structure and is crucial to the stability of the bridge. Each anchorage area is divided into a key inspection area. In this way, the 3D model of the entire cross-sea bridge is reasonably divided into multiple different types of inspection areas, with a total of 30 key inspection areas (20 bridge body and pier connections + 10 cable-beam anchorage areas).

[0058] Step 103 , with the goal of covering the entire area and taking images with a clarity greater than a preset threshold, a route is planned for each inspection area, and a preliminary inspection route for each inspection area is obtained.

[0059] Furthermore, for each divided inspection area, the health monitoring system plans the route with the primary goal of ensuring that the clarity of the image taken by the drone is greater than the preset threshold. The system calculates the optimal flight altitude, angle and path of the drone in the area based on the shape, size and spatial position of the inspection area, combined with the parameters of the visual shooting equipment carried by the drone (such as lens focal length, shooting angle, etc.). For key inspection areas, more detailed and intensive routes are usually planned to ensure that key parts are photographed from multiple angles to obtain comprehensive and clear images; for general inspection areas, relatively simple and efficient routes are planned on the premise of meeting the monitoring accuracy requirements to improve inspection efficiency. For example, for rectangular bridge surface inspection areas, drones may adopt parallel reciprocating flight paths; for irregularly shaped key inspection areas, such as cable-beam anchorage areas, circular flight paths are planned.

[0060] In one embodiment, in a key inspection area at the connection between a bridge body and a bridge pier, the health monitoring system calculates that the drone can capture a clear and complete image covering the area when flying at an altitude of 30 meters from the surface of the area based on the shape of the area (approximately the junction of a cylinder and a cuboid) and the parameters of the drone shooting equipment (lens focal length 25mm, shooting angle 120°). The system plans the drone to take pictures from different heights and angles with the area as the center using a spiral orbiting flight path. During the flight, the drone maintains a stable flight speed, takes a group of images at different angles for each circle, and circles 3 times in total, ensuring that the connection is photographed from at least 9 different angles to obtain sufficiently clear images for subsequent analysis. For a general inspection area on the surface of a rectangular bridge body with a length of 100 meters and a width of 30 meters, the system plans the drone to fly at an altitude of 50 meters from the surface of the bridge body in a parallel reciprocating path, with a flight speed set to 5 meters per second, and takes an image every 2 meters during the flight, so as to ensure that the clarity of the captured image meets the preset threshold and fully covers the inspection area.

[0061] Step 104 , optimizing the preliminary inspection route of each inspection area based on the geographic information system combined with weather information, and obtaining the target inspection route of the drone for each inspection area.

[0062] Furthermore, the health monitoring system optimizes the preliminary inspection route of each inspection area according to the geographic information system combined with weather information to obtain the target inspection route of the drone for each inspection area, as specifically described in steps 1041 to 1044 .

[0063] The embodiment of the present invention can plan a set of comprehensive, accurate and practical target inspection routes for drones. The routes are constructed based on the real structural characteristics and parameters of the cross-sea bridge, taking full account of the monitoring focus of different areas to ensure effective coverage of all parts of the bridge. At the same time, it is optimized in combination with real-time weather information and geographical environmental factors to ensure that the drone can obtain high-quality inspection images with the best shooting angle and parameters under the premise of safe flight, providing a solid data foundation for the subsequent accurate analysis of the structural health status of the cross-sea bridge, greatly improving the efficiency and accuracy of the health monitoring of the cross-sea bridge.

[0064] In one embodiment, the description of steps 1041 to 1044 is as follows:

[0065] Step 1041 , performing a first optimization on the preliminary inspection route of each inspection area based on weather information to obtain a first optimized inspection route.

[0066] Optionally, the health monitoring system obtains weather information in real time, such as wind speed, wind direction, rainfall, temperature and other data. For the preliminary inspection route of each inspection area, the system makes targeted adjustments based on these weather information. If the wind speed is large, in order to reduce the energy consumption of the drone flight and ensure flight stability, the system will adjust the route direction so that the flight direction of the drone is consistent with the wind direction as much as possible. In rainy weather, in order to ensure the quality of image acquisition, the system may shorten the flight path, reduce the exposure time of the drone in the rain, or adjust the flight altitude to avoid areas with dense rainfall. In one embodiment, when inspecting a cross-sea bridge, the wind speed is 8m / s, and the wind direction blows from the southwest end of the bridge to the northeast end. In a certain bridge surface inspection area, the preliminary inspection route is a reciprocating path perpendicular to the direction of the bridge. After analyzing the weather information, the health monitoring system adjusts the preliminary inspection route of the area to be consistent with the wind direction, that is, a one-way flight path from southwest to northeast. In this way, the drone can use wind power to reduce energy consumption and improve flight stability during flight.

[0067] Step 1042, based on the flight speed of the drone and the first optimized inspection route of each inspection area, calculate the estimated flight time required for the drone to cruise each inspection area.

[0068] Furthermore, the health monitoring system obtains the flight speed parameters of the drone, and combines the first optimized inspection route information of each inspection area. By calculating the total length of the route, and then according to the flight speed formula: time = distance / speed, the estimated flight time required for the drone to cruise each inspection area is obtained. This estimated flight time is crucial for the subsequent evaluation of the overall inspection time and further optimization of the route. In one embodiment, the total length of the first optimized inspection route of a certain inspection area is calculated to be 2000 meters, and the flight speed set for the drone is 10 meters per second. The health monitoring system calculates through a formula that the estimated flight time of the drone in this area is 2000 / 10 = 200 seconds. The system performs similar calculations for each inspection area in turn to obtain the estimated flight time data for each area.

[0069] Step 1043, based on the estimated flight time of each inspection area, the total cruising time for the cross-sea bridge is determined, and the first optimized inspection route of each inspection area is optimized for a second time with the total flight time of the UAV as a constraint to obtain the second optimized inspection route of each inspection area.

[0070] Furthermore, the health monitoring system summarizes the estimated flight time for each inspection area to determine the total cruise time required to cruise the entire cross-sea bridge. This total cruise time is compared with the total flight time of the drone. If the total cruise time exceeds the drone's flight time, the system adjusts the first optimized inspection route for each inspection area based on the drone's flight time. For example, the inspection path length of certain non-critical areas is appropriately reduced, or the flight speed is adjusted to balance the flight time allocation of each area, thereby obtaining the second optimized inspection route for each inspection area.

[0071] In one embodiment, for example, after calculation in step 1042, the total estimated flight time of all inspection areas is 30 minutes, while the total flight time of the drone is 25 minutes. The health monitoring system analysis found that the inspection paths of some approach bridge areas are long and relatively non-critical. Therefore, the system shortens the first optimized inspection routes of these approach bridge areas to reduce unnecessary flight paths. At the same time, the flight speed of the drone in some key areas is appropriately increased. On the premise of ensuring the monitoring quality of key areas, the total cruising time is adjusted to within 25 minutes to obtain the second optimized inspection route for each area.

[0072] Step 1044, based on the spatial analysis function of the geographic information system, collision detection is performed on the second optimized inspection route of each inspection area with the goal of no collision points, and the target inspection route of the drone for each inspection area is obtained.

[0073] Furthermore, the health monitoring system uses the spatial analysis function of the geographic information system (GIS) to import the second optimized inspection route of each inspection area into the GIS environment. The GIS system contains geographical environment information around the cross-sea bridge, such as location data of obstacles such as buildings, mountains, and high-voltage lines. The system performs collision detection on each route with the goal of no collision point. If a certain route is detected to have a collision risk with an obstacle, the system automatically adjusts the route to avoid the obstacle until a safe, collision-free target inspection route for each inspection area is obtained. In one embodiment, in an inspection area near the shore, the second optimized inspection route is found through GIS collision detection that the route will pass over a building. The health monitoring system uses the spatial analysis tool of GIS to adjust the route so that the drone flight path bypasses the building and passes through the open area next to the building, and finally obtains a safe target inspection route. The system performs similar collision detection and adjustment on the routes of all inspection areas to ensure that the drone will not collide with any obstacles during the entire inspection process.

[0074] The embodiment of the present invention can generate a highly safe, efficient and practical target inspection route for the UAV. The route not only takes into account the impact of weather factors on flight, ensuring that the UAV can fly stably and complete the monitoring task in complex weather conditions, but also reasonably plans the path according to the UAV's flight time to ensure that the inspection work can fully cover the entire cross-sea bridge. At the same time, with the help of the powerful spatial analysis function of GIS, the risk of collision during flight is completely eliminated, which greatly improves the reliability and success rate of UAV inspections, and provides a solid guarantee for the accurate health monitoring of cross-sea bridges.

[0075] In one embodiment, the description of steps 401 to 406 is as follows:

[0076] Step 401 : segment the target inspection image of each inspection area into a plurality of sub-image blocks.

[0077] Optionally, for the target inspection image of each inspection area, since the target inspection image is usually large in size, direct processing by the health monitoring system is not conducive to accurate analysis. The system adopts a fixed-size sliding window technology to slide the window on the image row by row and column by column to divide the image into multiple interrelated and consistent-size sub-image blocks. The window size is determined based on the image resolution, monitoring accuracy requirements, and the adaptability of the subsequent feature extraction algorithm. The segmentation process ensures that each part of the image can be included in the sub-image block, and there may be partial overlapping areas between adjacent sub-image blocks, so as to better capture the local and overall feature relationship of the image. In one embodiment, the resolution of the target inspection image of a certain inspection area is 4000*3000 pixels. The health monitoring system determines to use a sliding window of 200*200 pixels to segment the image according to the monitoring accuracy requirements. Starting from the upper left corner of the image, the window slides 200 pixels to the right each time. After completing a row of segmentation, the window moves down 200 pixels to continue the segmentation of the next row. At the edge of the image, if the remaining image width or height is less than 200 pixels, the window still performs the last covering segmentation. In this way, the target inspection image is divided into 3000 ((4000 / 200)*(3000 / 200)) sub-image blocks.

[0078] Step 402: Determine a first initial local feature of each first sub-image block based on the pixel brightness of each pixel in each first sub-image block.

[0079] Furthermore, for each first sub-image block segmented out, the health monitoring system deeply analyzes the pixel brightness information of each pixel therein. Pixel brightness is one of the most basic features of an image, reflecting the grayscale value or color intensity of the image at that point. The system preliminarily characterizes the local features of the sub-image block by statistically analyzing the distribution of pixel brightness within the sub-image block, such as calculating statistics such as average brightness, brightness standard deviation, maximum and minimum brightness. These statistics can intuitively reflect the overall brightness and darkness of the sub-image block and the severity of brightness changes, providing basic data for subsequent feature analysis.

[0080] In one embodiment, in a first sub-image block of 200×200 pixels, the health monitoring system traverses 40,000 pixels therein and reads the brightness value of each pixel (for example, a grayscale image with a brightness value range of 0-255). After calculation, the average brightness of the sub-image block is 120, the standard deviation of the brightness is 20, the maximum brightness is 200, and the minimum brightness is 50. These values ​​constitute the first initial local features of the first sub-image block, and the system records these feature values ​​for subsequent further feature analysis and processing.

[0081] Step 403: Based on the initial local features of each first sub-image block and the second initial local features of the adjacent second sub-image block, the initial local features of each first sub-image block are updated to obtain updated local features of each first sub-image block.

[0082] Furthermore, the health monitoring system takes into account that adjacent areas in an image often have a certain correlation, and the features of a single sub-image block depend not only on its own pixel information, but also on the surrounding areas. Therefore, the system conducts a comprehensive analysis of the initial local features of each first sub-image block and the second initial local features of its adjacent second sub-image block. By comparing the differences and similarities in initial local features such as brightness statistics of adjacent sub-image blocks, the initial local features of the first sub-image block are adjusted and updated using specific rules. For example, if the average brightness of adjacent sub-image blocks is similar, the average brightness of the current sub-image block may be smoothed to make it more coordinated with the surrounding area, thereby obtaining updated local features that better reflect the local real features of the image and are closely related to the surrounding area.

[0083] In one embodiment, there are two adjacent first sub-image blocks A and second sub-image blocks B, the initial average brightness of sub-image block A is 120, and the initial average brightness of sub-image block B is 115, which are relatively close. The health monitoring system updates the average brightness of A by weighted average (here only to illustrate the update method, not the weighted summation algorithm). Assuming that the weight of A is 0.6 and the weight of B is 0.4, the average brightness of sub-image block A after update is (120×0.6+115×0.4)=118. At the same time, similar association update operations are performed on other initial local features such as brightness standard deviation, and finally the updated local features of sub-image block A are obtained, so that it can be better integrated into the overall feature system of the image.

[0084] Step 404 : Perform structural texture analysis based on the arrangement pattern of pixels in each first sub-image block to obtain texture pattern features of each first sub-image block.

[0085] Furthermore, the health monitoring system performs structural texture analysis on each first sub-image block. Texture is a local pattern that recurs in an image and contains rich structural information. The system determines the texture pattern by identifying the arrangement pattern of pixels in the sub-image block, such as whether there are periodic lines, spot distribution and other features. Common methods include analysis based on the grayscale co-occurrence matrix, which calculates the grayscale co-occurrence probability of pixel pairs in different directions and distances, and extracts texture feature parameters such as contrast, correlation, energy and entropy to quantitatively describe the texture pattern characteristics of the sub-image block, helping to determine the structural characteristics of the corresponding area in the image, such as whether it is a flat surface, whether there are cracks and other texture anomalies.

[0086] In one embodiment, in a first sub-image block, the health monitoring system uses the gray level co-occurrence matrix method to perform texture analysis. The calculation directions are set to 0°, 45°, 90°, and 135°, and the distance is 1 pixel. After calculation, in the 0° direction, the contrast of the sub-image block is 30, the correlation is 0.8, the energy is 0.05, and the entropy is 3.5. These texture feature parameters indicate that the arrangement of pixels in the sub-image block shows a certain regularity, which may correspond to a relatively smooth area on the bridge surface. If the contrast calculated in other sub-image blocks is extremely high and the energy is low, it may indicate that there is a texture mutation in the area, such as an abnormal structure such as a crack edge.

[0087] Step 405: Perform Fourier transform on each first sub-image block to convert each first sub-image block from the spatial domain to the frequency domain, and obtain the frequency domain structural features of each first sub-image block.

[0088] Furthermore, the health monitoring system performs a Fourier transform on each first sub-image block. The Fourier transform can convert the image from the spatial domain to the frequency domain, revealing the distribution of different frequency components in the image. In the frequency domain, low-frequency components correspond to the overall outline and large-scale structural information of the image, and high-frequency components correspond to detailed information of the image, such as edges and texture changes. The system analyzes the transformed frequency domain data and extracts features such as the energy distribution in different frequency bands, the location and intensity of the main frequency components, so as to obtain the structural features of each first sub-image block in the frequency domain, supplement the structural information of the image from another dimension, and help to more comprehensively identify the key structural features in the image.

[0089] In one embodiment, after Fourier transforming a first sub-image block of 200×200 pixels, the health monitoring system obtains its frequency domain image. In the frequency domain image, analysis shows that the energy of the low-frequency area (corresponding to the overall structure of the image) accounts for a relatively high proportion of about 70%, which is concentrated near the center of the image; the energy of the high-frequency area (corresponding to the image details) accounts for 30%, which is distributed at the edge of the image. Further analysis of the high-frequency area shows that there are energy peaks at specific frequency positions (such as a horizontal frequency of 0.2 cycles / pixel and a vertical frequency of 0.15 cycles / pixel), indicating that there may be texture or structural changes of specific directions and frequencies in the sub-image block, such as the manifestation of a regular pattern or fine cracks on the surface of the bridge in the frequency domain.

[0090] Step 406 , extracting features based on the updated local features, texture pattern features and frequency domain structural features of each first sub-image block to obtain key structural features.

[0091] Furthermore, the health monitoring system performs feature extraction based on the updated local features, texture pattern features, and frequency domain structure features of each first sub-image block to obtain key structural features, as specifically described in steps 4061 to 4064.

[0092] The embodiment of the present invention divides the image into sub-image blocks and analyzes their features from multiple dimensions, so that the system can capture subtle changes in the structure of the cross-sea bridge in great detail. The updated local features take into account the correlation between image regions, the texture pattern features reveal the regularities and anomalies of the structural surface, and the frequency domain structure features supplement information from the frequency dimension. The comprehensive use of these multi-dimensional features greatly improves the recognition accuracy of key structural features such as crack edges, structural deformation contours, and displacement of structural joints, providing a solid data foundation for accurately assessing the structural health of the cross-sea bridge, helping to promptly discover potential safety hazards and ensuring the safe operation of the cross-sea bridge.

[0093] In one embodiment, the description of steps 4061 to 4064 is as follows:

[0094] Step 4061 : for the target inspection image of each inspection area, each first sub-image block is screened based on the updated local features, texture pattern features and frequency domain structure features to obtain a target sub-image block.

[0095] Optionally, the health monitoring system selects the first sub-image blocks in the target inspection image of each inspection area based on the updated local features, texture pattern features and frequency domain structure features. The system pre-sets a series of feature thresholds and screening rules. For example, for the updated local features, if the average brightness is within a certain abnormal range and the brightness standard deviation exceeds a certain value, it may indicate that the sub-image block is abnormal; for the texture pattern features, when the contrast is higher than a certain value and the energy is lower than a certain threshold, it may indicate that the texture has a mutation; in the frequency domain structure features, if the energy of a specific high frequency band exceeds the normal range, etc., the first sub-image blocks that meet these abnormal feature conditions will be screened out as target sub-image blocks, because they are more likely to contain information related to key structural features.

[0096] In one embodiment, in a target inspection image of a certain inspection area, after a first sub-image block is updated, the local features show an average brightness of 50 (far lower than the normal range of 80-150), and the brightness standard deviation is 35 (higher than the normal standard deviation of 20); the contrast in the texture pattern feature reaches 50 (normal is generally 20-30), and the energy is 0.02 (lower than the normal 0.05); the energy proportion of the high frequency band (0.3-0.5 cycles / pixel) in the frequency domain structure feature reaches 40% (normal is 20%-30%). The health monitoring system determines that the sub-image block meets the abnormal feature conditions based on the pre-set rules and selects it as the target sub-image block. After analyzing and screening all the first sub-image blocks in the inspection area, a total of 50 target sub-image blocks are obtained.

[0097] Step 4062: Fuse the updated local features, texture pattern features, and frequency domain structure features of each target sub-image block to obtain a fused feature vector.

[0098] Furthermore, for each selected target sub-image block, the health monitoring system fuses its updated local features (such as statistics such as average brightness and brightness standard deviation), texture pattern features (such as parameters such as contrast and correlation) and frequency domain structural features (such as data such as energy distribution in different frequency bands). By arranging and combining these different types of feature data in a certain order, a multi-dimensional feature vector is formed. Each dimension corresponds to a specific feature value. This fused feature vector comprehensively reflects the information of the target sub-image block in different feature dimensions, providing a richer data basis for further analysis of structural feature boundary points.

[0099] In one embodiment, taking a target sub-image block as an example, its updated local features are: average brightness 50, brightness standard deviation 35; texture pattern features are: contrast 50, correlation 0.6, energy 0.02, entropy 3.0; frequency domain structure features are: low-frequency energy accounts for 60%, high-frequency energy accounts for 40% (the energy peak is 0.15 in the 0.3-0.5 cycle / pixel frequency band). The health monitoring system combines these feature data into a fused feature vector [50, 35, 50, 0.6, 0.02, 3.0, 0.6, 0.4, 0.15]. In this way, the multi-dimensional feature information of the target sub-image block is integrated into a vector for subsequent processing. The system performs a similar fusion operation on each target sub-image block to obtain the corresponding fused feature vector.

[0100] Step 4063, based on the fused feature vector of each target sub-image block, the difference change rate between adjacent pixels is calculated to determine the structural feature boundary points of each target sub-image block. The structural feature boundary points include crack edge boundary points, structural deformation contour boundary points and structural connection boundary points.

[0101] Furthermore, the health monitoring system calculates the difference change rate between adjacent pixels based on the fused feature vector of each target sub-image block. In the multi-dimensional feature space represented by the fused feature vector, the change in the feature value of adjacent pixels reflects the change in the image structure characteristics. By setting an appropriate difference change rate threshold, when the difference change rate between adjacent pixels exceeds the threshold, the corresponding pixel is determined as a structural feature boundary point. These boundary points include crack edge boundary points (when the feature change conforms to the crack edge feature pattern), structural deformation contour boundary points (corresponding to feature changes related to structural deformation) and structural connection boundary points (related to feature changes at structural connections). In this way, the boundary position of key structural features can be accurately located from the feature data of the target sub-image block.

[0102] In one embodiment, in the fused feature vector corresponding to a target sub-image block, the health monitoring system calculates the difference change rate of adjacent pixels in each feature dimension. For example, in the average brightness dimension, the average brightness values ​​of a pair of adjacent pixels are 48 and 55, respectively, and the difference change rate is calculated as (55-48) / 48≈0.146. After calculating the difference change rate of all adjacent pixels in multi-dimensional features and comparing them with a preset threshold (such as 0.1), it is found that the difference change rate of some pixels in multiple feature dimensions exceeds the threshold, and the feature change pattern of these pixels matches the preset crack edge feature pattern, so the system determines these pixels as crack edge boundary points. In this target sub-image block, a total of 10 crack edge boundary points are determined. Similarly, for other target sub-image blocks, the system also performs similar calculations to determine various types of structural feature boundary points.

[0103] Step 4064: Integrate the structural feature boundary points of each target sub-image block to obtain key structural features of the target inspection image of each inspection area. The key structural features include crack edge features, structural deformation contour features, and structural joint displacement features.

[0104] Furthermore, the health monitoring system integrates the structural feature boundary points of each target sub-image block. For the target inspection image of each inspection area, the crack edge boundary points determined by all target sub-image blocks are connected to form a rough outline of the crack edge, thereby obtaining the crack edge features, including information such as the length and direction of the crack; the structural deformation contour boundary points are integrated to outline the approximate shape of the structural deformation, and the structural deformation contour features are obtained; the boundary points of the structural connection are sorted out to determine the abnormal position and displacement direction of the structural connection, and the displacement features of the structural connection are obtained. Through this integration operation, the local structural feature boundary point information scattered in each target sub-image block is summarized into complete information that can reflect the key structural features of the target inspection image of the entire inspection area.

[0105] In one embodiment, in a target inspection image of a certain inspection area, 100 crack edge boundary points are determined by analyzing 50 target sub-image blocks. The health monitoring system connects and fits these boundary points according to the spatial position, and finds that these points constitute a crack edge contour with a length of about 8 cm and a 45° angle, thereby obtaining the crack edge characteristics of the area. For the structural deformation contour feature, by integrating 60 structural deformation contour boundary points, a local structural depression area with an area of ​​about 5 square centimeters is outlined, and the approximate range and shape of the structural deformation are determined. In terms of the displacement feature of the structural connection, the information of 30 structural connection boundary points is integrated, and it is found that a certain structural connection has a horizontal displacement of 2 mm, which clarifies the abnormal situation of the structural connection. By integrating the structural feature boundary points of all target sub-image blocks, the key structural features of the target inspection image of the inspection area are fully obtained.

[0106] The embodiment of the present invention selects the target sub-image blocks to focus on the areas that may contain key information, reducing unnecessary calculations; integrates the feature vector to integrate multi-dimensional information, and improves the comprehensiveness of feature description; determines the structural feature boundary points to accurately locate the edge position of the key structure; integrates the boundary points to obtain complete and intuitive key structural features, such as cracks, deformations, and displacement of structural joints. The overall solution enables the health monitoring system to clearly grasp the structural health status of each inspection area of ​​the cross-sea bridge, providing strong support for the timely discovery of potential safety hazards, and greatly improving the accuracy and effectiveness of the health monitoring of the cross-sea bridge.

[0107] In one embodiment, the description of steps 501 to 503 is as follows:

[0108] Step 501 , associating the key structural features with the actual positions of the cross-sea bridge to obtain a mapping relationship.

[0109] Optionally, the health monitoring system has a detailed cross-sea bridge structure file, which contains the precise location of each actual part of the bridge, structural design parameters and corresponding standard structural feature descriptions. The system will extract key structural features from the target inspection image, such as crack edge features, structural deformation contour features and structural joint displacement features, and compare and match them with the information in the file. Through coordinate positioning, structural geometry comparison and other methods, the corresponding positions of these key structural features in the actual parts of the bridge are found, thereby establishing a mapping relationship between the two.

[0110] In one embodiment, during the inspection of a cross-sea bridge, a crack edge feature is extracted from the target inspection image, and its coordinate information is (x1, y1)-(x2, y2) in the image coordinate system. The health monitoring system converts the image coordinates into the actual geographic coordinates of the bridge based on the flight trajectory and shooting angle information of the drone during image acquisition. By comparing with the coordinate range of each part in the bridge structure archive, it is found that the actual position corresponding to the crack edge feature is located at the bottom of the bridge body of the third span on the left side of the main bridge, 5 meters away from the bridge pier. The system records this mapping relationship: the crack edge feature ((x1, y1)-(x2, y2)) corresponds to the bottom of the bridge body of the third span on the left side of the main bridge (geographic coordinate range: [a1, b1]-[a2, b2]). Similarly, for the structural deformation contour features and the displacement features at the structural joints, a mapping relationship is established in a similar manner.

[0111] Step 502: determine the abnormal part associated with the key structural feature based on the mapping relationship.

[0112] Furthermore, based on the mapping relationship established in the previous step, the health monitoring system searches for abnormal parts that are associated with each key structural feature in the actual parts of the cross-sea bridge. The system compares the actual structural features of the abnormal parts with the corresponding key structural features and calculates the degree of difference between the two. The calculation of the degree of difference covers many aspects, such as the difference in crack length and width, the difference in shape and size of structural deformation, the difference in distance and direction of displacement at the structural connection, etc. When these differences are less than the preset difference threshold, the actual part is determined as an abnormal part. The preset difference threshold is determined based on the design standards, safety specifications and past monitoring experience of the bridge, and is used to determine whether the changes in structural characteristics exceed the normal range.

[0113] In one embodiment, for the previously determined crack edge features at the bottom of the third span on the left side of the main bridge, the health monitoring system consults the bridge structure archives and finds that cracks should not appear in this part under normal circumstances. The actual detected crack length is 8 cm and the width is 1.5 mm. The preset crack difference threshold is: a length exceeding 5 cm and a width exceeding 1 mm are considered abnormal. Since the length and width of the crack exceed the preset threshold, the part at the bottom of the third span on the left side of the main bridge is determined to be an abnormal part. For another example, for a structural deformation contour feature, the corresponding actual part is the connection between a pier of the approach bridge and the bridge body. Under normal circumstances, the connection should be kept flat, and the structural deformation difference threshold is set to a displacement of no more than 3 mm. After measurement, the structural deformation of the actual connection caused the displacement to reach 4 mm, which exceeded the preset threshold, so the connection was also determined to be an abnormal part.

[0114] Step 503: Perform an abnormality analysis based on the abnormal parts and generate a health monitoring report for the cross-sea bridge.

[0115] Furthermore, the health monitoring system conducts a comprehensive abnormality analysis on the determined abnormal parts. The analysis includes the detailed location of the abnormal parts (based on the geographic coordinates in the mapping relationship), the abnormality type (cracks, structural deformation, displacement of structural joints, etc.) and the severity assessment. The severity assessment is determined based on the degree of difference between the structural characteristics of the abnormal parts and the standard characteristics, combined with the importance of the bridge structure and the impact on the overall structural stability. For example, for cracks, the longer the length and the wider the width, the higher the severity; for structural deformation, the larger the deformation range and the greater the impact on the structural bearing capacity, the higher the severity. The system organizes these analysis results into a standardized report format, which includes the location mark of the abnormal parts (with a schematic diagram), the description of the abnormality type, the severity rating and the corresponding recommended measures, such as regular observation for minor cracks and immediate reinforcement for severe structural deformation, and finally generates a health monitoring report for the cross-sea bridge, as described in steps 5031 to 5034.

[0116] The embodiment of the present invention establishes a mapping relationship so that the key structural features correspond accurately to the actual parts, providing a basis for locating abnormalities; determining the abnormal parts clarifies the specific location and type of the problems in the bridge structure; generating a health monitoring report comprehensively analyzes the abnormal situation and gives targeted suggestions. This report can help bridge management and maintenance personnel quickly understand the structural health of the bridge, timely discover potential safety hazards, reasonably arrange maintenance resources, and take effective maintenance measures to ensure the safe and stable operation of the cross-sea bridge.

[0117] In one embodiment, the description of steps 5031 to 5034 is as follows:

[0118] Step 5031, determining the location of the abnormal part based on the coordinate information of the key structural features of the abnormal part.

[0119] Optionally, the health monitoring system obtains the coordinate information corresponding to the key structural features of the abnormal part. During the image acquisition stage, the drone records information such as the shooting position and angle, through which the coordinates of the key structural features in the image can be converted into the actual geographical coordinates of the cross-sea bridge. Based on the pre-established coordinate conversion model, the system maps the pixel coordinates of the key structural features in the image to the geographical coordinate system of the bridge, thereby accurately determining the actual position of the abnormal part on the bridge, including the bridge section, specific orientation, etc. In one embodiment, a crack edge feature was found in the monitoring of a cross-sea bridge. Its starting pixel coordinates in the target inspection image are (x1, y1), and the ending pixel coordinates are (x2, y2). The health monitoring system converts the pixel coordinates into geographical coordinates based on the flight trajectory, posture, and camera parameters of the drone during shooting, using the coordinate conversion formula. After calculation, the crack is located on the upper surface of the bridge body 30 meters away from the starting pier of the fifth span on the right side of the main bridge. The system records the geographical coordinate information and clarifies the specific location of the abnormal part.

[0120] Step 5032: Determine the type of the abnormal part based on the morphological information and characteristic parameters of the key structural features of the abnormal part.

[0121] Furthermore, the health monitoring system analyzes the morphological information and characteristic parameters of the key structural features of the abnormal parts. For cracks, the morphological information includes the direction of the crack (straight line, curve, etc.), whether it is continuous, etc., and characteristic parameters such as crack length and width, etc.; for structural deformation, the morphological information includes the shape of the deformation (concave, convex, etc.), characteristic parameters such as the depth and range of the deformation, etc.; for displacement of the structural joint, the characteristic parameters are the distance and direction of the displacement, etc. Based on this information and the typical characteristics of various types of abnormalities, it is determined whether the abnormal part belongs to the type of crack, structural deformation or displacement of the structural joint.

[0122] In one embodiment, at an abnormal part, the key structural feature presents a continuous, slightly curved line-like shape. The line is measured to be 10 cm long and 2 mm wide. The health monitoring system compares these morphological information and characteristic parameters with the pre-set crack feature template and finds that they meet the typical characteristics of cracks, so the type of the abnormal part is determined to be a crack. For another example, another abnormal part presents a morphology of obvious depression in the local area, with a measured depression depth of 5 cm and a range diameter of about 20 cm. By comparing with the characteristic template of structural deformation, the system determines that the type of the abnormal part is structural deformation.

[0123] Step 5033, based on the environmental factors of the environment in which the cross-sea bridge is located and the key characteristic parameters corresponding to the type of abnormal part, a severity assessment is performed to obtain the severity of the abnormal part.

[0124] Furthermore, the health monitoring system considers environmental factors of the environment in which the cross-sea bridge is located, such as wind speed, humidity, temperature, traffic flow, etc., and combines the key characteristic parameters corresponding to the type of abnormal part to assess the severity. For cracks, if they are in a strong wind environment, longer and wider cracks may be aggravated by the wind and expand, and the severity will increase accordingly; for structural deformation, in a high humidity environment, the deformed parts may accelerate corrosion, affecting the bearing capacity of the structure, and the severity will increase. The system gives the severity rating of the abnormal parts, such as mild, moderate, and severe, based on the pre-established severity assessment criteria, comprehensive environmental factors and key characteristic parameters.

[0125] In one embodiment, a crack abnormal part has a length of 8 cm and a width of 1.5 mm. The environmental factors in the area are: average wind speed 5 m / s, humidity 60%, temperature 25°C, and normal traffic flow. The health monitoring system consults the severity assessment standard. For such cracks, under the current environment, the length exceeds 6 cm and the width exceeds 1 mm, which is judged to be of moderate severity. Another example is an abnormal part of structural deformation, with a deformation depth of 4 cm and a range diameter of 15 cm. The area is at the seaside, with high humidity all year round and strong sea breeze. According to the assessment standard, considering the adverse effects of high humidity and strong wind environment on the durability and stability of the structural deformation part, combined with the deformation characteristic parameters, the severity of the abnormal part is assessed as severe.

[0126] Step 5034 generates a health monitoring report for the cross-sea bridge based on the abnormal parts and their locations, types and severity.

[0127] Furthermore, the health monitoring system integrates the identified abnormal parts and their corresponding locations, types and severity. The report is presented in a clear format, usually in a combination of tables and text. The table lists in detail the location of the abnormal parts (precise geographic coordinates and description of the bridge section, etc.), type (cracks, structural deformation, etc.), and severity (mild, moderate, severe). The text part summarizes the abnormal situation and gives corresponding suggestions based on the severity. For example, regular review is recommended for mild abnormalities, professional inspections are required for moderate abnormalities, and emergency measures such as reinforcement are taken immediately for severe abnormalities to generate a complete health monitoring report for cross-sea bridges.

[0128] In one embodiment, the health monitoring report generated by the health monitoring system includes the following table:

[0129]

[0130] Summary of the text: Two abnormal areas were found during this monitoring. The crack on the 5th span on the right side of the main bridge is recommended to be inspected by professionals in the near future and rechecked after one month. The structural deformation of the bridge pier below the connection between the 2nd and 3rd spans of the approach bridge is serious. It is necessary to immediately stop the passage of heavy vehicles near the area and organize an expert team to develop a reinforcement and repair plan.

[0131] The embodiment of the present invention accurately determines the location of abnormal parts so that maintenance personnel can quickly locate the problem; accurately determining the type of abnormal parts helps to analyze the cause of the problem in a targeted manner; and scientifically assessing the severity makes resource allocation and treatment measures more reasonable. The integrated report provides comprehensive and intuitive structural health information for bridge management and maintenance departments, making it easy to discover potential risks in a timely manner, make maintenance decisions efficiently, and effectively ensure the safe operation of cross-sea bridges.

[0132] See also Figure 3 , Figure 3 FIG. 1 is an embodiment diagram of an electronic device provided by an embodiment of the present invention. Figure 3 As shown, an embodiment of the present invention provides an electronic device 300, including a memory 310, a processor 320, and a computer program 311 stored in the memory 310 and executable on the processor 320. When the processor 320 executes the computer program 311, the following steps are implemented:

[0133] Based on the structural characteristics and structural parameters of the cross-sea bridge to be monitored and the current weather information, the target inspection route of the UAV is planned through the geographic information system;

[0134] Controlling the UAV to collect images of the surface, structural joints and piers of the cross-sea bridge using visual shooting equipment according to the target inspection route to obtain an initial inspection image;

[0135] Based on an image feature recognition algorithm, invalid images containing light reflections and light shadows are removed from the initial inspection image to obtain a target inspection image;

[0136] Extracting features from the target inspection image to obtain key structural features;

[0137] A health monitoring report of the cross-sea bridge is generated based on the key structural features; the health monitoring report includes the corresponding abnormal parts of the cross-sea bridge and the location, type and severity of the abnormal parts.

[0138] See also Figure 4 , Figure 4 Detailed description of an embodiment of a computer-readable storage medium provided by an embodiment of the present invention. Figure 4As shown, this embodiment provides a computer-readable storage medium 400, on which a computer program 311 is stored. When the computer program 311 is executed by a processor, the following steps are implemented:

[0139] Based on the structural characteristics and structural parameters of the cross-sea bridge to be monitored and the current weather information, the target inspection route of the UAV is planned through the geographic information system;

[0140] Controlling the UAV to collect images of the surface, structural joints and piers of the cross-sea bridge using visual shooting equipment according to the target inspection route to obtain an initial inspection image;

[0141] Based on an image feature recognition algorithm, invalid images containing light reflections and light shadows are removed from the initial inspection image to obtain a target inspection image;

[0142] Extracting features from the target inspection image to obtain key structural features;

[0143] A health monitoring report of the cross-sea bridge is generated based on the key structural features; the health monitoring report includes the corresponding abnormal parts of the cross-sea bridge and the location, type and severity of the abnormal parts.

[0144] On the other hand, the present invention further provides a computer program product, the computer program product includes a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the health monitoring method of the cross-sea bridge provided by the above methods, the health monitoring method of the cross-sea bridge includes:

[0145] Based on the structural characteristics and structural parameters of the cross-sea bridge to be monitored and the current weather information, the target inspection route of the UAV is planned through the geographic information system;

[0146] Controlling the UAV to collect images of the surface, structural joints and piers of the cross-sea bridge using visual shooting equipment according to the target inspection route to obtain an initial inspection image;

[0147] Based on an image feature recognition algorithm, invalid images containing light reflections and light shadows are removed from the initial inspection image to obtain a target inspection image;

[0148] Extracting features from the target inspection image to obtain key structural features;

[0149] A health monitoring report of the cross-sea bridge is generated based on the key structural features; the health monitoring report includes the corresponding abnormal parts of the cross-sea bridge and the location, type and severity of the abnormal parts.

[0150] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Those of ordinary skill in the art may understand and implement it without creative work.

[0151] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0152] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A cross-sea bridge health monitoring system, characterized in that: It includes a monitoring and control center, an inspection route planning unit, a drone control unit, an image processing unit, a structural feature extraction unit, and a health monitoring unit; the monitoring and control center is connected to the inspection route planning unit, the drone control unit, the image processing unit, the structural feature extraction unit, and the health monitoring unit to manage each unit; The inspection route planning unit is used to plan the target inspection route of the UAV through the geographic information system based on the structural characteristics and structural parameters of the cross-sea bridge to be monitored and the weather information at the current time; A UAV control unit is used to control the UAV to collect images of the surface, structural joints and piers of the cross-sea bridge using a visual shooting device according to the target inspection route to obtain an initial inspection image; An image processing unit, used for removing invalid images containing light reflections and light shadows in the initial inspection image based on an image feature recognition algorithm, to obtain a target inspection image; A structural feature extraction unit, used to extract features from the target inspection image to obtain key structural features; A health monitoring unit is used to generate a health monitoring report of the cross-sea bridge based on the key structural features; the health monitoring report includes the corresponding abnormal parts of the cross-sea bridge and the location, type and severity of the abnormal parts.

2. A method for monitoring the health of a cross-sea bridge, implemented based on the health monitoring system for a cross-sea bridge as claimed in claim 1, characterized in that: The cross-sea bridge health monitoring method comprises: Based on the structural characteristics and structural parameters of the cross-sea bridge to be monitored and the current weather information, the target inspection route of the UAV is planned through the geographic information system; Controlling the UAV to collect images of the surface, structural joints and piers of the cross-sea bridge using visual shooting equipment according to the target inspection route to obtain an initial inspection image; Based on an image feature recognition algorithm, invalid images containing light reflections and light shadows are removed from the initial inspection image to obtain a target inspection image; Extracting features from the target inspection image to obtain key structural features; A health monitoring report of the cross-sea bridge is generated based on the key structural features; the health monitoring report includes the corresponding abnormal parts of the cross-sea bridge and the location, type and severity of the abnormal parts.

3. The method for monitoring the health of a cross-sea bridge according to claim 2, characterized in that: The target inspection route of the UAV is planned through the geographic information system based on the structural characteristics and structural parameters of the cross-sea bridge to be monitored and the weather information at the current time, including: Based on the three-dimensional modeling function of the geographic information system and the structural parameters, constructing a three-dimensional model of the cross-sea bridge in the geographic information system; Dividing the three-dimensional model based on the structural characteristics of the cross-sea bridge to obtain different inspection areas in the three-dimensional model; With the goal of covering the entire area and taking images with a clarity greater than a preset threshold, a route is planned for each inspection area to obtain a preliminary inspection route for each inspection area by the drone; Based on the geographic information system and the weather information, the preliminary inspection route of each inspection area is optimized to obtain the target inspection route of the drone for each inspection area.

4. The method for monitoring the health of a cross-sea bridge according to claim 3, characterized in that: The optimization of the preliminary inspection route of each inspection area based on the geographic information system combined with the weather information to obtain the target inspection route of the drone for each inspection area includes: Based on the weather information, a preliminary inspection route of each inspection area is optimized for the first time to obtain a first optimized inspection route; Calculate the estimated flight time required for the drone to cruise each inspection area according to the flight speed of the drone and the first optimized inspection route of each inspection area; Determine the total cruising time of the cross-sea bridge based on the estimated flight time of each inspection area, and perform a second optimization on the first optimized inspection route of each inspection area with the total flight time of the UAV as a constraint to obtain a second optimized inspection route of each inspection area; Based on the spatial analysis function of the geographic information system, collision detection is performed on the second optimized inspection route of each inspection area with the goal of no collision points, so as to obtain the target inspection route of the drone for each inspection area.

5. The method for monitoring the health of a cross-sea bridge according to claim 2, characterized in that: The feature extraction of the target inspection image to obtain key structural features includes: For each inspection area, the target inspection image is divided into a plurality of sub-image blocks; Determining a first initial local feature of each first sub-image block based on the pixel brightness of each pixel in each first sub-image block; Based on the initial local feature of each first sub-image block and the second initial local feature of the second sub-image block adjacent to the first sub-image block, the initial local feature of each first sub-image block is updated to obtain an updated local feature of each first sub-image block; Performing structural texture analysis based on an arrangement pattern of pixels in each first sub-image block to obtain a texture pattern feature of each first sub-image block; Performing Fourier transform on each first sub-image block to convert each first sub-image block from the spatial domain to the frequency domain, thereby obtaining a frequency domain structural feature of each first sub-image block; Feature extraction is performed based on the updated local features, texture pattern features and frequency domain structural features of each first sub-image block to obtain the key structural features.

6. The method for monitoring the health of a cross-sea bridge according to claim 5, characterized in that: The step of extracting features based on the updated local features, texture pattern features and frequency domain structural features of each first sub-image block to obtain the key structural features includes: For the target inspection image of each inspection area, each first sub-image block is screened based on the updated local features, texture pattern features, and frequency domain structure features to obtain a target sub-image block; The updated local features, texture pattern features and frequency domain structure features of each target sub-image block are fused to obtain a fused feature vector; Based on the fused feature vector of each target sub-image block, the difference change rate between adjacent pixels is calculated to determine the structural feature boundary points of each target sub-image block; the structural feature boundary points include crack edge boundary points, structural deformation contour boundary points and structural connection boundary points; The structural feature boundary points of each target sub-image block are integrated to obtain the key structural features of the target inspection image of each inspection area; the key structural features include crack edge features, structural deformation contour features and structural connection displacement features.

7. The method for monitoring the health of a cross-sea bridge according to any one of claims 2 to 6, characterized in that: The generating of the health monitoring report of the cross-sea bridge based on the key structural features includes: Associating the key structural features with the actual positions of the cross-sea bridge to obtain a mapping relationship; Determine an abnormal part associated with the key structural feature based on the mapping relationship; the difference between the structural feature of the abnormal part and the key structural feature is less than a preset difference threshold; An abnormality analysis is performed based on the abnormal parts to generate a health monitoring report of the cross-sea bridge.

8. The method for monitoring the health of a cross-sea bridge according to claim 7, characterized in that: The abnormality analysis based on the abnormal part is performed to generate a health monitoring report of the cross-sea bridge, including: Determine the position of the abnormal part based on the coordinate information of the key structural features of the abnormal part; Determining the type of the abnormal part based on morphological information and characteristic parameters of key structural features of the abnormal part; Performing a severity assessment based on environmental factors of the environment in which the cross-sea bridge is located and key characteristic parameters corresponding to the type of the abnormal part to obtain the severity of the abnormal part; A health monitoring report of the cross-sea bridge is generated based on the abnormal parts and the locations, types and severity of the abnormal parts.

9. An electronic device, comprising: A memory and a processor, characterized in that a computer software program is stored in the memory, and when the processor reads and executes the computer software program, the health monitoring method for a cross-sea bridge as described in any one of claims 2 to 8 is implemented.

10. A non-transitory computer-readable storage medium, characterized in that: The storage medium stores a computer software program, and when the computer software program is executed by the processor, the method for monitoring the health of a cross-sea bridge according to any one of claims 2 to 8 is implemented.

Images