A bridge and tunnel inspection, maintenance and management system and method based on 3D panoramic view and GIS map

Abstract

The present invention discloses a bridge and tunnel inspection, maintenance and management system and method based on three-dimensional panoramic and GIS maps, which relates to the field of bridge and tunnel inspection and maintenance. The method includes: processing and fusing the collected multi-dimensional environmental data of bridges and tunnels to obtain preliminary three-dimensional map data, performing super-resolution enhancement on the turbidity shadow area to construct a high-precision three-dimensional digital model, and fusing the model with GIS data to obtain a digital twin data body; extracting microscopic spectral features, performing spectral power authenticity discrimination, and generating a health status time series; planning the autonomous inspection path of the inspection machine, and introducing the quantum annealing algorithm to optimize the scheduling to solve the optimal path; constructing a knowledge graph to associate historical maintenance records with an expert experience base, and mining implicit association rules to generate maintenance suggestions. By integrating multi-source data, intelligent analysis and optimization technologies, high-precision inspection, accurate damage assessment and intelligent maintenance decision-making are realized, and the intelligent level of bridge and tunnel management is improved.

Description

Technical Field

[0001] The present invention relates to the field of bridge and tunnel inspection and maintenance, and specifically to a bridge and tunnel inspection and maintenance management system and method based on three-dimensional panorama and GIS map. Background Art

[0002] As important components of modern transportation infrastructure, the structural safety of bridges and tunnels is directly related to public safety and the stable operation of the transportation network. Traditional bridge and tunnel inspection and maintenance mainly rely on manual visual inspection and regular maintenance, which have problems such as low efficiency, limited coverage, and strong subjectivity. In recent years, with the application of technologies such as three-dimensional modeling, geographic information system, and artificial intelligence, bridge and tunnel management has gradually developed towards digitalization and intelligence. However, there are still many deficiencies in the existing technologies, such as: insufficient three-dimensional modeling accuracy, lack of dynamics and objectivity in health status assessment, low efficiency in inspection path planning, insufficient intelligence level in maintenance decision-making, weak data security and traceability, etc.

[0003] In response to the above problems, the existing technologies have tried to optimize through single technology improvement, but have failed to systematically solve the coordination problems of data quality, dynamic analysis, path planning, and knowledge mining. Therefore, there is an urgent need for a bridge and tunnel maintenance management method integrating high-precision three-dimensional modeling, dynamic health assessment, intelligent path optimization, and knowledge graph-driven to improve the automation and intelligence level of the whole process. Summary of the Invention

[0004] Based on the above-mentioned disadvantages of the existing technologies, the purpose of the present invention is to provide a bridge and tunnel inspection and maintenance management system and method based on three-dimensional panorama and GIS map to solve the above technical problems.

[0005] To achieve the above purpose, the present invention provides the following technical solution: A bridge and tunnel inspection and maintenance management method based on three-dimensional panorama and GIS map, including:

[0006] Processing and fusing the collected multi-dimensional environmental data of bridges and tunnels to obtain preliminary three-dimensional map data, performing super-resolution enhancement on the turbid shadow area to construct a high-precision three-dimensional digital model, and fusing the model with GIS data to obtain a digital twin data body;

[0007] Extracting microscopic spectral features, performing spectral power authenticity discrimination, and generating a health status time series;

[0008] Planning the autonomous inspection path of the inspection machine, and introducing the quantum annealing algorithm to optimize the scheduling to solve the optimal path;

[0009] Constructing a knowledge graph to associate historical maintenance records with an expert experience library, and mining implicit association rules to generate maintenance suggestions.

[0010] The present invention is further set as, the method includes:

[0011] Collect multi-dimensional data using drones, laser scanners, and panoramic cameras to form bridge and tunnel multi-dimensional environmental data;

[0012] Use dynamic SLAM to fuse the collected bridge and tunnel multi-dimensional environmental data to generate preliminary three-dimensional map data;

[0013] Screen the preliminary three-dimensional map according to the turbidity function to obtain the turbid shadow area;

[0014] Use a graph neural network to enhance the super-resolution of the turbid shadow area data, replace the original data to obtain complete high-quality data, and then use the Poisson surface reconstruction algorithm to perform surface fitting on the complete high-quality data to obtain a high-precision three-dimensional model;

[0015] Fuse the high-precision three-dimensional model with GIS to achieve digital twins and obtain a digital twin data body.

[0016] The present invention is further set as follows: the construction logic of the turbidity function: extract the local topological inconsistency and spectral phase aliasing index for a specified area, and use the local topological inconsistency and spectral phase aliasing index to obtain the turbidity function;

[0017] The construction logic of the spectral phase aliasing index: perform a two-dimensional Fourier transform on the spatial domain image to obtain spectral data, calculate the phase spectrum based on the spectral data, perform a gradient calculation on the phase spectrum to obtain the phase gradient, and define the weighted ratio of the phase gradient to the spectral amplitude as the spectral phase aliasing index;

[0018] The calculation logic of the turbidity function: , where, is the turbidity function, is the local topological inconsistency, is the spectral phase aliasing index;

[0019] The calculation logic of the local topological inconsistency: , where, is the local topological inconsistency of the current data point, is the current data point, is the neighbor set of the current data point, is the number of neighbor points, is the sum of the number of neighbor points of all neighbor points;

[0020] The calculation logic of the spectral phase aliasing index: , where, is the spectral phase aliasing index, is the spectral amplitude, is the phase spectrum, is the phase spectrum gradient;

[0021] Spectrum data calculation logic: , where is the spectrum data, is the number of rows of the input data, is the number of columns of the input data, is the intensity value of the input data, is the frequency domain coordinate, is the imaginary unit, is the angular frequency factor of the Fourier transform.

[0022] The present invention is further set as the blurred shadow area screening logic: , where is the blurred shadow area, is the current data point, is the set threshold;

[0023] Digital twin data body calculation logic: , where is the digital twin data body, is the fusion mapping operator, is the high-precision three-dimensional model, is the geographic information data of GIS.

[0024] The present invention is further set to specifically include:

[0025] Using a two-stream Transformer network architecture for the digital twin data body to fuse image and point cloud features to locate the damaged area;

[0026] Performing a Fourier transform on the damaged area to obtain the damaged area spectrum data;

[0027] Calculating the power spectral density of the damaged area spectrum data, constructing a microscopic spectrum feature vector set, and discriminating the authenticity of the damaged area based on the obtained microscopic spectrum feature vector set;

[0028] Predicting the health trend based on the discrimination result in combination with the damaged area and the spectrum data;

[0029] Damaged area spectrum data calculation logic: , where is the damaged area spectrum data, is the damaged area, is the intensity of the pixel points in the damaged area, is the frequency domain coordinate, is the imaginary unit representing the complex nature of the Fourier transform, is the angular frequency scaling factor;

[0030] Power spectral density calculation logic: , where is the power spectral density, Spectrum data for the damaged area

[0031] Extract key information from the power spectral density and microscopic spectral features to form a feature vector set: , where is the microscopic spectral feature vector set, is the real part mean, is the imaginary part mean, is the average power spectral density, is the standard deviation of the power spectrum;

[0032] Spectrum power authenticity discrimination logic: , is the classifier, , is the weight matrix, , is the bias vector, is the activation function, The output activation function, the discrimination logic is: , where is the discrimination result, indicates true damage, indicates false detection.

[0033] The present invention is further configured as a health trend calculation logic: , where the health trend calculation logic: , where is the current health status, is the health status at the next moment, is the health transfer coefficient, is the feature mapping of the damage information, is the damaged area, is the element-wise product, is the element-wise addition; the complete health status time series is obtained by recursive calculation according to the health trend calculation formula .

[0034] The present invention is further configured as, the method includes:

[0035] Generate a candidate path set for the damaged area predicted to be less than the final health status threshold using the A* search algorithm;

[0036] Screen and optimize the candidate path set using the quantum annealing scheduling operator to obtain the optimal inspection path;

[0037] Transmit the optimal inspection path to the inspection system to independently deploy inspection machines, and record data to generate an inspection record after the inspection is completed;

[0038] Candidate path calculation logic: , where is a candidate path, is a search algorithm, is the starting point, is the target point, is the weighted graph of the environmental map, and a set of candidate paths is constructed ;

[0039] Optimal inspection path calculation logic: , where is the optimal inspection path, is the candidate path, is the quantum annealing scheduling operator, and the calculation logic is: , where is the total number of path nodes, is the adjustment parameter, is the candidate path at the local cost of the node, Inspection record: , where is the timestamp, is the device information.

[0040] The present invention is further configured such that the method includes:

[0041] Extracting information from historical maintenance records and expert experience to construct a knowledge graph:

[0042] Using a graph neural network to mine implicit rules from the constructed knowledge graph to obtain a rule set;

[0043] Combining the health status with the rule set to generate maintenance suggestions;

[0044] Maintenance suggestion generation logic: , where is the maintenance suggestion, is the most matching rule, and the calculation logic is: , is the final health status, is the weight, is the rule set, where is the rule the optimal health status value applicable to, is the associated maintenance suggestion; is the applicability calculation function, and the calculation logic is: , where is the smoothing parameter.

[0045] The present invention is further configured such that the method further includes the preservation and traceability of bridge and tunnel data: calculating a hash value for the constructed bridge and tunnel state set, and combining blockchain storage to ensure data traceability and immutability;

[0046] Constructing the bridge and tunnel state set: , where is the bridge and tunnel state set, is the digital twin data body, is the inspection record, is the health status time series, is the maintenance suggestion; Hash value calculation logic: , where is the hash value, is the hash value calculation function.

[0047] The present invention also provides a bridge and tunnel inspection and maintenance management system based on 3D panorama and GIS map, and the system includes:

[0048] Data acquisition and model construction module: Processing and fusing the collected multi-dimensional environmental data of the bridge and tunnel to obtain preliminary 3D map data, performing super-resolution enhancement on the blurred area to construct a high-precision 3D digital model, and fusing the model with GIS data to obtain a digital twin data body;

[0049] Damage detection and health trend prediction module: Extracting microscopic spectral features, performing authenticity discrimination on spectral power, and generating a health status time series;

[0050] Inspection path construction and optimization module: Planning the autonomous inspection path of the inspection machine, and introducing the quantum annealing algorithm to optimize the scheduling to solve the optimal path;

[0051] Maintenance suggestion generation module: Constructing a knowledge graph to associate historical maintenance records with an expert experience library, and mining implicit association rules to generate maintenance suggestions.

[0052] The present invention provides a bridge and tunnel inspection and maintenance management system and method based on 3D panorama and GIS map. The method processes and fuses the collected multi-dimensional environmental data of the bridge and tunnel to obtain preliminary 3D map data, performs super-resolution enhancement on the blurred area to construct a high-precision 3D digital model, and fuses the model with GIS data to obtain a digital twin data body; extracts microscopic spectral features, performs authenticity discrimination on spectral power, and generates a health status time series; plans the autonomous inspection path of the inspection machine, introduces the quantum annealing algorithm to optimize the scheduling to solve the optimal path; constructs a knowledge graph to associate historical maintenance records with an expert experience library, and mines implicit association rules to generate maintenance suggestions. The beneficial effects generated include:

[0053] High-precision 3D digital modeling to improve the accuracy of bridge and tunnel inspections: Multidimensional environmental data of bridges and tunnels are obtained through multi-source sensors such as drones, laser scanners, and panoramic cameras, and data fusion is carried out using dynamic SLAM technology to construct a preliminary 3D map, improving the integrity and real-time nature of spatial information. The local topological inconsistency analysis and spectral phase aliasing index calculation methods are adopted to accurately identify the turbid shadow area, and super-resolution reconstruction is carried out using a generative adversarial network to enhance data accuracy, and finally a high-precision 3D digital model is generated. Combining with GIS geographic information data, the construction of a digital twin of bridges and tunnels is realized, providing intuitive and accurate visual support for inspections and maintenance, and enhancing the intelligent level of bridge and tunnel management.

[0054] Spectrum analysis and health assessment to improve the accuracy of damage detection: The Fourier transform is used to extract microscopic spectral features, and combined with power spectral density analysis, a spectral power authenticity discrimination model is constructed to accurately identify the true state of structural damage, reducing the false detection and missed detection rates. Through damage profile extraction and microscopic spectral feature analysis, a health state time series is established, and the health trend is calculated using a recurrence formula to realize the long-term health monitoring and prediction of bridge and tunnel structures, providing a scientific and reasonable early warning mechanism, and effectively avoiding potential safety hazards caused by sudden damage.

[0055] Knowledge graph-driven intelligent maintenance suggestions to improve the scientific nature of maintenance decisions: The knowledge graph construction method is adopted to integrate historical maintenance records and expert experience databases, mine the implicit association rules between bridge and tunnel damage and maintenance measures, and construct an intelligent maintenance knowledge base. Through graph neural network analysis of health state data, the optimal maintenance strategy is automatically matched, and combined with the long-term health trend of bridge and tunnel structures, targeted maintenance suggestions are provided, enhancing the scientific nature and accuracy of maintenance decisions. The maintenance strategy has the ability of self-learning and can be continuously optimized with the accumulation of data, improving the intelligent level of inspections and maintenance and reducing the need for manual intervention.

[0056] The above description is only an overview of the technical solution of this application. In order to be able to understand the technical means of this application more clearly, it can be implemented according to the content of the specification. And in order to make the above and other purposes, features and advantages of this application more obvious and understandable, the following specifically gives the specific implementation manners of this application. Brief Description of the Drawings

[0057] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following will briefly introduce the drawings required for the description of the embodiments. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings. In the drawings:

[0058] Figure 1Flowchart of a bridge and tunnel inspection, maintenance and management method based on 3D panorama and GIS map shown in an exemplary embodiment of the present invention;

[0059] Figure 2 Schematic structural diagram of a bridge and tunnel inspection, maintenance and management system based on 3D panorama and GIS map shown in an exemplary embodiment of the present invention. Detailed implementation mode

[0060] The following will describe the embodiments of the present invention with reference to the accompanying drawings and preferred embodiments. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation modes. Various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be understood that the preferred embodiments are only for explaining the present invention and not for limiting the protection scope of the present invention.

[0061] It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention in a schematic manner. Therefore, only the components related to the present invention are shown in the diagrams, rather than being drawn according to the number, shape and size of the components in actual implementation. The type, quantity and proportion of each component in its actual implementation can be arbitrarily changed, and the component layout type may also be more complex.

[0062] In the following description, a large number of details are explored to provide a more thorough explanation of the embodiments of the present invention. However, it is obvious to those skilled in the art that the embodiments of the present invention can be implemented without these specific details. In other embodiments, well-known structures and devices are shown in the form of block diagrams rather than in detail to avoid making the embodiments of the present invention difficult to understand.

[0063] Embodiment 1

[0064] A bridge and tunnel inspection, maintenance and management method based on 3D panorama and GIS map, as Figure 1 shown, includes:

[0065] Process and fuse the collected multi-dimensional environmental data of bridges and tunnels to obtain preliminary 3D map data, perform super-resolution enhancement on the turbid shadow area to construct a high-precision 3D digital model, and fuse the model with GIS data to obtain a digital twin data body;

[0066] Extract microscopic spectral features, perform spectral power authenticity discrimination, and generate a health status time series;

[0067] Plan the autonomous inspection path of the inspection machine, and introduce the quantum annealing algorithm to optimize the scheduling to solve the optimal path;

[0068] Build a knowledge graph to associate historical maintenance records with expert experience base, and mine implicit association rules to generate maintenance suggestions.

[0069] The present invention is further configured that the method comprises:

[0070] Use drones, laser scanners, and panoramic cameras to collect multi-dimensional data and build multi-dimensional environmental data of bridges and tunnels;

[0071] Use dynamic SLAM to fuse the collected multi-dimensional environmental data of bridges and tunnels to generate preliminary three-dimensional map data;

[0072] The preliminary three-dimensional map is screened according to the turbidity function to obtain the turbidity area;

[0073] The graph neural network is used to perform super-resolution enhancement on the turbidity area data, replacing the original data to obtain complete high-quality data, and then the Poisson surface reconstruction algorithm is used to perform surface fitting on the complete high-quality data to obtain a high-precision three-dimensional model;

[0074] The high-precision three-dimensional model is integrated with GIS to realize the digital twin and obtain the digital twin data body. Specifically, the inspection machine such as drone is used to collect information on the bridge and tunnel. The drone mainly collects local RGB image data, infrared image data, and low-precision point cloud data, among which: the local RGB image data can be used to observe the damaged area, range, and type, the infrared image data can be used to observe the temperature field distribution caused by the uneven temperature caused by hollowing and leakage, and the low-precision point cloud data can be used to observe the large volume and contour of the target or help to roughly model the vegetation obstruction area; the laser scanner mainly collects high-precision three-dimensional point cloud data and reflection intensity data, among which: the high-precision three-dimensional point cloud data can provide collection information including sub-millimeter precision point coordinates, and the reflection intensity can help to identify the difference in material properties and the rusted area; the panoramic camera mainly collects 360° spherical image data and partial depth information, which is mainly used for blind-angle visual recording and relative position marking of diseases. It is an existing technology to use dynamic SLAM to fuse multidimensional data and generate three-dimensional map data. It is a common technical means used by technicians in this field and will not be repeated here.

[0075] The present invention is further configured as follows: the turbidity function construction logic: extracting local topological inconsistency and spectrum phase aliasing index for a specified area, and obtaining a turbidity function using the local topological inconsistency and the spectrum phase aliasing index;

[0076] The logic of constructing the spectrum phase aliasing index is as follows: perform two-dimensional Fourier transform on the spatial domain image to obtain spectrum data, calculate the phase spectrum based on the spectrum data, calculate the gradient of the phase spectrum to obtain the phase gradient, and define the weighted ratio of the phase gradient to the spectrum amplitude as the spectrum phase aliasing index;

[0077] Turbidity function calculation logic: , where is the turbidity function, is the local topological inconsistency, is the spectral phase aliasing index;

[0078] Local topological inconsistency calculation logic: , where is the local topological inconsistency of the current data point, is the current data point, is the neighbor set of the current data point, is the number of neighbor points, is the sum of the number of neighbor points of all neighbor points;

[0079] Spectral phase aliasing index calculation logic: , where is the spectral phase aliasing index, is the spectral amplitude, is the phase spectrum, is the phase spectrum gradient;

[0080] Spectral data calculation logic: , where is the spectral data, is the number of rows of the input data, is the number of columns of the input data, is the intensity value of the input data, is the frequency domain coordinate, is the imaginary unit, is the angular frequency factor of the Fourier transform. Specifically, the turbidity function combines the local topological inconsistency and the spectral phase aliasing index to distinguish high-quality data from the turbidity shadow area. An S-shaped function is used to ensure smooth determination of data quality, making the turbidity between . Among them, the local topological inconsistency captures the geometric anomalies of the laser point cloud by summing the number of neighbor points of all neighbor points. The spectral phase aliasing index judges the state of the image by observing the phase. The larger the value, the more it indicates that the phase information of the image is disturbed, and it may be the turbidity shadow area. For example, motion blur in the image will cause high-frequency phase chaos, and water stains or shadows will cause low-frequency phase distortion. These fluctuations will cause drastic changes in the phase, so a higher value will be obtained. It is constrained by an S-shaped function to keep the final data range within . In the formula of local topological inconsistency, the neighbor set of the current pixel point Represents a pixel point The set of adjacent points around, the number of neighbor points Represents the connectivity of a pixel point in the local topological structure, the number of neighbors of the neighbor points of the pixel point Represents an adjacent point The local topological complexity of; this formula measures the ratio of the number of neighbors of a pixel point to the sum of the number of neighbors of all its neighbor points. The larger the ratio, the more abnormal the topological structure of the point, and it is more likely to be low-quality data; the phase spectrum in the spectral phase aliasing index Represents the phase information in the spectral data, which is related to the geometric structure of the image and is obtained by calculating the arctangent of the spectral data. The specific calculation formula is: , where the and Are the real part and the imaginary part of the spectrum respectively, the phase spectrum gradient Is used to measure the rate of change of the phase in the frequency domain and is approximately differentiated by the Sobel operator or the finite difference method. The specific calculation formula is: , which is used to detect the sharpness of the image. The phase change of a clear image is smoother, and the phase gradient of a blurred image is more intense. The spectral amplitude Is the absolute value of the spectral data, representing the energy intensity of each frequency component. Its calculation formula is: , which is used to normalize the phase gradient to ensure that The index can not only reflect the phase change trend but also not be affected by the overall energy size. If the amplitude is large, it means that the frequency component contributes more to the image and the influence of aliasing may be small. If the amplitude is small, it means that the frequency component contributes less. However, if the phase gradient is large, it may be an artifact or data anomaly. The spectral data Is obtained by performing a Fourier transform on the image or signal and is a complex matrix containing the real part and the imaginary part , representing the energy distribution of the signal at different frequencies, which is prior art and will not be elaborated here.

[0081] The present invention is further set as the shadow area screening logic: , where Is the shadow area Is the current data point Is the set threshold;

[0082] Digital twin data body calculation logic: , where Is the digital twin data body Is the fusion mapping operator Is the high-precision three-dimensional model Geographic information data for GIS. Specifically, the screening of the turbidity shadow area is carried out by calculating the turbidity function for the regional image. If the turbidity value of the area is less than the set threshold judge that the data point belongs to a low-quality image and classify it into the turbidity shadow area. The threshold usually takes a value in , and is adjusted according to experience; the data fusion of the digital twin data body belongs to the prior art and is a common technical means for those skilled in the art, so it will not be elaborated here.

[0083] The present invention is further set to specifically include:

[0084] Use a two-stream Transformer network architecture for the digital twin data body to fuse image and point cloud features to locate the damaged area;

[0085] Perform a Fourier transform on the damaged area to obtain the spectral data of the damaged area;

[0086] Calculate the power spectral density of the spectral data of the damaged area, construct a set of microscopic spectral feature vectors, and perform authenticity discrimination on the damaged area based on the obtained set of microscopic spectral feature vectors;

[0087] Predict the health trend based on the discrimination result in combination with the damaged area and the spectral data;

[0088] Calculation logic of the spectral data of the damaged area: , where is the spectral data of the damaged area, is the damaged area, is the intensity of the pixel points in the damaged area, is the frequency domain coordinate, is the imaginary unit representing the complex nature of the Fourier transform, is the angular frequency scaling factor;

[0089] Calculation logic of the power spectral density: , where is the power spectral density, is the spectral data of the damaged area;

[0090] Extract key information from the power spectral density and microscopic spectral features to form a set of feature vectors: , where is the set of microscopic spectral feature vectors, is the real part mean value, is the imaginary part mean value, is the average power spectral density, is the power spectral standard deviation;

[0091] Spectral power authenticity discrimination logic: , is the classifier, , is the weight matrix, , is the bias vector, is the activation function, The output activation function, and the discrimination logic is: , where is the discrimination result, has real damage, is a false detection. Specifically, the spectral data of the damaged area is the frequency-domain data obtained by performing a two-dimensional Fourier transform on the damaged area, which is prior art and will not be elaborated here; the power spectral density is used to measure the power distribution of the signal at different frequencies and reflects the energy characteristics of the signal; the real part mean represents the average value of the real part of the obtained spectral data in this area, reflecting the energy distribution of the low-frequency components in the damaged area, which is related to the overall shape of the damage. A higher value indicates better structural integrity, while a lower value means more serious damage; the imaginary part mean represents the average value of the imaginary part of the obtained spectral data in this area, reflecting the energy distribution of the high-frequency components in the damaged area, which is related to the edge characteristics and texture details. A larger value indicates that there are more high-frequency details in the damaged area, such as sharper edges of cracks. If the value is too low, it means that the damaged area lacks high-frequency information, indicating a blurred damage boundary; the average power spectral density is the average value of the power spectrum, representing the total energy intensity of the signal. The average power spectral density being high indicates that the signal energy in the damaged area is strong, the crack is deeper or the boundary is clear. The average power spectral density being low indicates that the energy in the damaged area is low, the crack is shallower or the boundary is blurred; the power spectrum standard deviation is used to measure the degree of dispersion of the spectral energy in the damaged area and quantify the uniformity of the energy distribution. The specific calculation formula is: , where is the size of the frequency-domain matrix. A higher value indicates a more uneven tiled energy distribution, a higher complexity of the damaged area, and diverse damage forms; in the discrimination of spectral power authenticity is the first-layer weight matrix used to linearly transform and adjust the influence degree of the input features to prevent gradient vanishing or explosion. The value is usually between or , is the first-layer bias term used to help the neural network fit the non-linear relationship and prevent over-reliance on the weight matrix , and the initial value is usually set as a small random number and optimized through training. is the second-layer weight matrix used to perform weighted calculation on the output of the first layer to obtain the final classification score. The initial value is usually set as a small random number and optimized through training. The bias vector Used to adjust the classification decision boundary, with an initial value usually of 0 and adjusted during training; the output activation function is the Sigmoid function, used to compress the output value to between.

[0092] The present invention is further configured as follows for the health trend calculation logic: , where is the current health state, is the health state at the next moment, is the health transfer coefficient, is the feature mapping of the damage information, is the precise damage profile, is the element-wise product, is the element-wise addition; the complete health state time series is obtained by recursive calculation according to the formula . Specifically, the current health state represents the health condition of the system at time t, represents complete health, represents complete damage, and the health transfer coefficient is used to measure the influence degree of the past health condition on the future. The closer the value is to the smaller the influence on the stable health state; the feature mapping of the damage information represents the influence of the damage on the health state, usually normalized to , representing the influence degree of the damage, and the activation function is the Sigmoid function, used to compress the output value to between; the health state at the next moment is generated by recursive calculation to generate the complete health state time series.

[0093] The present invention is further configured as follows: for the damage area predicted to be less than the final health state threshold, the A* search algorithm is used to generate a set of candidate paths;

[0094] The quantum annealing scheduling operator is used to screen and optimize the set of candidate paths to obtain the optimal inspection path;

[0095] The optimal inspection path is transmitted to the inspection system to independently allocate inspection machines, and the data is recorded after the inspection is completed to generate an inspection record;

[0096] Candidate path calculation logic: , where is the candidate path, is the search algorithm, is the starting point, is the target point, is the weighted graph of the environmental map, and a set of candidate paths is constructed ;

[0097] Optimal inspection path calculation logic: , where is the optimal inspection path, is the candidate path, is the quantum annealing scheduling operator, and the calculation logic is: , where is the total number of path nodes, is the adjustment parameter, is the candidate path at the local cost of the node, inspection record: , where is the timestamp, is the device information. Specifically, the candidate path is the inspection path calculated from the starting point to the target point, and is used as an optional solution for the inspection robot. The weighted graph of the environmental map is used to calculate the path cost, , is the inspection point in the environment, is the passage path between nodes, is the path cost, such as distance, obstacle weight; the quantum annealing scheduling operator is used to optimize the path selection, calculate the global cost of the path, and select the path with the minimum cost as the optimal inspection path. The local cost includes: path length cost, energy consumption cost, safety cost, time cost, environmental adaptability, and the adjustment parameter determines the influence degree of the local cost on the total cost, and the value range is in .0, and the value is adjusted according to different scenarios.

[0098] The present invention is further configured to extract information from historical maintenance records and expert experience to construct a knowledge graph:

[0099] Use a graph neural network to perform implicit rule mining on the constructed knowledge graph to obtain a rule set;

[0100] Generate maintenance suggestions by combining the health status and the rule set;

[0101] Maintenance suggestion generation logic: , where is the maintenance suggestion, is the most matching rule, and the calculation logic is: , is the final health status, is the weight, is the rule set, where is the rule Optimal health state value for application is the associated maintenance recommendation is the applicability calculation function, and the calculation logic is , where is the smoothing parameter. Specifically, the knowledge graph is a structured knowledge representation method that contains entities and the relationships between entities. The nodes of the constructed knowledge graph include: bridge and tunnel components, damage types, maintenance measures, and environmental factors; among them, the bridge and tunnel component nodes include attributes: component type, casting material, design life, manufacturing year, affiliated bridge and tunnel, current health state; the damage type nodes include attributes: damage type, damage length, damage width, damage location, discovery time; the maintenance measure nodes include attributes: maintenance method, maintenance cost, validity period, construction conditions; the environmental factor nodes include attributes: environmental type, influence degree, influence range; using graph neural networks for implicit rule mining is an existing technology and a common technical means for those skilled in the art, so it will not be elaborated here; the rule set contains all the rules mined by using graph neural networks. Each rule has an associated health state and corresponding maintenance recommendation. By matching the predicted final health state with the health states in the rule set, the most matching one is selected, and the maintenance recommendation corresponding to this rule is fed back; the weight represents the importance of the rule, and the value range is usually , where represents the most important represents the least important, and the smoothing parameter represents controlling the influence of the health state difference on the applicability, and the common values are , and the smaller the value, the more sensitive the matching degree

[0102] The present invention is further set such that the method further includes the storage and traceability of bridge and tunnel data: calculating the hash value of the constructed bridge and tunnel state set, and combining blockchain storage to ensure data traceability and immutability

[0103] Constructing the bridge and tunnel state set , where is the bridge and tunnel state set is the digital twin data body is the inspection record is the health state time series is the maintenance recommendation; the hash value calculation logic , where is the hash value It is a hash value calculation function. Specifically, the hash value is a unique identifier with a fixed length, representing the uniqueness of the data. Whenever the bridge-tunnel status set is updated, its hash value is calculated, and the hash value and its corresponding data are stored in the blockchain. Each new data point will form a new block and be added to the blockchain. Once the data enters the blockchain, it cannot be tampered with or deleted, ensuring the plasticity and credibility of the data.

[0104] Embodiment 2

[0105] Please refer to Figure 2 , the exemplary bridge-tunnel inspection, maintenance and management system based on 3D panoramic view and GIS map includes:

[0106] Data acquisition and model construction module: Process and fuse the collected multi-dimensional environmental data of the bridge-tunnel to obtain preliminary 3D map data, perform super-resolution enhancement on the blurred area to construct a high-precision 3D digital model, and fuse the model with GIS data to obtain a digital twin data body;

[0107] Damage detection and health trend prediction module: Extract microscopic spectral features, perform authenticity discrimination on spectral power, and generate a health status time series;

[0108] Inspection path construction and optimization module: Plan the autonomous inspection path of the inspection machine, and introduce the quantum annealing algorithm to optimize the scheduling to solve the optimal path;

[0109] Maintenance suggestion generation module: Construct a knowledge graph to associate historical maintenance records with the expert experience library, and mine implicit association rules to generate maintenance suggestions.

[0110] It should be noted that the bridge-tunnel inspection, maintenance and management system based on 3D panoramic view and GIS map provided in the above embodiment belongs to the same concept as the bridge-tunnel inspection, maintenance and management method based on 3D panoramic view and GIS map provided in the above embodiment. The specific ways in which each module and unit perform operations have been described in detail in the method embodiment, and will not be repeated here. The bridge-tunnel inspection, maintenance and management system based on 3D panoramic view and GIS map provided in the above embodiment can, in actual application, allocate the above functions to different functional modules as needed, that is, divide the internal structure of the system into different functional modules to complete all or part of the functions described above. This is not limited here either.

[0111] The above embodiments can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, the above embodiments can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the processes or functions described in the embodiments of the present application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center by wired (such as infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more collections of available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, or a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium. The semiconductor medium can be a solid-state drive.

[0112] It should be understood that the term "and / or" in this document is merely a description of the association relationship between associated objects, indicating that there can be three relationships. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. Here, A and B can be singular or plural. In addition, the character " / " in this document generally represents an "or" relationship between the associated objects before and after, but it may also represent an "and / or" relationship, which can be specifically understood with reference to the context before and after.

[0113] In this application, "at least one" means one or more, and "a plurality" means two or more. "At least one of the following" or its similar expressions refer to any combination of these items, including any combination of single items or plural items. For example, at least one of a, b, or c can represent: a, b, c, a - b, a - c, b - c, or a - b - c, where a, b, and c can be single or multiple.

[0114] It should be understood that in various embodiments of the present application, the magnitudes of the sequence numbers of the above processes do not mean the order of execution is prior or posterior. The order of execution of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application.

[0115] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented in electronic hardware, or in a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of this application.

[0116] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be repeated herein.

[0117] In the several embodiments provided in this application, it should be understood that the disclosed systems can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of devices or units can be electrical, mechanical, or other forms.

[0118] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they can be located in one place, or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0119] In addition, the functional units in each embodiment of this application can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0120] When the above-mentioned functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art or part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in various embodiments of this application. The aforementioned storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0121] As described above, the above is only the specific implementation manner of this application, but the protection scope of this application is not limited thereto. Any person skilled in the art within the technical scope disclosed by this application can easily think of changes or substitutions, and all should be covered within the protection scope of this application. Therefore, the protection scope of this application should be subject to the protection scope of the claimed rights.

Claims

1. A bridge and tunnel inspection, maintenance and management method based on 3D panorama and GIS map, characterized in that Including: Processing and fusing the collected multi-dimensional environmental data of bridges and tunnels to obtain preliminary three-dimensional map data, performing super-resolution enhancement on the turbidity shadow area to construct a high-precision three-dimensional digital model, and fusing the model with GIS data to obtain a digital twin data body, including: screening the preliminary three-dimensional map according to the turbidity function to obtain the turbidity shadow area; using a graph neural network to perform super-resolution enhancement on the turbidity shadow area data, replacing the original data to obtain complete high-quality data, and then using the Poisson surface reconstruction algorithm to perform surface fitting on the complete high-quality data to obtain a high-precision three-dimensional model; fusing the high-precision three-dimensional model with GIS to achieve digital twin to obtain the digital twin data body; turbidity function construction logic: extracting local topological inconsistency and spectral phase aliasing index for a specified area, and obtaining the turbidity function using the local topological inconsistency and spectral phase aliasing index; Spectral phase aliasing index construction logic: performing two-dimensional Fourier transform on the spatial domain image to obtain spectral data, calculating the phase spectrum according to the spectral data, performing gradient calculation on the phase spectrum to obtain the phase gradient, and defining the weighted ratio of the phase gradient to the spectral amplitude as the spectral phase aliasing index; Turbidity function calculation logic: , where is the turbidity function, is the local topological inconsistency, is the spectral phase aliasing index; Local topological inconsistency calculation logic: , where is the local topological inconsistency of the current data point, is the current data point, is the neighbor set of the current data point, is the number of neighbor points, is the sum of the number of neighbor points of all neighbor points; Spectrum phase aliasing index calculation logic: , where is the spectrum phase aliasing index, is the spectrum amplitude, is the phase spectrum, is the phase spectrum gradient; Spectrum data calculation logic: , where is the spectrum data, is the number of rows of the input data, is the number of columns of the input data, is the intensity value of the input data, is the frequency domain coordinate, is the imaginary unit, is the angular frequency factor of the Fourier transform; Shadow region screening logic: , where is the shadow region, is the current data point, is the set threshold; Digital twin data body calculation logic: , where is the digital twin data body, is the fusion mapping operator, is the high-precision three-dimensional model, is the geographic information data of GIS; Extracting microscopic spectral features, performing spectral power authenticity discrimination, and generating a health status time series; Planning the autonomous inspection path of the inspection machine, and introducing a quantum annealing algorithm to optimize the scheduling to solve the optimal path; Constructing a knowledge graph to associate historical maintenance records with an expert experience library, and mining implicit association rules to generate maintenance suggestions.

2. The bridge and tunnel inspection and maintenance management method based on 3D panorama and GIS map according to claim 1, characterized in that The method includes: Using drones, laser scanners, and panoramic cameras to collect multi-dimensional data and form multi-dimensional environmental data of bridges and tunnels; Using dynamic SLAM to perform data fusion on the collected multi-dimensional environmental data of bridges and tunnels to generate preliminary three-dimensional map data.

3. The bridge and tunnel inspection, maintenance and management method based on 3D panorama and GIS map according to claim 1, characterized in that Specifically including: Using a two-stream Transformer network architecture to fuse image and point cloud features on the digital twin data body to locate the damage area; Performing Fourier transform on the damage area to obtain the spectral data of the damage area; Calculating the power spectral density of the spectral data of the damage area, constructing a set of microscopic spectral feature vectors, and performing authenticity discrimination on the damage area based on the obtained set of microscopic spectral feature vectors; Predicting the health trend based on the discrimination result in combination with the damage area and spectral data; Calculation logic for spectral data of the damaged area: , where is the spectral data of the damaged area, is the damaged area, is the intensity of the pixel points within the damaged area, is the frequency domain coordinate, is the imaginary unit representing the complex nature of the Fourier transform, is the angular frequency scaling factor; Power spectral density calculation logic: , where is the power spectral density, is the spectral data of the damaged area; Extract key information from the power spectral density and the spectral data of the damaged area to form a feature vector set: , where is the feature vector set of microscopic spectral features, is the mean value of the real part, is the mean value of the imaginary part, is the average power spectral density, is the standard deviation of the power spectrum; Spectrum power authenticity discrimination logic: , is a classifier, , is a weight matrix, , is a bias vector, is an activation function, The output activation function, the discrimination logic is: , where, is the discrimination result, indicates having a real damage, is a false detection.

4. A bridge and tunnel inspection and maintenance management method based on 3D panoramic view and GIS map according to claim 3, characterized in that, Health trend calculation logic: , where is the current health status, is the health status at the next moment, is the health transfer coefficient, is the feature mapping of damage information, is the damaged area, is the element-wise product, is the element-wise addition; the complete health status time series is obtained by recursion according to the health trend calculation formula .

5. A bridge and tunnel inspection and maintenance management method based on 3D panoramic and GIS maps according to claim 1, characterized in that The method includes: Using the A* search algorithm to generate a candidate path set for the predicted damage area smaller than the final health status threshold; Using the quantum annealing scheduling operator to screen and optimize the candidate path set to obtain the optimal inspection path; Transmitting the optimal inspection path to the inspection system to autonomously deploy the inspection machine, and recording data to generate an inspection record after the inspection is completed; Candidate path calculation logic: , where is the candidate path, is the search algorithm, is the starting point, is the target point, is the weighted graph of the environmental map, constructing a candidate path set ; Optimal inspection path calculation logic: , where is the optimal inspection path, is the candidate path, is the quantum annealing scheduling operator, and the calculation logic is: , where is the total number of path nodes, is the adjustment parameter, is the candidate path at the local cost of the node, inspection record: , where is the timestamp, is the device information.

6. The bridge and tunnel inspection, maintenance and management method based on 3D panorama and GIS map according to claim 1, characterized in that The method includes: Extracting information from historical maintenance records and expert experience to construct a knowledge graph: Using a graph neural network to perform implicit rule mining on the constructed knowledge graph to obtain a rule set; Generating maintenance suggestions in combination with the health status and the rule set; Maintenance recommendation generation logic: , where is the maintenance recommendation, is the most matching rule, and the calculation logic is: , is the final health status, is the weight, is the rule set, where is the rule the optimal health status value applicable to, is the associated maintenance recommendation; is the applicability calculation function, and the calculation logic is: , where is the smoothing parameter.

7. A bridge and tunnel inspection, maintenance and management method based on 3D panoramic view and GIS map according to claim 1, characterized in that, The method also includes the preservation and traceability of bridge and tunnel data: calculating the hash value of the constructed bridge and tunnel state set, and combining blockchain storage to ensure data traceability and immutability; Construct the bridge-tunnel status set: , where is the bridge-tunnel status set, is the digital twin data body, is the inspection record, is the health status time series, is the maintenance suggestion; Hash value calculation logic: , where is the hash value, is the hash value calculation function.

8. A bridge and tunnel inspection, maintenance and management system based on 3D panorama and GIS map, which is used to implement the method for bridge and tunnel inspection, maintenance and management based on 3D panorama and GIS map according to any one of claims 1-7, characterized in that, Including: Data Acquisition and Model Building Module: Process and fuse the collected multi-dimensional environmental data of bridges and tunnels to obtain preliminary three-dimensional map data, perform super-resolution enhancement on the turbid shadow area to build a high-precision three-dimensional digital model, and fuse the model with GIS data to obtain a digital twin data body; Damage Detection and Health Trend Prediction Module: Extract microscopic spectral features, perform authenticity discrimination on spectral power, and generate a time series of health status; Inspection Route Construction and Optimization Module: Plan the autonomous inspection route of the inspection machine, and introduce the quantum annealing algorithm to optimize the scheduling to solve the optimal route; Maintenance Suggestion Generation Module: Build a knowledge graph to associate historical maintenance records with the expert experience library, and mine implicit association rules to generate maintenance suggestions.

Images