Multimodal perception-based automatic inspection path optimization method and system for flood discharge tunnel

Abstract

This application discloses a method and system for optimizing the automatic inspection path of a flood discharge tunnel using multimodal sensing, relating to the field of path planning technology. The method includes: acquiring multi-source intrinsic data of the target tunnel and constructing a twin simulation model of the target tunnel; training and acquiring a barrier mechanism model for predicting barrier risk sections and an inspection prediction model for predicting target inspection sections; acquiring the barrier risk sections and target inspection sections for the current inspection cycle; configuring optimization targets for global path planning to generate an automatic inspection path; and controlling the inspection equipment to execute the automatic inspection path. This solves the problems of existing flood discharge tunnel inspections relying on manual labor or single sensors, lacking global optimization in path planning, and being unable to adapt to complex and changing environments, resulting in low inspection efficiency and inaccurate risk assessment.

Description

Technical Field

[0001] This application relates to the field of path planning technology, specifically to a method and system for optimizing the automatic inspection path of flood discharge tunnels using multimodal sensing. Background Technology

[0002] With the continuous development of water conservancy projects, the scale and number of flood discharge tunnels are increasing, and their safe operation affects the entire water conservancy system. However, traditional flood discharge tunnel inspection methods mainly rely on manual labor, which is not only inefficient but also poses certain safety risks, making it difficult to meet the growing inspection needs.

[0003] Meanwhile, existing automatic inspection technologies cannot fully consider the actual conditions of tunnels and various complex factors, which can easily lead to problems such as unreasonable inspection paths and failure to fully cover important areas, resulting in poor inspection results. Summary of the Invention

[0004] This application provides a method and system for optimizing the automatic inspection path of flood discharge tunnels with multimodal perception, which solves the technical problems of low inspection efficiency and inaccurate risk assessment caused by existing flood discharge tunnel inspections relying on manual labor or a single sensor, lacking global optimization in path planning and being difficult to adapt to complex and changing environments.

[0005] The technical solution to the above-mentioned technical problems in this application is as follows:

[0006] In a first aspect, this application provides a multimodal sensing method for optimizing the automatic inspection path of flood discharge tunnels, the method comprising:

[0007] Acquire multi-source intrinsic data of the target tunnel, and construct a twin simulation model of the target tunnel based on the multi-source intrinsic data;

[0008] Based on the twin simulation model and the historical inspection records of the target tunnel, an obstacle mechanism model for predicting obstacle risk sections and an inspection prediction model for predicting target inspection sections are trained and obtained.

[0009] Multimodal perception data of the target tunnel is collected in real time, and the multimodal perception data is input into the obstacle mechanism model and the inspection prediction model respectively to obtain the obstacle risk section and the target inspection section of the current inspection cycle.

[0010] Based on the target inspection section and the obstacle risk section, configure and optimize the target to perform global path planning and generate an automatic inspection path;

[0011] Control the inspection equipment to execute the automatic inspection path.

[0012] Secondly, this application provides a multimodal sensing automatic inspection path optimization system for flood discharge tunnels, including:

[0013] The information acquisition module is used to acquire multi-source intrinsic data of the target tunnel and construct a twin simulation model of the target tunnel based on the multi-source intrinsic data.

[0014] The model training module is used to train and obtain an obstacle mechanism model for predicting obstacle risk sections and an inspection prediction model for predicting target inspection sections based on the twin simulation model and the historical inspection records of the target tunnel.

[0015] The model prediction module is used to collect multimodal perception data of the target tunnel in real time, and input the multimodal perception data into the obstacle mechanism model and the inspection prediction model respectively to obtain the obstacle risk section and the target inspection section of the current inspection cycle.

[0016] The path planning module is used to configure optimization targets based on the target inspection section and the obstacle risk section to perform global path planning and generate an automatic inspection path;

[0017] The inspection execution module is used to control the inspection equipment to execute the automatic inspection path.

[0018] This application provides one or more technical solutions, which have at least the following technical effects or advantages:

[0019] This application provides a method and system for optimizing the automatic inspection path of flood discharge tunnels using multimodal sensing. First, a twin simulation model of the target tunnel is constructed using multi-source intrinsic data, achieving accurate mapping and simulation of the actual tunnel conditions and providing a reliable foundation for subsequent path planning and risk prediction. Second, based on the twin simulation model and historical inspection records, an obstacle mechanism model and an inspection prediction model are trained, accurately predicting obstacle risk sections and target inspection sections, fully considering the complexity and variability of the tunnel environment. Third, by collecting multimodal sensing data in real time and inputting it into the model, relevant information for the current inspection cycle is obtained, ensuring the timeliness and accuracy of path planning.

[0020] Through the above technical solution, this application overcomes the shortcomings of the prior art, which relies on manual labor or a single sensor, lacks global optimization in path planning, and is difficult to adapt to complex and ever-changing environments, thus providing a strong guarantee for the safe operation of flood discharge tunnels. Attached Figure Description

[0021] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0022] Figure 1This is a flowchart illustrating the multimodal sensing automatic inspection path optimization method for flood discharge tunnels provided in this application embodiment;

[0023] Figure 2 This is a schematic diagram of the structure of the multimodal sensing automatic inspection path optimization system for flood discharge tunnels provided in the embodiments of this application.

[0024] The components represented by each number in the attached diagram are explained below:

[0025] Information acquisition module 11, model training module 12, model prediction module 13, path planning module 14, and inspection execution module 15. Detailed Implementation

[0026] This application provides a method and system for optimizing the automatic inspection path of flood discharge tunnels using multimodal perception. This system addresses the technical problems of low inspection efficiency and inaccurate risk assessment caused by flood discharge tunnel inspections relying on manual labor or single sensors, lacking global optimization in path planning, and being unable to adapt to complex and changing environments.

[0027] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0028] In the description of this application, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "multiple" means two or more, unless otherwise explicitly specified.

[0029] In the description of this application, the term "for example" is used to mean "used as an example, illustration, or description." Any embodiment described as "for example" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is provided to enable any person skilled in the art to make and use this application. Details are set forth in the following description for purposes of explanation. It should be understood that those skilled in the art will recognize that this application can be made without using these specific details. In other instances, well-known structures and processes will not be described in detail to avoid unnecessarily obscuring the description of this application. Therefore, this application is not intended to be limited to the embodiments shown, but is consistent with the broadest scope of the principles and features disclosed in this application.

[0030] Example 1, as Figure 1 As shown in the embodiments of this application, a method for optimizing the automatic inspection path of a flood discharge tunnel based on multimodal perception is provided, including:

[0031] S10: Obtain multi-source intrinsic data of the target tunnel, and construct a twin simulation model of the target tunnel based on the multi-source intrinsic data;

[0032] In this embodiment, multi-source intrinsic data of the target tunnel is first obtained. This multi-source intrinsic data covers the geological structure, water flow characteristics, and building material properties of the target tunnel. The geological structure data reflects the stability of the strata where the tunnel is located, the water flow characteristic data reflects the erosion of the tunnel wall by the water flow, and the building material property data can assess the durability of the tunnel.

[0033] By integrating and analyzing multi-source intrinsic data, a twin simulation model of the target tunnel was constructed using simulation technology. This model can present the physical structure of the tunnel and simulate its operating status under different working conditions.

[0034] In the process of constructing the twin simulation model, big data processing technology and machine learning algorithms are employed. Big data processing technology can handle massive amounts of multi-source intrinsic data, while machine learning algorithms can extract potential patterns and features from the data, thereby improving the accuracy and reliability of the twin simulation model.

[0035] By constructing a twin simulation model of the target tunnel, a foundation is provided for subsequent obstacle risk prediction and inspection path planning, which can more scientifically and accurately guide the automatic inspection of flood discharge tunnels and improve inspection efficiency and safety.

[0036] Specifically, step S10 in the method includes:

[0037] Based on the multi-source intrinsic data, a tunnel topology model characterizing the structural connection relationship of the target tunnel is constructed;

[0038] Based on the tunnel topology model, a twin simulation model of the target tunnel is constructed by combining the multi-source intrinsic data.

[0039] In this embodiment, firstly, based on the structural information in the multi-source intrinsic data, the connection relationships between various parts of the target tunnel are determined, such as different branches and nodes of the tunnel and the connectivity between tunnels, thereby constructing a tunnel topology model. The tunnel topology model presents the overall architecture of the tunnel in a graph structure, showing the spatial relationships inside the tunnel.

[0040] Secondly, based on the tunnel topology model, further information from multi-source intrinsic data, such as geological structure, water flow characteristics, and building material properties, is integrated. By combining multi-source intrinsic data with the topology model, simulation algorithms and techniques are used to simulate the operating state of the target tunnel under various actual working conditions. Simultaneously, the impact of different water flow velocities and pressures on the tunnel structure, as well as potential risks arising from changes in geological conditions, are considered.

[0041] During the construction process, the twin simulation model was optimized and adjusted. The accuracy and reliability of the model were verified through comparative analysis with actual monitoring data. If deviations were found between the model and the actual situation, the model parameters and algorithms were adjusted promptly to ensure that the model could more accurately reflect the true condition of the target tunnel.

[0042] S20: Based on the twin simulation model and the historical inspection records of the target tunnel, train and obtain an obstacle mechanism model for predicting obstacle risk sections and an inspection prediction model for predicting target inspection sections.

[0043] In this embodiment, the twin simulation model is combined with the historical inspection records of the target tunnel to provide rich data for model training. For training the obstacle mechanism model, the obstacle situations appearing in the historical inspection records are correlated with the corresponding working conditions simulated by the twin simulation model.

[0044] For example, when historical records show that a section of a tunnel experiences cracks and obstacles under specific water flow velocity and geological conditions, a twin simulation model can be used to study the stress distribution, structural changes, and other factors of the tunnel under these conditions. Correlated data is used as training samples and input into machine learning algorithms, such as decision tree algorithms and support vector machine algorithms, to continuously adjust model parameters, enabling the model to accurately learn the intrinsic mechanisms and patterns of obstacle occurrence.

[0045] Furthermore, when training the inspection prediction model for predicting target inspection sections, key inspection areas under different time periods and working conditions in historical inspection records are analyzed. Considering factors such as tunnel usage frequency, water flow patterns, and geological stability, sections more prone to problems under specific conditions are prioritized for inspection. This is combined with a twin simulation model to obtain the characteristics of the target inspection sections. Machine learning algorithms are also used for training, allowing the model to predict the target inspection sections within the current inspection cycle based on current working conditions and historical data.

[0046] During training, to ensure the accuracy and generalization ability of the model, the training data is preprocessed. This includes data cleaning to remove erroneous and duplicate data from historical inspection records; and data normalization to ensure that different types of data have the same scale range, thus preventing certain data features from having an excessive impact on the model training results.

[0047] Simultaneously, cross-validation is employed to evaluate and optimize the trained model, ensuring its good performance across different datasets. Through this training process, an obstacle mechanism model and an inspection prediction model are obtained, providing support for subsequent path planning and inspection tasks.

[0048] Specifically, step S20 in the method includes:

[0049] Analyze the historical inspection records to extract historical sensing data and associated historical obstacle risk sections, wherein the historical sensing data and the multimodal sensing data have the same dimension;

[0050] The historical sensing data is input into the twin simulation model for simulation analysis. Based on the simulation analysis results, the sample obstacle risk segment is determined, and a mapping relationship between the sample obstacle risk segment and the historical sensing data is established.

[0051] By comparing the sample obstacle risk segments with the historical obstacle risk segments, the confidence obstacle risk segments are determined, and obstacle training data is constructed by combining the historical perception data with the mapping relationship.

[0052] Based on the obstacle training data, construct and train the obstacle mechanism model based on a graph neural network.

[0053] In this embodiment, firstly, historical sensing data and associated historical obstacle risk sections are extracted from the historical inspection records of the target tunnel. The historical sensing data maintains dimensional consistency with the subsequently collected real-time multimodal sensing data, ensuring data coherence and comparability. The historical sensing data includes geological conditions, water flow pressure, temperature changes, etc., while the historical obstacle risk sections clearly identify specific tunnel areas where potential obstacle risks were discovered during past inspections.

[0054] Secondly, the extracted historical sensing data is input into the twin simulation model of the target tunnel for simulation analysis. Based on the historical sensing data, the twin simulation model simulates the tunnel's operating state under corresponding working conditions. Through the study of the simulation results, sample obstacle risk sections are identified. In this process, the model considers the response of different geological structures to water flow impact, the performance changes of building materials under specific temperatures and pressures, etc., thereby identifying sections that may have obstacle risks. Then, a mapping relationship between sample obstacle risk sections and historical sensing data is established, providing a data foundation for subsequent model training.

[0055] Next, the sample obstacle risk segments are compared with historical obstacle risk segments. During the comparison, the differences and overlaps between the two are analyzed. Regions identified as obstacle risk segments in both the sample and historical records are selected and designated as confidence obstacle risk segments. Confidence obstacle risk segments have high reliability and can provide dependable data support for model training. By combining historical perception data and the established mapping relationships, obstacle training data is constructed. The obstacle training data reflects the inherent relationship between obstacle risk and various perception data.

[0056] Finally, based on the constructed obstacle training data, a barrier mechanism model based on a graph neural network was built and trained. Graph neural networks can handle data with complex topological structures. During training, the obstacle training data was input into the graph neural network, and by adjusting the network parameters, the model learned the intrinsic mechanisms and patterns of obstacle occurrence. Simultaneously, methods such as cross-validation were used to evaluate and optimize the model, ensuring good performance on different datasets, thereby providing accurate obstacle risk prediction for the automated inspection of flood discharge tunnels.

[0057] The process of constructing and training the obstacle mechanism model based on a graph neural network, based on the obstacle training data, includes:

[0058] The nodes and node edges in the configuration graph structure are configured, wherein the nodes are used to represent path environmental elements in the inspection path, and the node edges are used to represent the association relationships between the path environmental elements;

[0059] A graph neural network structure is constructed based on the nodes and node edges, and the graph neural network is trained using the obstacle training data as input data to obtain an initial obstacle mechanism model;

[0060] The initial obstacle mechanism model was validated, and the network parameters were corrected based on the validation results.

[0061] Based on the validated initial obstacle mechanism model, multiple rule conditions are generated and output, and these multiple rule conditions are stored as the obstacle mechanism model.

[0062] In this embodiment, the graph structure is first designed, defining the nodes and node edges. Nodes represent environmental elements along the inspection path, such as different locations of tunnels, geological conditions, and water flow characteristics. Node edges represent the relationships between these environmental elements, such as connectivity between different locations and the mutual influence between geological conditions and water flow characteristics. By configuring nodes and node edges, the complex environmental information along the inspection path is transformed into graph structure data.

[0063] Secondly, a graph neural network structure is constructed based on the configured nodes and node edges. Graph neural networks can handle complex topological data and adapt to the complex and ever-changing environmental information in the inspection path. The constructed obstacle training data is used as input data and fed into the graph neural network for training. During the training process, the graph neural network learns the intrinsic mechanisms and patterns of obstacle occurrence by adjusting its own parameters, thereby obtaining an initial obstacle mechanism model.

[0064] Next, after obtaining the initial obstacle mechanism model, it is validated. The validation process employs methods such as cross-validation, applying the model to different datasets to evaluate its accuracy and generalization ability. Based on the validation results, problems with the model are analyzed, such as unreasonable parameter settings and insufficient ability to handle special cases. To address these issues, the network parameters are adjusted to improve the model's performance.

[0065] Finally, based on the validated initial obstacle mechanism model, multiple rule-based conditional expressions are generated and output. These rule-based conditional expressions are the concrete manifestations of the obstacle occurrence patterns learned by the model, used to characterize the obstacle generation patterns, and can provide a basis for judgment in the automatic inspection of flood discharge tunnels. The rule-based conditional expressions are stored to form the final obstacle mechanism model.

[0066] In actual inspection work, based on the rule-based conditions, obstacle risk sections can be predicted quickly and accurately, providing strong support for the planning and adjustment of inspection paths, thereby improving the efficiency and safety of automatic inspection of flood discharge tunnels.

[0067] Furthermore, based on the twin simulation model and the historical inspection records of the target tunnel, training and acquiring an obstacle mechanism model for predicting obstacle risk sections and an inspection prediction model for predicting target inspection sections, further includes:

[0068] The twin simulation model is used to determine multiple minimum inspection units in the target tunnel;

[0069] Based on the historical inspection records, evaluate the risk assessment parameters corresponding to multiple smallest inspection units;

[0070] Based on the risk assessment parameters, the plurality of minimum inspection units are screened to determine the target inspection unit set;

[0071] An ensemble learning method is used to configure at least one base model for each of the smallest inspection units in the target inspection unit set, and multiple base models are integrated through the ensemble learning method to obtain the inspection prediction model.

[0072] The number of base models configured for each of the minimum inspection units is positively correlated with the corresponding risk assessment parameters.

[0073] In this embodiment, the target tunnel is first divided into multiple minimum inspection units using a twin simulation model. Each minimum inspection unit is a basic component of the tunnel, possessing relatively independent structure and operational characteristics, such as a specific length of the tunnel, an independent branch, or a critical node area.

[0074] Secondly, based on the historical inspection records of the target tunnel, the risk assessment parameters corresponding to each smallest inspection unit are evaluated. These risk assessment parameters cover multiple aspects, such as geological stability risk (if the geological area where a smallest inspection unit is located has unstable factors such as faults and fissures, the geological stability risk is relatively high); water erosion risk (when the unit is subjected to high-speed water flow for a long time, the water erosion risk will increase); and building material aging risk (for smallest inspection units with long service lives or poor building material quality, the building material aging risk will be more prominent). Through comprehensive consideration and quantitative assessment of risk factors, the risk assessment parameters for each smallest inspection unit are derived.

[0075] Then, multiple minimum inspection units are screened based on risk assessment parameters. Minimum inspection units with higher risk assessment parameters are more likely to encounter obstacles and problems, and are included in the target inspection unit set; for minimum inspection units with lower risk assessment parameters, the frequency and intensity of inspections are appropriately reduced to improve inspection efficiency.

[0076] Subsequently, an ensemble learning approach is used to configure at least one base model for each minimum inspection unit in the target inspection unit set. These models are of different types of machine learning models, such as decision tree models, support vector machine models, and neural network models. Furthermore, the number of base models configured for each minimum inspection unit is positively correlated with the corresponding risk assessment parameters. Since high-risk minimum inspection units are more complex, more models are needed to analyze and predict from different perspectives to improve prediction accuracy. That is, the higher the risk assessment parameters of the minimum inspection unit, the more base models are configured.

[0077] The base models of ensemble learning are divided into multiple clusters, each cluster corresponding to a minimum tunneling unit. The number of base models in each cluster is determined based on the weighted result of the statistical risk probability and subjective attention of each minimum tunneling unit. Before establishing the base models, multiple minimum tunneling units are screened, and minimum tunneling units with tunneling risk less than the threshold are removed, that is, tunneling parts that are basically impossible to have problems are removed.

[0078] Finally, multiple base models are integrated using ensemble learning methods. Ensemble learning can employ a voting method, where multiple base models predict the target inspection section, and the final prediction result is determined based on the prediction results of the majority of base models; or a weighted average method, where different weights are assigned to each base model based on its performance, and the prediction results of each base model are then weighted and averaged to obtain the final prediction result.

[0079] Through an integration process, an inspection prediction model is obtained. This model combines the advantages of multiple base models to more accurately predict target inspection sections, providing a basis for automatic inspection path planning of flood discharge tunnels and further improving the efficiency and safety of inspection work.

[0080] S30: Collect multimodal perception data of the target tunnel in real time, and input the multimodal perception data into the obstacle mechanism model and the inspection prediction model respectively to obtain the obstacle risk section and the target inspection section of the current inspection cycle;

[0081] In this embodiment, various sensors installed at different locations within the target tunnel are used to collect multimodal sensing data in real time. These sensors include geological sensors to monitor minute changes in the geological structure surrounding the tunnel; water pressure sensors to measure water pressure in different areas inside the tunnel; and temperature sensors to record fluctuations in the tunnel's ambient temperature. The multimodal sensing data covers information related to the tunnel's operation.

[0082] After collecting multimodal sensing data, it is input into the trained obstacle mechanism model and inspection prediction model respectively. The obstacle mechanism model will analyze and judge each area of ​​the current tunnel based on the input data and combined with the previously learned obstacle occurrence patterns and rule conditions, and predict the sections that may have obstacle risks within the current inspection cycle.

[0083] For example, if multimodal sensing data shows a sudden increase in water flow pressure in a certain section and a slight anomaly in the geological structure, the obstacle mechanism model will determine that there may be an obstacle risk in the area based on the rule-based conditional expression.

[0084] The inspection prediction model comprehensively considers multimodal sensing data and historical inspection records to predict the target inspection sections within the current inspection cycle. By analyzing the correlation between the current operating conditions and sections that are prone to problems in the past, and combining the risk assessment parameters of different minimum inspection units, it determines the areas that need to be prioritized for inspection.

[0085] For example, when multimodal sensing data shows that the water flow scouring of a certain high-risk minimum inspection unit is intensified, the inspection prediction model will identify the section where the unit is located as the target inspection section.

[0086] By inputting multimodal sensing data into the obstacle mechanism model and the inspection prediction model, the obstacle risk section and target inspection section of the current inspection cycle can be quickly obtained, which helps to improve the pertinence and effectiveness of automatic inspection of flood discharge tunnels.

[0087] Specifically, step S30 in the method includes:

[0088] Obtain the tunnel topology model of the target tunnel, and update the tunnel topology model with the obstacle risk section as the taboo space to obtain the timely inspection topology model;

[0089] With the constraint of covering the target inspection section and the optimization objective of minimizing the overall path cost, initialize the path planning rules;

[0090] Based on the greedy algorithm, a first alternative path is generated in the real-time inspection topology model, and the first alternative path is randomly transformed in combination with the constraints to obtain a first alternative path set.

[0091] Based on the first set of candidate paths, the path planning rules, and the timely inspection topology model, global path planning based on multi-objective optimization is performed to obtain the automatic inspection path.

[0092] In this embodiment, firstly, a tunnel topology model of the target tunnel is obtained. This model shows the overall structure of the tunnel, the connection relationships between various areas, and related geographical information. The tunnel topology model is then updated using identified obstacle risk sections as forbidden spaces. Since obstacle risk sections may pose dangers and are unsuitable for inspection equipment to enter, they are excluded from the passable areas, thus obtaining a timely inspection topology model.

[0093] Secondly, using coverage of the target inspection area as a constraint, the final planned path must reach all areas requiring focused inspection to ensure comprehensiveness of the inspection work. Simultaneously, minimizing the overall path cost is the optimization objective, encompassing factors such as inspection equipment uptime, energy consumption, and equipment wear and tear. Based on these two conditions, the path planning rules are initialized.

[0094] Then, a first alternative path is generated in the timely inspection topology model based on a greedy algorithm. The greedy algorithm selects the optimal path for the current state at each step, quickly generating a path that satisfies some of the conditions.

[0095] Then, the first alternative path is randomly transformed in combination with the constraints of the coverage target inspection section. Random transformation can increase the diversity of the path and avoid getting trapped in local optima, thereby obtaining the first alternative path set, which contains multiple different alternative paths.

[0096] Finally, based on the first set of candidate paths, path planning rules, and the real-time inspection topology model, global path planning based on multi-objective optimization is performed. Multi-objective optimization considers multiple conflicting objectives, minimizing the overall path cost while ensuring coverage of the target inspection sections. By further filtering and optimizing the first set of candidate paths, and combining the characteristics of the path planning rules and the real-time inspection topology model, the final automatic inspection path is obtained.

[0097] The final automatic inspection path, while ensuring full coverage of the target inspection section, minimizes the overall cost of the path and improves the efficiency and economy of automatic inspection of the flood discharge tunnel.

[0098] The comprehensive cost of the inspection path includes at least the total length of the inspection path, the total energy consumption of the inspection, and the risks associated with the inspection path.

[0099] In this embodiment, the total inspection path length refers to the total distance traveled by the automatic inspection equipment during the execution of the inspection task, which directly affects the inspection time cost and the degree of equipment wear. A shorter total inspection path length can reduce inspection time, improve inspection efficiency, and also reduce equipment wear and tear, extending the equipment's service life.

[0100] Total energy consumption during inspections mainly refers to the energy consumed by the inspection equipment during operation, such as electricity and fuel. The amount of energy consumed is related to factors such as the length of the inspection path, the operating speed of the equipment, and the load. Reducing total energy consumption during inspections not only saves resources and lowers operating costs but also aligns with the development requirements of energy conservation and environmental protection.

[0101] Inspection route risk refers to the various potential hazards and adverse factors that may be encountered during the inspection process, which can be categorized as average risk or cumulative risk. Examples include obstacle risk sections, geologically unstable areas, and areas with rapid water flow. When planning inspection routes, risk factors should be considered, and high-risk areas should be avoided as much as possible to ensure the safety of inspection equipment and personnel.

[0102] Furthermore, a mathematical model is needed to accurately calculate the overall cost of the inspection path. The total length of the inspection path is obtained by summing the lengths of each segment along the automated inspection path. The calculation of the total energy consumption of the inspection requires comprehensive consideration of parameters such as equipment power, operating time, and load factor. The assessment of the inspection path risk requires combining obstacle mechanism models and historical data to quantitatively analyze the risk level of each area along the path.

[0103] In the actual path planning process, it is necessary to make reasonable trade-offs and optimizations on the total length of the inspection path, the total energy consumption of the inspection, and the risks of the inspection path according to different application scenarios and needs.

[0104] For example, in scenarios with high time requirements, the weight of the total inspection path length should be appropriately increased, prioritizing shorter paths; while in scenarios more sensitive to energy consumption, greater emphasis should be placed on reducing the total energy consumption of inspections. At the same time, the risks associated with the inspection path should not be ignored, ensuring the safety of the inspection path.

[0105] By comprehensively considering various factors of the overall cost of the route and conducting scientific and reasonable optimization, the efficiency, economy, and safety of automatic inspection of flood discharge tunnels can be further improved.

[0106] S40: Based on the target inspection section and the obstacle risk section, configure the optimized target to perform global path planning and generate an automatic inspection path;

[0107] In this embodiment, the optimization objective is defined by combining the identified target inspection sections and obstacle risk sections. In global path planning, the optimization objective must consider not only the overall path cost but also the importance of the target inspection sections and the degree of danger of the obstacle risk sections. For target inspection sections, different priorities are determined based on their risk assessment parameters and historical inspection records. High-priority target inspection sections need to be covered first, and path planning should ensure that the inspection equipment can reach the obstacle risk sections efficiently and safely.

[0108] When considering obstacle risk sections, these should be excluded from traversable areas as forbidden spaces. Further constraints should be imposed on route planning based on the severity of the risk. For high-risk obstacle sections, a larger safety buffer zone should be established to prevent inspection equipment from approaching the obstacle area, ensuring the safety of inspection work.

[0109] Furthermore, with the basic constraint of covering all target inspection sections, global path planning is performed with multiple objectives: minimizing the overall path cost, maximizing the priority satisfaction of target inspection sections, and maximizing the degree of obstacle risk avoidance. Intelligent optimization algorithms, such as genetic algorithms and ant colony algorithms, are used to find the optimal path solution in the search space.

[0110] During the planning process, the path is continuously iterated and optimized. Adjustments and corrections are made based on real-time feedback, such as the latest data collected by sensors and dynamic changes in obstacle risks. Simultaneously, the actual performance and operational limitations of the inspection equipment are considered to ensure the feasibility of the planned path in practical applications.

[0111] By using a global path planning method based on target inspection sections and obstacle risk sections, the generated automatic inspection path can minimize the overall path cost and improve inspection efficiency while meeting the requirements of the inspection task, and effectively avoid obstacle risks to ensure the safety and reliability of the inspection work.

[0112] S50: Control the inspection equipment to execute the automatic inspection path.

[0113] In this embodiment, after determining the automatic inspection path, the inspection equipment undergoes corresponding parameter settings and preparation. First, based on the characteristics and requirements of the automatic inspection path, the motion parameters of the inspection equipment are adjusted, such as setting an appropriate travel speed to ensure efficient and stable operation under different road conditions and inspection tasks. For complex tunnel environments, if encountering narrow passages or curves, the speed can be appropriately reduced to ensure equipment safety. Simultaneously, based on the path length and the complexity of the inspection task, the operating mode of the inspection equipment is set, such as continuous inspection mode or segmented inspection mode.

[0114] Furthermore, before the inspection equipment departs, functional checks and status verifications are conducted. The equipment's sensors are checked to ensure they are functioning properly and can accurately collect multimodal sensing data, such as the sensitivity of the geological sensor and the accuracy of the water pressure sensor. The communication module is also checked for stability to ensure timely transmission of collected data and equipment status information back to the control center during the inspection. In addition, the equipment's power system is checked to ensure sufficient power or fuel to support the completion of the entire inspection mission.

[0115] When the inspection equipment begins executing its automatic inspection path, the control center monitors the equipment's operating status and location information in real time. By receiving data transmitted from the inspection equipment, the center promptly understands whether the equipment is following the planned path and whether any abnormalities have been encountered. If the equipment deviates from the path, the control center can immediately issue instructions to adjust the equipment's direction of travel, bringing it back to the correct path. Simultaneously, the collected multimodal sensing data is analyzed in real time. If potential obstacles, risks, or anomalies are detected, appropriate measures are taken promptly, such as further inspection or adjustments to subsequent inspection plans.

[0116] Furthermore, controlling the inspection equipment to execute the automatic inspection path also includes:

[0117] Based on the historical inspection records, volatility analysis is performed, and the target tunnel topology network is partitioned according to the volatility analysis results.

[0118] Combining the volatility analysis results with the preset baseline update frequency, the partitioning results are sensor-updated and collected to obtain a local multimodal sensing dataset;

[0119] Based on the historical inspection records, determine the significance threshold corresponding to the partitioning results, and determine whether the local multimodal sensing dataset exceeds the significance threshold.

[0120] If the number of cases exceeds the limit, the local inspection paths corresponding to the multiple partitions will be paused, and the paths will be reconstructed by combining the local multimodal perception dataset.

[0121] In this embodiment, firstly, volatility analysis is performed based on historical inspection records to analyze the changes in multimodal sensing data of different areas of the tunnel during different inspection cycles. By analyzing the fluctuation amplitude, frequency, and other characteristics of the data, the stability and changing trends of each area of ​​the tunnel are understood.

[0122] For example, if the water pressure data for a certain area fluctuates significantly over multiple inspection cycles, it indicates that there may be potential problems in that area, requiring close monitoring. Based on the fluctuation analysis results, the target tunnel topology network is partitioned, grouping areas with similar fluctuation characteristics into a single partition, allowing for more targeted subsequent sensing updates and path planning.

[0123] Secondly, based on the volatility analysis results and the preset baseline update frequency, the partitioning results are updated using a perceptual update mechanism. For partitions with large fluctuations, the perceptual update frequency is appropriately increased to capture data changes in a timely manner; for partitions with small fluctuations and relatively stable data, the update frequency is followed.

[0124] Using the above method, a local multimodal sensing dataset is obtained, which contains multimodal sensing data of each partition within a specific time period.

[0125] Then, based on historical inspection records, the significance threshold corresponding to the partitioning results is determined. The significance threshold is a numerical boundary for judging whether the local multimodal sensing dataset reaches a statistically significant level. It is set based on historical data distribution, such as setting a 95% confidence interval or a p-value < 0.05. Different partitions may have different significance thresholds. By comparing the local multimodal sensing dataset with the significance threshold, it is determined whether the local multimodal sensing dataset exceeds the significance threshold.

[0126] If the local multimodal sensing dataset exceeds the significance threshold, it indicates that an anomaly may have occurred in the corresponding partition. In this case, the local inspection paths for the corresponding multiple partitions are paused to prevent the inspection equipment from entering potentially dangerous areas. Simultaneously, path reconstruction is performed using the local multimodal sensing data.

[0127] Furthermore, path reconstruction needs to consider the location, extent, and potential hazard level of abnormal areas, as well as the inspection needs of other unaffected areas. Intelligent algorithms, such as Dijkstra's algorithm or A*, can be employed. Algorithms and other technologies are used to replan inspection routes, ensuring that inspection equipment can avoid abnormal areas while covering other areas that need to be inspected as much as possible.

[0128] For example, taking Dijkstra's algorithm, this algorithm starts from the starting node and gradually expands to adjacent nodes, calculating the shortest path to each node. In path reconstruction, nodes in abnormal regions are excluded, and path searching is based on nodes in normal regions. First, the distance to the starting node is set to 0, and the distances to other nodes are set to infinity. Then, starting from the node with the smallest distance, the distances to its adjacent nodes are updated. If the distance to an adjacent node through the current node is shorter than the previously recorded distance, the distances to the adjacent nodes and their predecessors are updated. This process is repeated until the distances to all reachable nodes are updated. Finally, the shortest path from the starting node to the target node is obtained by backtracking based on the predecessors.

[0129] During the path reconfiguration process, it is also necessary to consider the actual performance and operational limitations of the inspection equipment to ensure the feasibility of the newly planned path in practical applications. Through dynamic path adjustment and reconfiguration mechanisms, the safety and effectiveness of automatic inspection of flood discharge tunnels can be further improved, enabling timely detection and handling of abnormal situations during tunnel operation.

[0130] In summary, compared to existing technologies, this application focuses on minimizing the overall cost of the inspection path. It comprehensively considers multiple factors such as the total length of the inspection path, total energy consumption, and risks associated with the path, ensuring comprehensive inspection while minimizing costs. Furthermore, by rationally balancing and optimizing various cost factors according to different application scenarios and needs, the method's flexibility and adaptability are improved.

[0131] In summary, the embodiments of this application have at least the following technical effects:

[0132] This application provides a method for optimizing the automatic inspection path of flood discharge tunnels using multimodal sensing. First, a twin simulation model of the target tunnel is constructed using multi-source intrinsic data, achieving accurate mapping and simulation of the actual tunnel conditions and providing a reliable foundation for subsequent path planning and risk prediction. Second, based on the twin simulation model and historical inspection records, an obstacle mechanism model and an inspection prediction model are trained, accurately predicting obstacle risk sections and target inspection sections, fully considering the complexity and variability of the tunnel environment. Third, by collecting multimodal sensing data in real time and inputting it into the model, relevant information for the current inspection cycle is obtained, ensuring the timeliness and accuracy of path planning.

[0133] Through the above technical solution, this application overcomes the shortcomings of the prior art, which relies on manual labor or a single sensor, lacks global optimization in path planning, and is difficult to adapt to complex and ever-changing environments, thus providing a strong guarantee for the safe operation of flood discharge tunnels.

[0134] Example 2, as Figure 2As shown, based on the same inventive concept as the multimodal sensing automatic inspection path optimization method for flood discharge tunnels provided in Embodiment 1, this application also provides a multimodal sensing automatic inspection path optimization system for flood discharge tunnels, including:

[0135] The information acquisition module 11 is used to acquire multi-source intrinsic data of the target tunnel and construct a twin simulation model of the target tunnel based on the multi-source intrinsic data.

[0136] The model training module 12 is used to train and obtain an obstacle mechanism model for predicting obstacle risk sections and an inspection prediction model for predicting target inspection sections based on the twin simulation model and the historical inspection records of the target tunnel.

[0137] The model prediction module 13 is used to collect multimodal perception data of the target tunnel in real time, and input the multimodal perception data into the obstacle mechanism model and the inspection prediction model respectively to obtain the obstacle risk section and the target inspection section of the current inspection cycle.

[0138] The path planning module 14 is used to configure optimization targets and perform global path planning based on the target inspection section and the obstacle risk section to generate an automatic inspection path;

[0139] The inspection execution module 15 is used to control the inspection equipment to execute the automatic inspection path.

[0140] In one embodiment, the information acquisition module 11 is specifically used for:

[0141] Based on the multi-source intrinsic data, a tunnel topology model characterizing the structural connection relationship of the target tunnel is constructed;

[0142] Based on the tunnel topology model, a twin simulation model of the target tunnel is constructed by combining the multi-source intrinsic data.

[0143] In one embodiment, the model training module 12 is specifically used for:

[0144] Analyze the historical inspection records to extract historical sensing data and associated historical obstacle risk sections, wherein the historical sensing data and the multimodal sensing data have the same dimension;

[0145] The historical sensing data is input into the twin simulation model for simulation analysis. Based on the simulation analysis results, the sample obstacle risk segment is determined, and a mapping relationship between the sample obstacle risk segment and the historical sensing data is established.

[0146] By comparing the sample obstacle risk segments with the historical obstacle risk segments, the confidence obstacle risk segments are determined, and obstacle training data is constructed by combining the historical perception data with the mapping relationship.

[0147] Based on the obstacle training data, construct and train the obstacle mechanism model based on a graph neural network.

[0148] Furthermore, in one embodiment of the application, based on the twin simulation model and the historical inspection records of the target tunnel, a barrier mechanism model for predicting obstacle risk sections and an inspection prediction model for predicting target inspection sections are trained and obtained, including:

[0149] The twin simulation model is used to determine multiple minimum inspection units in the target tunnel;

[0150] Based on the historical inspection records, evaluate the risk assessment parameters corresponding to multiple smallest inspection units;

[0151] Based on the risk assessment parameters, the plurality of minimum inspection units are screened to determine the target inspection unit set;

[0152] An ensemble learning method is used to configure at least one base model for each of the smallest inspection units in the target inspection unit set, and multiple base models are integrated through the ensemble learning method to obtain the inspection prediction model.

[0153] The number of base models configured for each of the minimum inspection units is positively correlated with the corresponding risk assessment parameters.

[0154] In one embodiment, the model prediction module 13 is specifically used for:

[0155] Obtain the tunnel topology model of the target tunnel, and update the tunnel topology model with the obstacle risk section as the taboo space to obtain the timely inspection topology model;

[0156] With the constraint of covering the target inspection section and the optimization objective of minimizing the overall path cost, initialize the path planning rules;

[0157] Based on the greedy algorithm, a first alternative path is generated in the real-time inspection topology model, and the first alternative path is randomly transformed in combination with the constraints to obtain a first alternative path set.

[0158] Based on the first set of candidate paths, the path planning rules, and the timely inspection topology model, global path planning based on multi-objective optimization is performed to obtain the automatic inspection path.

[0159] The comprehensive cost of the inspection path includes at least the total length of the inspection path, the total energy consumption of the inspection, and the risks associated with the inspection path.

[0160] Furthermore, in one embodiment of the application, constructing and training the obstacle mechanism model based on a graph neural network according to the obstacle training data includes:

[0161] The nodes and node edges in the configuration graph structure are configured, wherein the nodes are used to represent path environmental elements in the inspection path, and the node edges are used to represent the association relationships between the path environmental elements;

[0162] A graph neural network structure is constructed based on the nodes and node edges, and the graph neural network is trained using the obstacle training data as input data to obtain an initial obstacle mechanism model;

[0163] The initial obstacle mechanism model was validated, and the network parameters were corrected based on the validation results.

[0164] Based on the validated initial obstacle mechanism model, multiple rule conditions are generated and output, and these multiple rule conditions are stored as the obstacle mechanism model.

[0165] Furthermore, in one embodiment, controlling the inspection device to execute the automatic inspection path further includes:

[0166] Based on the historical inspection records, volatility analysis is performed, and the target tunnel topology network is partitioned according to the volatility analysis results.

[0167] Combining the volatility analysis results with the preset baseline update frequency, the partitioning results are sensor-updated and collected to obtain a local multimodal sensing dataset;

[0168] Based on the historical inspection records, determine the significance threshold corresponding to the partitioning results, and determine whether the local multimodal sensing dataset exceeds the significance threshold.

[0169] If the number of cases exceeds the limit, the local inspection paths corresponding to the multiple partitions will be paused, and the paths will be reconstructed by combining the local multimodal perception dataset.

[0170] It should be noted that the order of the embodiments described above is merely for descriptive purposes and does not represent the superiority or inferiority of the embodiments. Furthermore, the above description focuses on specific embodiments of this specification. Additionally, the processes depicted in the accompanying drawings do not necessarily require a specific or sequential order to achieve the desired results. In some implementations, multitasking and parallel processing are possible or may be advantageous.

[0171] The above description is only a preferred embodiment of this application and is not intended to limit this application. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this application should be included within the protection scope of this application.

[0172] This specification and accompanying drawings are merely illustrative examples of this application and are intended to cover any and all modifications, variations, combinations, or equivalents within the scope of this application. Clearly, those skilled in the art can make various alterations and modifications to this application without departing from its scope. Therefore, if such modifications and modifications fall within the scope of this application and its equivalents, this application intends to include such modifications and modifications.

Claims

1. A method for optimizing the automatic inspection path of flood discharge tunnels based on multimodal sensing, characterized in that, include: Acquire multi-source intrinsic data of the target tunnel, and construct a twin simulation model of the target tunnel based on the multi-source intrinsic data; Based on the twin simulation model and the historical inspection records of the target tunnel, an obstacle mechanism model for predicting obstacle risk sections and an inspection prediction model for predicting target inspection sections are trained and obtained. Multimodal perception data of the target tunnel is collected in real time, and the multimodal perception data is input into the obstacle mechanism model and the inspection prediction model respectively to obtain the obstacle risk section and the target inspection section of the current inspection cycle. Based on the target inspection section and the obstacle risk section, configure and optimize the target to perform global path planning and generate an automatic inspection path; Control the inspection equipment to execute the automatic inspection path; Based on the twin simulation model and the historical inspection records of the target tunnel, an obstacle mechanism model for predicting obstacle risk sections and an inspection prediction model for predicting target inspection sections are trained and obtained, including: The twin simulation model is used to determine multiple minimum inspection units in the target tunnel; Based on the historical inspection records, evaluate the risk assessment parameters corresponding to multiple smallest inspection units; Based on the risk assessment parameters, the plurality of minimum inspection units are screened to determine the target inspection unit set; An ensemble learning method is used to configure at least one base model for each of the smallest inspection units in the target inspection unit set, and multiple base models are integrated through the ensemble learning method to obtain the inspection prediction model. The number of base models configured for each minimum inspection unit is positively correlated with the corresponding risk assessment parameters. Based on the target inspection section and the obstacle risk section, global path planning is performed by configuring and optimizing the target to generate an automatic inspection path, including: Obtain the tunnel topology model of the target tunnel, and update the tunnel topology model with the obstacle risk section as the taboo space to obtain the timely inspection topology model; With the constraint of covering the target inspection section and the optimization objective of minimizing the overall path cost, initialize the path planning rules; Based on the greedy algorithm, a first alternative path is generated in the real-time inspection topology model, and the first alternative path is randomly transformed in combination with the constraints to obtain a first alternative path set. Based on the first set of candidate paths, the path planning rules, and the timely inspection topology model, global path planning based on multi-objective optimization is performed to obtain the automatic inspection path.

2. The method for optimizing the automatic inspection path of a flood discharge tunnel based on multimodal sensing as described in claim 1, characterized in that, Acquire multi-source intrinsic data of the target tunnel, and construct a twin simulation model of the target tunnel based on the multi-source intrinsic data, including: Based on the multi-source intrinsic data, a tunnel topology model characterizing the structural connection relationship of the target tunnel is constructed; Based on the tunnel topology model, a twin simulation model of the target tunnel is constructed by combining the multi-source intrinsic data.

3. The method for optimizing the automatic inspection path of a flood discharge tunnel based on multimodal sensing as described in claim 1, characterized in that, Based on the twin simulation model and the historical inspection records of the target tunnel, an obstacle mechanism model for predicting obstacle risk sections and an inspection prediction model for predicting target inspection sections are trained and obtained, including: Analyze the historical inspection records to extract historical sensing data and associated historical obstacle risk sections, wherein the historical sensing data and the multimodal sensing data have the same dimension; The historical sensing data is input into the twin simulation model for simulation analysis. Based on the simulation analysis results, the sample obstacle risk segment is determined, and a mapping relationship between the sample obstacle risk segment and the historical sensing data is established. By comparing the sample obstacle risk segments with the historical obstacle risk segments, the confidence obstacle risk segments are determined, and obstacle training data is constructed by combining the historical perception data with the mapping relationship. Based on the obstacle training data, construct and train the obstacle mechanism model based on a graph neural network.

4. The method for optimizing the automatic inspection path of a flood discharge tunnel based on multimodal sensing as described in claim 3, characterized in that, Based on the obstacle training data, construct and train the obstacle mechanism model based on a graph neural network, including: The nodes and node edges in the configuration graph structure are configured, wherein the nodes are used to represent path environmental elements in the inspection path, and the node edges are used to represent the association relationships between the path environmental elements; A graph neural network structure is constructed based on the nodes and node edges, and the graph neural network is trained using the obstacle training data as input data to obtain an initial obstacle mechanism model; The initial obstacle mechanism model was validated, and the network parameters were corrected based on the validation results. Based on the validated initial obstacle mechanism model, multiple rule conditions are generated and output, and these multiple rule conditions are stored as the obstacle mechanism model.

5. The method for optimizing the automatic inspection path of a flood discharge tunnel based on multimodal sensing as described in claim 1, characterized in that, The overall cost of the inspection path includes at least the total length of the inspection path, the total energy consumption of the inspection, and the risks associated with the inspection path.

6. The method for optimizing the automatic inspection path of a flood discharge tunnel based on multimodal sensing as described in claim 1, characterized in that, Controlling the inspection equipment to execute the automatic inspection path also includes: Based on the historical inspection records, volatility analysis is performed, and the target tunnel topology network is partitioned according to the volatility analysis results. Combining the volatility analysis results with the preset baseline update frequency, the partitioning results are sensor-updated and collected to obtain a local multimodal sensing dataset; Based on the historical inspection records, determine the significance threshold corresponding to the partitioning results, and determine whether the local multimodal sensing dataset exceeds the significance threshold. If the number of cases exceeds the limit, the local inspection paths corresponding to the multiple partitions will be paused, and the paths will be reconstructed by combining the local multimodal perception dataset.

7. A multimodal sensing-based automatic inspection path optimization system for flood discharge tunnels, characterized in that: The method for optimizing the automatic inspection path of a flood discharge tunnel with multimodal sensing as described in any one of claims 1-6 includes: The information acquisition module is used to acquire multi-source intrinsic data of the target tunnel and construct a twin simulation model of the target tunnel based on the multi-source intrinsic data. The model training module is used to train and obtain an obstacle mechanism model for predicting obstacle risk sections and an inspection prediction model for predicting target inspection sections based on the twin simulation model and the historical inspection records of the target tunnel. The model prediction module is used to collect multimodal perception data of the target tunnel in real time, and input the multimodal perception data into the obstacle mechanism model and the inspection prediction model respectively to obtain the obstacle risk section and the target inspection section of the current inspection cycle. The path planning module is used to configure optimization targets based on the target inspection section and the obstacle risk section to perform global path planning and generate an automatic inspection path; The inspection execution module is used to control the inspection equipment to execute the automatic inspection path.

Images