Power grid infrastructure construction site safety dynamic control system and method based on multimodal data

Abstract

The present invention discloses a system and method for dynamic safety control of power grid infrastructure construction sites based on multimodal data, and relates to the technical field of infrastructure construction site control. This system, based on multimodal data, comprehensively acquires and analyzes information such as the power grid infrastructure construction site environment, roads, and vehicles using multimodal data. It determines the optimal driving route through a scientific route planning algorithm, uses drones and satellites to monitor road conditions in real time, combines AI models to accurately assess the vehicle's own status, and quickly matches emergency response plans based on the monitoring results. This significantly improves the safety and control efficiency of vehicle driving at power grid infrastructure construction sites, reduces the occurrence of safety accidents, and ensures the smooth progress of power grid infrastructure projects. It can address the problems of existing power grid infrastructure construction sites, such as unscientific vehicle route planning, inadequate real-time and comprehensive monitoring of road conditions and vehicle status, and inadequate timely and effective emergency response.

Description

Technical Field

[0001] The present invention relates to the technical field of infrastructure site control, and in particular to a system and method for dynamic control of power grid infrastructure site safety based on multimodal data. Background Art

[0002] The construction industry is a high-risk sector, with numerous and diverse safety control points at construction sites. This is particularly challenging for open transmission line construction sites. With the rapid growth of power grids and the increasing scale of grid construction, the safety risks associated with on-site construction management are increasing day by day.

[0003] Related technologies disclose a dynamic safety management and control system for power grid infrastructure construction sites based on the ubiquitous power Internet of Things. The system includes: a wearable detection system for managing personnel and safety equipment at the construction site; a virtual fence system for monitoring key areas at the construction site; a multimodal data acquisition system for collecting personnel data at the construction site; a local server for managing the wearable detection system, the virtual fence system, and the multimodal data acquisition system; a remote server for data exchange with the local server; and a backend for data exchange with the remote server.

[0004] However, when planning vehicle routes at existing power grid infrastructure sites, road conditions and vehicle status monitoring are not real-time and comprehensive enough, and emergency response is not timely and effective enough. Summary of the Invention

[0005] In response to the shortcomings of the existing technology, the present invention provides a dynamic safety management and control system and method for power grid infrastructure construction sites based on multimodal data, which solves the problems of insufficient real-time and comprehensive monitoring of road conditions and vehicle status during vehicle route planning at existing power grid infrastructure sites, and insufficient timely and effective emergency response.

[0006] To achieve the above objectives, the present invention is implemented through the following technical solutions: a power grid infrastructure construction site safety dynamic management and control system based on multimodal data, including a power grid infrastructure construction site multimodal data acquisition subsystem, a vehicle driving route determination subsystem, a vehicle driving route monitoring subsystem, a vehicle driving vehicle monitoring subsystem, and an emergency response plan determination subsystem, wherein:

[0007] The multi-modal data acquisition subsystem at the power grid infrastructure construction site is used to acquire multi-modal data at the power grid infrastructure construction site, including environmental data, road condition information, and soil condition information; the vehicle route determination subsystem is used to plan the vehicle route based on multi-modal data and determine the optimal route; the route monitoring subsystem during vehicle driving is used to monitor the road conditions in real time based on drones and satellites when the vehicle is driving on the optimal route and determine the real-time monitoring results of the road conditions; the vehicle monitoring subsystem during vehicle driving is used to acquire the vehicle operating status data when the vehicle is driving on the optimal route, and monitor the vehicle's own status based on the vehicle operating status data to obtain the real-time monitoring results of the vehicle status; the emergency response plan determination subsystem is used to match the database emergency plan based on the real-time monitoring results of the road condition and the real-time monitoring results of the vehicle status to determine the emergency response plan.

[0008] Furthermore, the vehicle driving route determination subsystem includes an initial planned route determination module and an optimal route determination module, wherein: the initial planned route determination module is used to determine several initial driving routes based on road condition information and soil condition information, and preliminarily screen the initial driving routes based on environmental data to determine multiple driving routes to be selected; the optimal route determination module is used to screen the driving routes to be selected based on the actual route information obtained by the drone to determine the optimal driving route.

[0009] Furthermore, the initial planning route determination module includes a grid division unit, a first initial route determination unit, a second initial route determination unit and a route summary unit, wherein: the grid division unit is used to grid the area including the vehicle starting point and the vehicle end point; the first initial route determination unit is used to connect the grid where the vehicle starting point is located and the grid where the vehicle end point is located to form n1 first initial candidate routes; the n1 first initial candidate routes are arranged in order of distance from small to large, and the first n2 first initial candidate routes are selected as first pre-selected routes; the grids traversed by the first pre-selected routes are determined, the soil condition information corresponding to each grid is determined, and the bearing capacity quantitative value of each grid is determined by comparison from a database based on the soil condition information; the bearing capacity quantitative values ​​of each grid traversed by each first pre-selected route are accumulated, and the accumulated results are arranged in order from large to small, and the first n3 first pre-selected routes are selected as first alternative paths;

[0010] A second initial route determination unit is configured to extract a normalized road surface roughness value and a quantitative carrying capacity value corresponding to each grid from the road condition information; perform a weighted summation of the normalized road surface roughness value and the quantitative carrying capacity value to determine a transport capacity assessment value for each grid; connect the grid where the vehicle's starting point is located with the grid where the vehicle's end point is located to form m1 second initial candidate routes; calculate a total value of the grid transport capacity assessment values ​​for each second initial candidate route, and arrange each second initial candidate route in descending order based on the total value, selecting the first m2 second initial candidate routes as second candidate routes;

[0011] The route summarizing unit is used to summarize the first candidate route and the second candidate route, and perform preliminary screening to obtain multiple candidate travel routes.

[0012] Furthermore, a preliminary screening is performed to obtain multiple candidate driving routes as follows: the first alternative path and the second alternative path are screened for security maintenance points. If the distance between any two adjacent security maintenance points in all the security maintenance points in the first alternative path or the second alternative path is greater than a set distance threshold, the first alternative path or the second alternative path corresponding to the two security maintenance points is screened out; environmental data of the grids passed by the first alternative path and the second alternative path are obtained, and the probability of severe weather occurrence is extracted from the environmental data. If the probability of severe weather occurrence corresponding to any grid in the grids passed by the first alternative path and the second alternative path is greater than the set severe weather probability threshold, the first alternative path or the second alternative path corresponding to the grid is screened out; the first alternative path and the second alternative path that are not screened out are retained and recorded as candidate driving routes.

[0013] Furthermore, the process of screening the selected driving routes based on the actual route information obtained by the drone and determining the optimal driving route is as follows: extracting parameters from the actual route information to obtain road condition characteristics and energy endurance characteristics. The road condition characteristics include the mean square error of the slope , the probability of wild animals going out , density of low-lying areas on the road and the number of unavoidable obstacles , energy endurance characteristics include the average available resources at maintenance points near the selected driving route and the number of available support vehicles ; Based on the road condition characteristics and energy endurance characteristics, the path driving safety assessment is carried out to obtain the safety assessment coefficient :

[0014] ;

[0015] in, and are weight factors, and are all transfer functions, is the slope mean square error benchmark value, The baseline value for the probability of wild animals going out. is the density benchmark value of low-lying areas on the road surface, is the benchmark value for the number of unavoidable obstacles, To ensure the average baseline value of available resources at maintenance points, It is the benchmark value of the number of available support vehicles;

[0016] The alternative driving route corresponding to the largest safety assessment coefficient is recorded as the optimal driving route.

[0017] Furthermore, the route monitoring subsystem during vehicle driving includes a drone monitoring module and a satellite monitoring module, wherein: the drone monitoring module is used to obtain the first path real-time data of the vehicle driving collected by the drone when the drone is allowed to monitor, diagnose and identify the road condition at a set distance in front of the vehicle based on the first path real-time data, and output the real-time monitoring result of the road condition, and the first path real-time data includes high-definition images and point cloud data; the satellite monitoring module is used to obtain the second path real-time data of the vehicle driving collected by the satellite when the drone is not allowed to monitor, diagnose and identify the road condition at a set distance in front of the vehicle based on the second path real-time data, and output the real-time monitoring result of the road condition, and the second path real-time data includes visible light images and microwave data; the real-time monitoring result of the road condition includes a clear road, a partially blocked road, and a completely interrupted road.

[0018] Furthermore, the judgment process of whether the drone is allowed to be monitored is as follows: obtain the drone's own operating status data. If any parameter in the drone's own operating status data is abnormal, the drone is not allowed to be monitored; if all parameters in the drone's own operating status data are normal, obtain the drone's environmental data, including the maximum wind speed, maximum temperature and maximum electromagnetic signal strength; based on the drone's environmental data, obtain the drone's environmental difference rate, including the wind speed difference rate, temperature difference rate and electromagnetic signal strength difference rate; perform weighted summation on the wind speed difference rate, temperature difference rate and electromagnetic signal strength difference rate to obtain a difference evaluation coefficient. If the difference evaluation coefficient is greater than the set evaluation threshold, the drone is not allowed to be monitored. If the difference evaluation coefficient is not greater than the set evaluation threshold, the drone is allowed to be monitored.

[0019] Furthermore, the process of diagnosing and identifying the road condition at a set distance ahead of the vehicle based on the first path real-time data is as follows: the preprocessed high-definition image and point cloud data are input into the trained lightweight YOLO model, and the real-time monitoring results of the road condition are output; the process of diagnosing and identifying the road condition at a set distance ahead of the vehicle based on the second path real-time data is as follows: the preprocessed visible light image and microwave data are input into the trained lightweight neural network model, and the real-time monitoring results of the road condition are output.

[0020] Furthermore, the process of monitoring the vehicle's own status based on the vehicle operating status data and obtaining the real-time monitoring result of the vehicle status is as follows: performing item-by-item inspection based on the vehicle operating status data, including power system inspection, braking system inspection, tire status inspection, load condition and fuel level inspection: comparing each parameter in the vehicle operating status data with the preset rule library one by one to determine whether the parameter is abnormal and the corresponding degree of abnormality, and outputting the real-time monitoring result of the vehicle status; if all parameters in the vehicle operating status data are normal, prediction is performed based on the trained LSTM neural network; the process of prediction based on the trained LSTM neural network includes: inputting the vehicle's historical operating status data at the current moment and the previous t moments into the trained LSTM neural network, and outputting the vehicle's predicted operating status data at the next moment; checking the vehicle's predicted operating status data item by item, and outputting the vehicle's real-time monitoring result.

[0021] A method for dynamic safety control of power grid infrastructure construction sites based on multimodal data includes the following steps: obtaining multimodal data of the power grid infrastructure construction site, including environmental data, road condition information, and soil condition information; planning vehicle routes based on the multimodal data to determine an optimal route; when a vehicle is traveling on the optimal route, monitoring road conditions in real time based on drones and satellites to determine real-time monitoring results of road conditions; obtaining vehicle operating status data while the vehicle is traveling on the optimal route, and monitoring the vehicle's own status based on the vehicle operating status data to obtain real-time monitoring results of the vehicle status; and matching emergency plans with a database based on the real-time monitoring results of road conditions and vehicle status to determine an emergency response plan.

[0022] The present invention has the following beneficial effects:

[0023] This multimodal data-based dynamic safety management and control system for power grid infrastructure construction sites uses multimodal data to fully acquire and analyze information such as the power grid infrastructure site environment, roads, and vehicles. It determines the optimal driving route through a scientific route planning algorithm, uses drones and satellites to monitor road conditions in real time, combines AI models to accurately assess the vehicle's own status, and quickly matches emergency response plans based on the monitoring results, thereby significantly improving the safety and management efficiency of vehicle driving at power grid infrastructure sites, reducing the occurrence of safety accidents, and ensuring the smooth progress of power grid infrastructure projects. It can solve the problems of insufficient real-time and comprehensive monitoring of road conditions and vehicle status during vehicle route planning at existing power grid infrastructure sites, and insufficient timely and effective emergency response.

[0024] Of course, any product implementing the present invention does not necessarily need to achieve all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 This is a flowchart of the dynamic control system for power grid infrastructure site safety based on multimodal data of the present invention.

[0026] Figure 2 This is a flow chart of the method for dynamic control of power grid infrastructure site safety based on multimodal data of the present invention. DETAILED DESCRIPTION

[0027] See also Figure 1 The embodiment of the present invention provides a technical solution: a power grid infrastructure site safety dynamic management and control system based on multimodal data, including a power grid infrastructure site multimodal data acquisition subsystem, a vehicle route determination subsystem, a vehicle route monitoring subsystem during vehicle driving, a vehicle monitoring subsystem during vehicle driving, and an emergency response plan determination subsystem, wherein:

[0028] The multi-modal data acquisition subsystem at the power grid infrastructure construction site is used to obtain multi-modal data at the power grid infrastructure construction site, including environmental data, road condition information, and soil condition information.

[0029] Environmental data: Contains meteorological information of the construction site, such as wind speed, wind direction, rainfall, temperature, humidity, etc. This data can be collected in real time through weather stations or sensors.

[0030] Road condition information: covers road geometric parameters such as road direction, length, width, slope, curve radius, etc.; road surface conditions, including road surface type (cement, asphalt, gravel, etc.), road surface flatness, degree of damage (cracks, potholes, subsidence, etc.); and information on obstacles on the road, such as the location and size of construction equipment, stacked building materials, temporary buildings, etc., which can be obtained through drone aerial photography, lidar scanning, etc.

[0031] Soil condition information: including soil type (sand, clay, silt, etc.), soil physical and mechanical properties, such as soil density, water content, porosity, compressibility, bearing capacity, etc., which can be obtained through geological exploration, soil sampling and testing, in-situ testing (such as dynamic probing and static probing), etc.

[0032] The vehicle route determination subsystem is used to plan the vehicle route based on multimodal data and determine the optimal route;

[0033] The vehicle driving route determination subsystem includes an initial planning route determination module and an optimal route determination module, wherein: the initial planning route determination module is used to determine a number of initial driving routes based on road condition information and soil condition information, and preliminarily screen the initial driving routes based on environmental data to determine a number of candidate driving routes;

[0034] The initial planning route determination module includes a grid division unit, a first initial route determination unit, a second initial route determination unit and a route summary unit, wherein: the grid division unit is used to grid the area containing the vehicle starting point and the vehicle end point; the area from the starting point to the end point is divided into grids, so that the route generation is based on standardized spatial units, avoiding the arbitrariness of route planning in complex terrain and laying the foundation for subsequent quantitative analysis.

[0035] The first initial route determination unit is configured to connect the grid where the vehicle's starting point is located and the grid where the vehicle's end point is located to form n1 first initial candidate routes; arrange the n1 first initial candidate routes in ascending order of distance, and select the first n2 first initial candidate routes as first pre-selected routes; determine the grids traversed by the first pre-selected routes, determine soil condition information corresponding to each grid, and determine the quantitative bearing capacity value of each grid by comparing it with a database based on the soil condition information; accumulate the quantitative bearing capacity values ​​of the grids traversed by each first pre-selected route, and arrange them in descending order based on the accumulated results, and select the first n3 first pre-selected routes as first alternative routes.

[0036] The first n3 routes are filtered by distance, with the shortest path being given priority. Accumulated screening is performed using the quantitative values ​​of soil bearing capacity to ensure that the grids through which the route passes can bear the weight of vehicles and equipment, avoiding problems such as vehicles getting stuck and road damage due to insufficient road bearing capacity.

[0037] The second initial route determination unit is configured to extract a normalized road surface roughness value and a quantitative carrying capacity value corresponding to each grid from the road condition information; perform a weighted summation of the normalized road surface roughness value and the quantitative carrying capacity value to determine a transport capacity assessment value for each grid; connect the grid where the vehicle's starting point is located with the grid where the vehicle's end point is located to form m1 second initial candidate routes; calculate the total grid transport capacity assessment value of each second initial candidate route, and arrange each second initial candidate route in descending order based on the total value, thereby selecting the first m2 second initial candidate routes as the second alternative path.

[0038] The transport capacity assessment value is calculated based on the road surface smoothness and carrying capacity, and routes with high transport capacity assessment values ​​are given priority. This can reduce the risk of vehicle bumps, ensure the safe transportation of heavy equipment, and reduce the probability of equipment damage due to bumps or vehicle failure.

[0039] The route aggregation unit aggregates the first and second candidate routes and performs preliminary screening to generate multiple potential routes. After aggregating the two types of alternative routes, the unit screens the routes based on distance to maintenance points and the probability of severe weather to ensure that appropriate maintenance points are available around the routes. This reduces the risk of vehicle interruption in complex environments such as uninhabited areas, while also avoiding high-risk weather areas and improving the routes' all-weather accessibility.

[0040] The first alternative path and the second alternative path are screened for support and maintenance points. If the distance between any two adjacent support and maintenance points in all the support and maintenance points in the first alternative path or the second alternative path is greater than the set distance threshold, the first alternative path or the second alternative path corresponding to the two support and maintenance points will be screened out. By judging whether the distance between adjacent support and maintenance points exceeds the threshold, the path with too large a distance between maintenance points is screened out, thereby avoiding problems such as fuel / power depletion and unrepairable faults caused by insufficient supply points in uninhabited areas or complex terrain. This solves the defect of ignoring logistics support in traditional route planning and improves the endurance reliability of the route in actual operation.

[0041] Obtain environmental data of the grids that the first alternative path and the second alternative path pass through, and extract the probability of severe weather from the environmental data. If the probability of severe weather corresponding to any grid among the grids passed by the first alternative path and the second alternative path is greater than the set severe weather probability threshold, the first alternative path or the second alternative path corresponding to the grid is screened out; extracting the probability of severe weather of the grids passed by the path and screening out high-risk paths can avoid the impact of extreme weather such as blizzards and strong sandstorms on vehicle driving (such as obstructed vision, slippery road surface, and equipment damage), solve the problem of sudden safety hazards caused by environmental factors, and ensure the safety of route traffic under different climatic conditions.

[0042] The first and second alternative routes that survived the screening process are retained and recorded as candidate routes. Through dual screening based on distance to maintenance points and the probability of severe weather, the remaining candidate routes both meet logistical support needs and reduce environmental risks. This addresses the one-sided issue of route planning that prioritizes distance over support and terrain over climate, providing a more scientific and reliable set of candidates for subsequent optimal route determination.

[0043] The optimal route determination module is used to screen the candidate driving routes based on the actual route information obtained by the drone and determine the optimal driving route.

[0044] Extract parameters from the actual route information to obtain road condition characteristics and energy endurance characteristics. Road condition characteristics include slope mean square error , the probability of wild animals going out , density of low-lying areas on the road and the number of unavoidable obstacles , energy endurance characteristics include the average available resources at maintenance points near the selected driving route and the number of available support vehicles ;Road condition characteristics (slope mean square deviation, probability of wild animal appearance, etc.) and energy endurance characteristics (mean value of available resources at maintenance points, etc.) are extracted from the actual route information obtained by drones, comprehensively covering the safety risk points and security needs of vehicles driving at power grid infrastructure sites, solving the problem of traditional route planning with single parameters and being out of touch with actual scenarios, making route evaluation more suitable for complex on-site environments.

[0045] Based on the road condition characteristics and energy endurance characteristics, the path driving safety is evaluated to obtain the safety assessment coefficient :

[0046] ;

[0047] in, and are weight factors, and are all transfer functions, is the slope mean square error benchmark value, The baseline value for the probability of wild animals going out. is the density benchmark value of low-lying areas on the road surface, is the benchmark value for the number of unavoidable obstacles, To ensure the average baseline value of available resources at maintenance points, The safety assessment coefficient formula integrates multi-dimensional parameters, using weighting factors and transfer functions to quantify the impact of different characteristics on safety, thus avoiding errors caused by subjective judgment. For example, the mean square error of slope reflects the impact of route terrain undulation on vehicle stability, while the probability of wildlife sighting quantifies the risk of biological interference. This solves the problem of multi-factor unified assessment and makes route safety assessment more scientific and operational.

[0048] The average available resources and the number of available support vehicles at the support maintenance points are included in the assessment to ensure that the selected route can receive timely support in the event of vehicle failure or energy shortage. This solves the problem of weak logistical support in uninhabited areas or remote infrastructure sites, and improves the reliability and emergency response capabilities of the route in actual operation.

[0049] The alternative driving route corresponding to the largest safety assessment coefficient is recorded as the optimal driving route.

[0050] The route monitoring subsystem during vehicle driving is used to monitor road conditions in real time based on drones and satellites when the vehicle is driving on the optimal route and determine the real-time monitoring results of road conditions;

[0051] The vehicle route monitoring subsystem includes a drone monitoring module and a satellite monitoring module, wherein: the drone monitoring module is used to obtain real-time data of the first route of the vehicle during driving collected by the drone, if the drone is allowed to perform monitoring, diagnose and identify the road condition at a set distance ahead of the vehicle based on the first route real-time data, and output real-time monitoring results of the road condition, wherein the first route real-time data includes high-definition images and point cloud data;

[0052] The pre-processed high-definition images and point cloud data are input into the trained lightweight YOLO model to output real-time road condition monitoring results.

[0053] The process of determining whether a drone is allowed to be monitored is as follows: obtain the drone's own operating status data. If any parameter in the drone's own operating status data is abnormal, the drone is not allowed to be monitored; if all parameters in the drone's own operating status data are normal, obtain the drone's environmental data, including the maximum wind speed, maximum temperature and maximum electromagnetic signal strength; based on the drone's environmental data, obtain the drone's environmental difference rate, including the wind speed difference rate, temperature difference rate and electromagnetic signal strength difference rate, where the wind speed difference rate = (maximum wind speed - maximum wind speed for safe flight operation of the drone) / maximum wind speed for safe flight operation of the drone, and the temperature difference rate and electromagnetic signal strength difference rate are calculated in the same way as the wind speed difference rate; perform weighted summation on the wind speed difference rate, temperature difference rate and electromagnetic signal strength difference rate to obtain a difference evaluation coefficient. If the difference evaluation coefficient is greater than the set evaluation threshold, the drone is not allowed to be monitored. If the difference evaluation coefficient is not greater than the set evaluation threshold, the drone is allowed to be monitored.

[0054] By assessing the drone's operating parameters (such as battery life and sensors) and environmental variances (wind speed, temperature, and electromagnetic signals), the system prevents distortion of monitoring data due to drone failure or adverse environmental conditions. For example, if wind speeds exceed safety thresholds, forced monitoring by the drone could result in blurred images or even a crash. This assessment mechanism can proactively mitigate such risks, ensuring the accuracy of monitoring data and the safety of the drone.

[0055] The satellite monitoring module is used to obtain real-time data on the second path of the vehicle during driving, collected by satellite, when drones are not allowed to conduct monitoring. Based on the real-time data of the second path, the module diagnoses and identifies the road conditions at a set distance ahead of the vehicle, and outputs real-time monitoring results of the road conditions. The real-time data of the second path includes visible light images and microwave data. The real-time monitoring results of the road conditions include clear roads, partially blocked roads, and completely interrupted roads.

[0056] The preprocessed visible light images and microwave data are input into the trained lightweight neural network model to output real-time road condition monitoring results.

[0057] Feeding pre-processed image and point cloud data into a lightweight YOLO (You Only Look Once) model or a neural network model solves the inefficiency and high error associated with traditional manual analysis. The model quickly identifies road conditions (such as obstacles and road damage) and outputs a status of "unblocked / partially blocked / completely interrupted," enabling vehicles to adjust their routes in real time and improving emergency response times.

[0058] The backbone network of the lightweight YOLO model (such as MobileNet and ShuffleNet) extracts features from preprocessed high-definition images and point cloud data, generating feature maps at different scales. These feature maps contain both semantic and spatial information from the image and point cloud data. These feature maps are fused using structures such as the Feature Pyramid Network (FPN) to enhance the model's ability to detect objects of varying sizes. Sliding window detection is performed on the feature maps to predict the object's category and location. Because the model generates multiple prediction boxes for the same object, the NMS algorithm removes highly overlapping prediction boxes to retain the optimal prediction. The object category and location information output by the model is mapped to the original image and point cloud data to generate real-time road condition monitoring results, such as clear, partially blocked, or completely blocked roads.

[0059] The dual monitoring methods of drones and satellites address the limitations of single monitoring methods in complex environments. Drones can capture high-definition images and point cloud data at close range, enabling precise identification of road surface details (such as cracks and obstacles). Satellites, when drones are unable to operate, provide large-scale, all-weather monitoring using visible light images and microwave data, ensuring comprehensive monitoring of road conditions and resolving monitoring interruptions in complex scenarios such as uninhabited areas.

[0060] The vehicle monitoring subsystem is used to obtain vehicle operating status data when the vehicle is traveling on the optimal driving route, and monitor the vehicle's own status based on the vehicle operating status data to obtain real-time monitoring results of the vehicle status;

[0061] Perform item-by-item inspections based on vehicle operating status data, including power system inspections, brake system inspections, tire status inspections, load conditions, and fuel and power level inspections: compare each parameter in the vehicle operating status data with the preset rule library one by one to determine whether the parameter is abnormal and the corresponding degree of abnormality, and output real-time monitoring results of the vehicle status; compare key system parameters such as power, brakes, and tires with the preset rule library to detect abnormalities such as engine overheating and brake failure in real time, solve the problems of missed or delayed inspections in traditional manual inspections, achieve early warning of faults, and prevent small problems from turning into safety accidents.

[0062] If all parameters in the vehicle operating status data are normal, a prediction is performed based on a trained long short-term memory neural network (LSTM). The process of performing the prediction based on the trained LSTM neural network includes: inputting the historical vehicle operating status data at the current moment and the previous t moments into the trained LSTM neural network, outputting the predicted vehicle operating status data at the next moment; and checking the predicted vehicle operating status data item by item to output the real-time monitoring results of the vehicle status.

[0063] An LSTM neural network consists of an input layer, a hidden layer (composed of multiple LSTM units), and an output layer. The input layer receives preprocessed historical operating status data. The LSTM units in the hidden layer selectively memorize and forget historical information through the control of forget gates, input gates, and output gates, capturing long-term dependencies and dynamic changes in the data. When processing data at the current moment, each LSTM unit updates the cell state and hidden state based on the previous moment, thereby memorizing and processing time series data. Through the processing of multiple layers of LSTM units, the network can extract features relevant to the vehicle's future state from historical operating status data, such as parameter trends and periodic patterns.

[0064] Output the vehicle's predicted operating status data at the next moment, whose dimensions are consistent with the input data, that is, it also includes predicted values ​​of the power system, braking system, tire status, load conditions, and fuel and power levels.

[0065] When current parameters are normal, the LSTM system uses historical data as input to predict the next state, enabling early detection of potential faults. For example, real-time vehicle status monitoring results, including brake pad wear trends and battery life degradation, eliminate the need for reactive response to faults and allow drivers to plan maintenance or adjust driving strategies in advance. By first checking current parameters in real time and then revalidating the predicted data, the system avoids misjudgments caused by data fluctuations in a single check. This ensures comprehensive and accurate vehicle status monitoring, addresses unreliable monitoring results, and provides dual protection for safe driving.

[0066] The emergency response plan determination subsystem matches emergency plans against a database based on real-time road condition and vehicle status monitoring results to determine the emergency response plan. Based on these road and vehicle status monitoring results, the system automatically matches emergency plans (such as rescue routes for faulty vehicles and detour strategies for poor road conditions) from the database. This addresses the slow decision-making process inherent in traditional emergency response, shortens response times, and reduces the risk of accident losses and delays.

[0067] The specific matching process is as follows: obtaining the stored monitoring result-emergency plan mapping set from the database, the monitoring result-emergency plan mapping set including multiple monitoring result-emergency plan branches, comparing the real-time road condition monitoring results and the real-time vehicle status monitoring results with the monitoring results in each monitoring result-emergency plan branch for similarity, determining the most similar monitoring result, and determining the emergency plan from the corresponding monitoring result-emergency plan branch.

[0068] For example, 1. The result of real-time monitoring of road conditions is that the road is clear, and the result of real-time monitoring of vehicle status is that the brake pads are worn at 60%, which is determined to be level three wear. Through the matching of monitoring results-emergency plan branches, it can be determined that the emergency plan is to push a warning message to the driver: "The brake pads are moderately worn, it is recommended to go to the nearest maintenance point within 30 minutes." 2. The result of real-time monitoring of road conditions is that the road is partially blocked, and the result of real-time monitoring of vehicle status is that the brake pads are worn at 85%, which is determined to be level four wear. Through comparison, it is determined that the emergency plan is to forcibly limit the vehicle speed to ≤15km / h through the on-board control system, turn on the double flash warning lights, send a rescue instruction to the nearest maintenance point, and require the vehicle to arrive at its location within 15 minutes, carrying brake pads, jacks and other equipment. Dynamic management and control method for on-site safety of power grid infrastructure based on multimodal data, such as Figure 2 As shown, the method includes the following steps: obtaining multimodal data of the power grid infrastructure construction site, including environmental data, road condition information, and soil condition information; planning the vehicle route based on the multimodal data to determine the optimal route; when the vehicle is traveling on the optimal route, monitoring the road condition in real time based on drones and satellites to determine the real-time monitoring results of the road condition; and when the vehicle is traveling on the optimal route, obtaining vehicle operation status data, and monitoring the vehicle's own state based on the vehicle operation status data to obtain the real-time monitoring results of the vehicle state; matching the database emergency plan based on the real-time monitoring results of the road condition and the real-time monitoring results of the vehicle state to determine the emergency treatment plan.

[0069] The present disclosure proposes an electronic device, including: a processor; and a memory, wherein computer program instructions are stored in the memory, and when the computer program instructions are executed by the processor, the processor executes the above-mentioned power grid infrastructure site safety dynamic management and control system based on multimodal data.

[0070] The present disclosure provides a computer-readable storage medium for storing a program, which, when executed by a processor, implements the above-mentioned multi-modal data-based on-site dynamic safety management and control system for power grid infrastructure.

[0071] Those skilled in the art will appreciate that embodiments of the present invention may be provided as methods, systems, or computer program products. Thus, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0072] The present invention is described with reference to flowcharts and / or block diagrams of systems, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowcharts and / or block diagrams, as well as combinations of processes and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowcharts and / or block diagrams. Figure 1 a process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0073] These computer program instructions may also be stored in a computer readable memory that can direct a computer or other programmable data processing device to work in a specific manner, so that the instructions stored in the computer readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 a process or multiple processes and / or boxes Figure 1 The function specified in one or more boxes.

[0074] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operational steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing the instructions executed on the computer or other programmable device for implementing the process. Figure 1 a process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0075] Although the preferred embodiments of the present invention have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention.

[0076] Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if such changes and modifications fall within the scope of the claims and their equivalents, the present invention is intended to include such changes and modifications.

Claims

1. A dynamic control system for power grid infrastructure site safety based on multimodal data, characterized by: include: The multi-modal data acquisition subsystem for power grid infrastructure construction sites is used to acquire multi-modal data at power grid infrastructure construction sites, including environmental data, road condition information, and soil condition information; The vehicle route determination subsystem is used to plan the vehicle route based on multimodal data and determine the optimal route; It includes an initial planning route determination module and an optimal route determination module, wherein the initial planning route determination module includes: A grid division unit, used for dividing the area including the vehicle starting point and the vehicle end point into grids; A first initial route determining unit is configured to connect the grid where the vehicle's starting point is located with the grid where the vehicle's end point is located to form n1 first initial candidate routes; Arrange the n1 first initial candidate routes in ascending order of distance, and select the first n2 first initial candidate routes as the first pre-selected routes; Determining the grids traversed by each first pre-selected route, determining soil condition information corresponding to the grids traversed by each first pre-selected route, and determining a quantitative value of the bearing capacity of the grids traversed by each first pre-selected route by comparing the soil condition information with a database; Accumulate the quantified carrying capacity values ​​of the grids traversed by each first pre-selected route, and sort them in descending order based on the accumulated results, and select the first n3 first pre-selected routes as the first alternative paths; A second initial route determination unit is configured to extract a normalized road surface roughness value and a quantified load-bearing capacity value corresponding to each grid from the road condition information; The normalized value of road surface smoothness and the quantitative value of bearing capacity are weighted and summed to determine the transport capacity assessment value of each grid; Connect the grid where the vehicle's starting point is located with the grid where the vehicle's end point is located to form m1 second initial candidate routes; Calculate the total value of the grid transport capacity evaluation value of each second initial candidate route, and sort the second initial candidate routes in descending order according to the total value, and select the first m2 second initial candidate routes as the second candidate routes; A route summarizing unit is used to summarize the first candidate route and the second candidate route, and perform preliminary screening to obtain multiple candidate travel routes; The optimal route determination module is used to screen the candidate routes based on the actual route information obtained by the drone and determine the optimal route; The route monitoring subsystem during vehicle driving is used to monitor road conditions in real time based on drones and satellites when the vehicle is driving on the optimal route and determine the real-time monitoring results of road conditions; The vehicle monitoring subsystem is used to obtain vehicle operating status data when the vehicle is traveling on the optimal driving route, and monitor the vehicle's own status based on the vehicle operating status data to obtain real-time monitoring results of the vehicle status; The emergency response plan determination subsystem is used to match the database emergency plan based on the real-time monitoring results of road conditions and vehicle status, and determine the emergency response plan.

2. The power grid infrastructure construction site safety dynamic management and control system based on multimodal data according to claim 1 is characterized in that: The process of performing preliminary screening to obtain multiple candidate travel routes is as follows: Performing security maintenance point screening on the first alternative path and the second alternative path respectively, if the distance between any two adjacent security maintenance points in all the security maintenance points in the first alternative path or the second alternative path is greater than a set distance threshold, then screening out the first alternative path or the second alternative path corresponding to the two security maintenance points; Obtaining environmental data for the grids that the first candidate path and the second candidate path pass through, extracting the probability of severe weather from the environmental data, and if the probability of severe weather corresponding to any grid that the first candidate path and the second candidate path pass through is greater than a set severe weather probability threshold, screening out the first candidate path or the second candidate path corresponding to the grid; The first candidate path and the second candidate path that are not eliminated are retained and recorded as to-be-selected driving routes.

3. The power grid infrastructure construction site safety dynamic management and control system based on multimodal data according to claim 1 is characterized in that: The process of screening the candidate driving routes based on the actual route information obtained by the drone and determining the optimal driving route is as follows: Parameters are extracted from the actual route information to obtain road condition characteristics and energy endurance characteristics. The road condition characteristics include the mean square error of the slope. , the probability of wild animals going out , density of low-lying areas on the road and the number of unavoidable obstacles The energy endurance characteristics include the average available resources of the maintenance points near the selected driving route and the number of available support vehicles ; Based on the road condition characteristics and the energy endurance characteristics, a path driving safety assessment is performed to obtain a safety assessment coefficient. : ； in, and are weight factors, and are all transfer functions, is the slope mean square error benchmark value, The baseline value for the probability of wild animals going out. is the density benchmark value of low-lying areas on the road surface, is the benchmark value for the number of unavoidable obstacles, To ensure the average baseline value of available resources at maintenance points, It is the benchmark value of the number of available support vehicles; The selected driving route corresponding to the largest safety assessment coefficient is recorded as the optimal driving route.

4. The power grid infrastructure construction site safety dynamic management and control system based on multimodal data according to claim 1 is characterized in that: The route monitoring subsystem during vehicle driving includes a drone monitoring module and a satellite monitoring module, wherein: The drone monitoring module is configured to obtain first-path real-time data collected by the drone during vehicle travel, if the drone is permitted to perform monitoring, diagnose and identify road conditions at a set distance ahead of the vehicle based on the first-path real-time data, and output real-time road condition monitoring results, wherein the first-path real-time data includes high-definition images and point cloud data; The satellite monitoring module is configured to obtain, when the drone is not permitted to perform monitoring, second-path live data collected by the satellite during the vehicle's travel, diagnose and identify road conditions at a set distance ahead of the vehicle based on the second-path live data, and output real-time road condition monitoring results, wherein the second-path live data includes visible light images and microwave data; The real-time monitoring results of road conditions include clear roads, partially blocked roads, and completely interrupted roads.

5. The power grid infrastructure construction site safety dynamic management and control system based on multimodal data according to claim 4 is characterized in that: The process of determining whether the drone is allowed to conduct monitoring is as follows: Obtain the drone's own operating status data. If any parameter in the drone's own operating status data is abnormal, the drone is not allowed to be monitored, or, If all parameters in the UAV's own operating status data are normal, then obtain the UAV's environmental data, including maximum wind speed, maximum temperature, and maximum electromagnetic signal strength; Obtaining a difference rate of the drone's environment based on the drone's environment data, including a wind speed difference rate, a temperature difference rate, and an electromagnetic signal strength difference rate; The wind speed difference rate, temperature difference rate and electromagnetic signal strength difference rate are weightedly summed to obtain a difference evaluation coefficient. If the difference evaluation coefficient is greater than a set evaluation threshold, the drone is not allowed to monitor, or if the difference evaluation coefficient is not greater than the set evaluation threshold, the drone is allowed to monitor.

6. The power grid infrastructure construction site safety dynamic management and control system based on multimodal data according to claim 4 is characterized in that: The process of diagnosing and identifying the road condition at a set distance ahead of the vehicle based on the first path real-time data is as follows: Inputting the pre-processed high-definition image and the point cloud data into the trained lightweight YOLO model to output the real-time road condition monitoring result; The process of diagnosing and identifying the road condition at a set distance ahead of the vehicle based on the second path real-time data is as follows: The pre-processed visible light image and the microwave data are input into a trained lightweight neural network model to output the real-time road condition monitoring result.

7. The power grid infrastructure construction site safety dynamic management and control system based on multimodal data according to claim 1 is characterized in that: The process of monitoring the vehicle's own state based on the vehicle operating state data to obtain the real-time monitoring result of the vehicle state is as follows: Perform item-by-item inspections based on the vehicle operating status data, including power system inspections, brake system inspections, tire status inspections, load conditions, and fuel and power level inspections; Compare each parameter in the vehicle operation status data with a preset rule library one by one to determine whether the parameter is abnormal and the corresponding degree of abnormality, and output a real-time monitoring result of the vehicle status; If all parameters in the vehicle operating status data are normal, prediction is performed based on the trained LSTM neural network; The process of prediction based on the trained LSTM neural network includes: Input the historical vehicle operating status data at the current moment and the previous t moments into the trained LSTM neural network, and output the predicted vehicle operating status data at the next moment; The vehicle predicted operating status data is checked item by item, and a real-time monitoring result of the vehicle status is output.

8. A control method for a power grid infrastructure construction site safety dynamic control system based on multimodal data as described in any one of claims 1 to 7, characterized in that: The following steps are involved: Acquire multimodal data from power grid infrastructure construction sites, including environmental data, road condition information, and soil condition information; Performing vehicle route planning based on the multimodal data to determine an optimal route; When the vehicle is traveling on the optimal driving route, real-time road condition monitoring is performed based on the drone and satellite to determine the real-time road condition monitoring results. When the vehicle is traveling on the optimal driving route, vehicle operation status data is obtained, and the vehicle's own state is monitored based on the vehicle operation status data to obtain the real-time vehicle state monitoring results. Based on the real-time monitoring results of the road condition and the real-time monitoring results of the vehicle status, a database emergency plan is matched to determine an emergency treatment plan.

Images