Hydropower station unmanned vehicle inspection system, method, device, equipment, medium and product

Abstract

The invention relates to the technical field of power system monitoring, and discloses a hydropower station unmanned vehicle inspection system, method, device, equipment, medium and product, the system comprises a cloud data management center, a plurality of edge computing units and a plurality of unmanned inspection vehicles; the cloud data management center is used for generating inspection task information of a plurality of areas in the hydropower station according to an equipment list and an inspection period of the hydropower station, and issuing the inspection task information of each area to the corresponding edge computing unit; each edge calculation unit is connected with at least one unmanned inspection vehicle in the corresponding area, and is used for updating the space-time task graph of the corresponding area according to the inspection task information to obtain a target space-time task graph; performing task allocation and inspection path planning on the unmanned inspection vehicles in the corresponding area based on the state data of the unmanned inspection vehicles in the corresponding area and the target space-time task graph to obtain an inspection instruction; the unmanned inspection vehicle is used for executing an inspection task according to the inspection instruction.

Description

Technical Field

[0001] This invention relates to the field of power system monitoring technology, specifically to unmanned vehicle inspection systems, methods, devices, equipment, media, and products for hydropower stations. Background Technology

[0002] With the continuous commissioning of large-scale hydropower projects, the inspection of hydropower stations has become crucial to ensure their safe operation. Currently, equipment inspection is typically conducted manually. However, the downstream access roads of hydropower stations are characterized by steep slopes, narrow spaces, and alternating hot and humid conditions, along with a high density of high-voltage electrical equipment. This complex operating environment exposes three major problems for manual inspection: first, the work is highly dangerous, with personnel facing risks such as falls and electric shocks; second, the inspection coverage is insufficient, making it difficult to achieve full-area, all-weather equipment monitoring; and third, the inspection data is highly subjective and easily influenced by personnel experience and condition. Especially under special operating conditions such as fluctuating sunlight, condensation, equipment obstruction, and signal blind spots, the accuracy and timeliness of manual inspection decrease significantly, making it difficult to meet the high requirements for the safe and stable operation of the power station. Summary of the Invention

[0003] This invention provides an unmanned vehicle inspection system for hydropower stations to solve the problems of low accuracy and efficiency in manual inspection of hydropower stations in related technologies.

[0004] In a first aspect, the present invention provides an unmanned vehicle inspection system for hydropower stations. The system includes: a cloud data management center, multiple edge computing units, and multiple unmanned inspection vehicles. The cloud data management center generates inspection task information for multiple areas within the hydropower station based on the station's equipment list and inspection cycle, and distributes this information to the corresponding edge computing units. Each edge computing unit corresponds one-to-one with each area. Each edge computing unit is connected to at least one unmanned inspection vehicle within its corresponding area. It updates the spatiotemporal task map of the corresponding area based on the inspection task information to obtain a target spatiotemporal task map. Based on the status data of the unmanned inspection vehicles within the corresponding area and the target spatiotemporal task map, it performs task allocation and inspection path planning for the unmanned inspection vehicles within the area, obtaining inspection instructions. Nodes in the spatiotemporal task map represent targets to be inspected, and edges represent passable paths between different nodes. The unmanned inspection vehicles receive inspection instructions from the edge computing units connected to them and execute inspection tasks according to the instructions.

[0005] The unmanned vehicle inspection system for hydropower stations provided by this invention utilizes a three-tiered collaborative architecture comprised of a cloud-based data management center, regionalized edge computing units, and multiple unmanned inspection vehicles. This architecture replaces manual inspections in high-risk areas such as the dam's back walkway, characterized by steep slopes, narrow spaces, and dense high-voltage equipment. It directly mitigates safety risks such as falls and electric shocks, significantly improving the safety level of power station operation and maintenance. The cloud-based data management center generates standardized inspection tasks based on equipment lists and inspection cycles, distributing them to corresponding edge computing units by region. The edge computing units combine unmanned vehicle status data with spatiotemporal task maps to perform precise task allocation and path planning. Differential path instructions enable non-overlapping collaborative operation among multiple vehicles, eliminating the problem of insufficient coverage in manual inspections and ensuring unmanned vehicles are unaffected by special operating conditions such as light fluctuations, humidity condensation, equipment obstruction, and signal blind spots. This allows for continuous, 24 / 7 inspections. Furthermore, the multimodal data collected by the unmanned inspection vehicles is objective and traceable, resolving the issues of subjective data acquisition and accuracy fluctuations due to personnel experience and condition in manual inspections.

[0006] In one optional implementation, the unmanned inspection vehicle is pre-equipped with a multimodal data acquisition device, which is used to collect multimodal data of the target to be inspected during the execution of the inspection task; the unmanned inspection vehicle is also used to send the multimodal data of the target to be inspected to the edge computing unit connected to it through the data transmission module; the edge computing unit is also used to perform anomaly identification on the target to be inspected based on the multimodal data and obtain anomaly identification results.

[0007] The system provided in this optional implementation acquires multi-dimensional data of the target to be inspected simultaneously through a multi-modal data acquisition device. This overcomes the limitations of single-sensor detection in traditional local automation methods, enabling a more comprehensive and three-dimensional capture of equipment operating status information. It avoids missed or misjudged faults due to the one-sidedness of a single data dimension. The unmanned inspection vehicle transmits the collected multi-modal data to the edge computing unit in real time, allowing for direct anomaly identification at the edge. This eliminates the need to upload a large amount of raw data to the cloud, significantly reducing data transmission bandwidth consumption. Simultaneously, it achieves millisecond-level anomaly analysis response, enabling faster detection of potential equipment faults and gaining valuable time for timely alarm triggering and fault control. Furthermore, the edge computing unit performs fusion analysis based on the multi-modal data.

[0008] In one optional implementation, the unmanned inspection vehicle has a driving mode including a tracked mode and a wheeled differential mode. The unmanned inspection vehicle is equipped with a vibration sensor, a lidar, and a main control module. When the unmanned inspection vehicle performs an inspection task, the main control module is used to control the driving mode of the unmanned inspection vehicle based on the vibration data collected by the vibration sensor and the road surface information collected by the lidar.

[0009] The system provided in this optional implementation, with its dual-mode tracked and wheeled travel, combined with vibration sensors, lidar, and intelligent control by the main control module, can precisely adapt to the inspection environment of steep slopes and complex, ever-changing road conditions on the trails behind hydropower station dams. The main control module can automatically switch travel modes based on road vibration data collected by the vibration sensors and road surface smoothness and slope information obtained by the lidar. On steep slopes, slippery surfaces, and other complex road sections, it automatically switches to tracked mode to enhance the vehicle's traction and stability, preventing slippage and getting stuck. On straight, open roads, it switches to wheeled differential mode to improve inspection efficiency and reduce overall inspection time. This intelligent switching mode requires no manual intervention, perfectly meeting the needs of unmanned inspection. It ensures high vehicle passability under complex conditions while maintaining inspection efficiency on regular road sections, effectively improving inspection coverage and task completion rate. Meanwhile, a stable driving state can avoid deviations in multimodal data acquisition caused by vehicle bumps and slippage, ensuring the accuracy and reliability of multimodal data, providing high-quality data support for subsequent anomaly identification by the edge computing unit, and further improving the overall inspection accuracy of the system.

[0010] In one optional implementation, when the unmanned inspection vehicle is performing an inspection task, the main control module of the unmanned inspection vehicle is also used to acquire environmental data within the target area, and to perform local route planning for the unmanned inspection vehicle based on the environmental data. If the local route planning is successful, the unmanned inspection vehicle is controlled to travel according to the planned local route. The target area is determined based on the position and direction of travel of the unmanned inspection vehicle.

[0011] The system provided by this optional implementation allows the main control module to define the target area and acquire environmental data based on its own position and driving direction. It can dynamically identify temporary obstacles, potholes, and other unexpected situations along the path, and promptly plan the optimal detour or passage route, preventing the vehicle from getting stuck in skidding or collisions. This significantly improves the driving safety and autonomous navigation robustness of the unmanned vehicle in complex conditions. Simultaneously, the vehicle does not need to rely entirely on the edge computing unit to issue real-time global path commands; it can autonomously adjust and optimize local paths, effectively reducing the bandwidth pressure and command transmission latency of vehicle-side communication. Even in signal blind spots, it ensures uninterrupted inspection tasks, further improving the continuity and full coverage of inspections. Furthermore, precise local route planning ensures that the unmanned vehicle drives smoothly and accurately reaches each inspection node, avoiding missed inspections due to path deviations. It also reduces the impact of vehicle body vibrations on multimodal data acquisition equipment, ensuring the stability and accuracy of the collected data and providing reliable data support for subsequent anomaly identification at the edge.

[0012] Secondly, the present invention provides a method for unmanned vehicle inspection of hydropower stations, applied to any edge computing unit of an unmanned vehicle inspection system for hydropower stations as described in the first aspect above or any corresponding embodiment. The method includes: acquiring inspection task information of a target area and status data of multiple unmanned inspection vehicles, wherein the inspection task information includes identification information and priority information of multiple targets to be inspected; updating the spatiotemporal task map of the target area based on the identification information of multiple targets to be inspected to obtain a target spatiotemporal task map; assigning tasks to multiple targets to be inspected using a genetic algorithm based on the target spatiotemporal task map, the priority information of multiple targets to be inspected, and the status data of multiple unmanned inspection vehicles to determine the set of targets to be inspected for each unmanned inspection vehicle in the target area; determining the inspection path of the corresponding unmanned inspection vehicle based on the set of targets to be inspected for each unmanned inspection vehicle and the target spatiotemporal task map; and determining the inspection command of the corresponding unmanned inspection vehicle based on the set of targets to be inspected for each unmanned inspection vehicle and the inspection path.

[0013] The unmanned vehicle inspection method for hydropower stations provided by this invention relies on edge computing units to intelligently allocate and plan the path of inspection tasks for target areas locally. By acquiring inspection task information and unmanned inspection vehicle status data, and updating the spatiotemporal task map with the identifiers of the targets to be inspected, the task planning can be made to perfectly match the actual inspection needs of the target area, ensuring that no targets are missed. Based on the target spatiotemporal task map, task priority, and vehicle status data, a genetic algorithm is used to allocate tasks, which can prioritize the inspection needs of high-risk and high-priority equipment. At the same time, the remaining battery power and current location of the vehicles are combined to reasonably delineate the set of targets to be inspected for each vehicle, avoiding task interruptions due to mismatched vehicle status, and improving the feasibility and rationality of task execution. Based on this, the inspection paths and instructions for each vehicle are further determined, enabling collaborative operation of multiple unmanned inspection vehicles. This effectively avoids problems such as overlapping vehicle paths and regional conflicts, significantly improving inspection efficiency and reducing blind spots. At the same time, the planning calculations are completed locally on the edge side, without relying on a large amount of computing power from the cloud, significantly reducing cloud-edge communication bandwidth consumption, improving the response speed of instruction issuance, adapting to the real-time inspection needs of the complex environment of the dam trail at hydropower stations, providing precise guidance for unmanned vehicles to efficiently execute inspection tasks, and further ensuring the intelligent and efficient operation of the entire inspection system.

[0014] In an optional implementation, the method further includes: acquiring multimodal data of multiple targets to be inspected fed back by an unmanned inspection vehicle, the multimodal data including visible light image data acquired by a visible light camera, thermal image data acquired by an infrared thermal imager, voiceprint data acquired by a sound acquisition sensor, and target gas concentration acquired by a gas sensor; using a pre-built semantic segmentation model to identify key monitoring areas in the visible light image data of each target to be inspected, and determining at least one region of interest (ROI) corresponding to the visible light image data; based on the ROI of each target to be inspected, determining target thermal image data in the corresponding thermal image data, the target thermal image data being the data corresponding to the ROI in the thermal image data; performing anomaly detection on the ROI of the target to be inspected based on the target thermal image data, obtaining a first anomaly detection result for the target to be inspected; performing anomaly detection on the target to be inspected based on the voiceprint data, obtaining a second anomaly detection result for the target to be inspected; and performing anomaly detection on the environment where the target to be inspected is located based on the target gas concentration, obtaining a third anomaly detection result for the target to be inspected.

[0015] The method provided in this optional implementation accurately identifies key monitoring areas and determines regions of interest (ROIs) of the target to be inspected from visible light images using a semantic segmentation model. This effectively filters out interference from complex backgrounds, allowing subsequent infrared, acoustic signature, and gas concentration detection to focus on the core components of the equipment, avoiding the impact of invalid data on detection efficiency. By matching the target thermal image data with the ROI, precise spatial alignment of visual and infrared data is achieved, significantly improving the targeting and accuracy of temperature rise anomaly detection. Furthermore, by combining acoustic signature data and target gas concentration data for multi-dimensional anomaly detection, the limitations of single-data-dimensional detection are overcome. This allows for the comprehensive capture of different types of fault characteristics, such as equipment temperature rise, acoustic abnormalities, and gas leaks, effectively reducing the probability of missed fault detection and misjudgment.

[0016] Thirdly, the present invention provides an unmanned vehicle inspection device for hydropower stations, applied to any edge computing unit of the unmanned vehicle inspection system for hydropower stations described in the first aspect or any corresponding embodiment. The device includes: a first acquisition module, used to acquire inspection task information of a target area and status data of multiple unmanned inspection vehicles, wherein the inspection task information includes identification information and priority information of multiple targets to be inspected; an update module, used to update the spatiotemporal task map of the target area based on the identification information of multiple targets to be inspected, to obtain a target spatiotemporal task map; a task allocation module, used to allocate tasks to multiple targets to be inspected using a genetic algorithm based on the target spatiotemporal task map, the priority information of multiple targets to be inspected, and the status data of multiple unmanned inspection vehicles, to determine the set of targets to be inspected for each unmanned inspection vehicle in the target area; a first determination module, used to determine the inspection path of the corresponding unmanned inspection vehicle based on the set of targets to be inspected for each unmanned inspection vehicle and the target spatiotemporal task map; and a second determination module, used to determine the inspection instruction of the corresponding unmanned inspection vehicle based on the set of targets to be inspected for each unmanned inspection vehicle and the inspection path.

[0017] Fourthly, the present invention provides an electronic device, comprising: a memory and a processor, wherein the memory and the processor are communicatively connected to each other, the memory stores computer instructions, and the processor executes the computer instructions to perform the unmanned vehicle inspection method for hydropower stations described in the second aspect or its corresponding embodiments.

[0018] Fifthly, the present invention provides a computer-readable storage medium storing computer instructions, which are used to cause a computer to execute the unmanned vehicle inspection method for hydropower stations described in the second aspect or its corresponding embodiments.

[0019] Sixthly, the present invention provides a computer program product, including computer instructions, which are used to cause a computer to execute the unmanned vehicle inspection method for hydropower stations described in the second aspect or its corresponding embodiments. Attached Figure Description

[0020] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0021] Figure 1 This is a structural block diagram of an unmanned vehicle inspection system for hydropower stations according to an embodiment of the present invention;

[0022] Figure 2This is a flowchart illustrating the unmanned vehicle inspection method for hydropower stations according to an embodiment of the present invention. Figure 3 This is a structural block diagram of an unmanned vehicle inspection device for hydropower stations according to an embodiment of the present invention; Figure 4 This is a schematic diagram of the hardware structure of an electronic device according to an embodiment of the present invention. Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0024] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0025] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0026] In related technologies, hydropower station equipment is generally inspected manually. However, the access roads downstream of hydropower stations are characterized by steep slopes, narrow spaces, and alternating hot and humid conditions, and are densely populated with high-voltage electrical equipment. This complex operating environment exposes three major problems for manual inspections: First, the work is highly dangerous, with personnel facing safety risks such as falls and electric shocks; second, the inspection coverage is insufficient, making it difficult to achieve full-area, all-weather equipment monitoring; and third, the inspection data is highly subjective and easily affected by personnel experience and condition. Especially under special operating conditions such as fluctuating light, condensation, equipment obstruction, and signal blind spots, the accuracy and timeliness of manual inspections drop significantly, making it difficult to meet the high requirements for the safe and stable operation of the power station.

[0027] In view of this, this application provides an unmanned vehicle inspection system for hydropower stations. Through a three-tiered collaborative architecture consisting of a cloud data management center, regionalized edge computing units, and multiple unmanned inspection vehicles, it replaces manual inspections in high-risk areas such as the dam's back walkway—areas with steep slopes, narrow spaces, and dense high-voltage equipment. This directly avoids safety risks such as falls and electric shocks, significantly improving the safety level of power station operation and maintenance. The cloud data management center generates standardized inspection tasks based on the equipment list and inspection cycle and distributes them to the corresponding edge computing units by region. The edge units combine unmanned vehicle status data with spatiotemporal task maps to perform precise task allocation and path planning. Differential path instructions enable non-overlapping collaborative operation among multiple vehicles, eliminating the problem of insufficient coverage in manual inspections and ensuring unmanned vehicles are unaffected by special operating conditions such as light fluctuations, humidity condensation, equipment obstruction, and signal blind spots, achieving continuous 24 / 7 inspections. Furthermore, the multimodal data collected by the unmanned inspection vehicles is objective and traceable, solving the problems of subjective data acquisition and accuracy fluctuations due to personnel experience and condition in manual inspections.

[0028] This embodiment provides an unmanned vehicle inspection system for hydropower stations, such as... Figure 1 As shown, the system includes: a cloud data management center 101, multiple edge computing units 102, and multiple unmanned inspection vehicles 103; The cloud data management center 101 is used to generate inspection task information for multiple areas of the hydropower station based on the equipment list and inspection cycle of the hydropower station, and to send the inspection task information of each area to the corresponding edge computing unit 102. Multiple edge computing units 102 correspond one-to-one with multiple areas.

[0029] For example, a hydropower station includes multiple areas, which are management areas divided according to the physical area of ​​the hydropower station. Multiple edge computing units 102 correspond one-to-one with multiple areas, and each area is equipped with at least one unmanned inspection vehicle 103. Each edge computing unit 102 manages the unmanned inspection vehicle 103 in its corresponding area. The equipment list is used to represent information about all equipment in the hydropower station, and the inspection cycle can be determined according to actual needs; this embodiment does not impose specific limitations.

[0030] Each edge computing unit 102 is connected to at least one unmanned inspection vehicle 103 in the corresponding area. It is used to update the spatiotemporal task map of the corresponding area according to the inspection task information to obtain the target spatiotemporal task map. Based on the status data of the unmanned inspection vehicle 103 in the corresponding area and the target spatiotemporal task map, it performs task allocation and inspection path planning for the unmanned inspection vehicle 103 in the area to obtain inspection instructions. The nodes in the spatiotemporal task map represent the targets to be inspected, and the edges represent the passable paths between different nodes.

[0031] For example, an area contains at least one target to be inspected, which may include, but is not limited to, equipment to be inspected and key points of the passageway. The targets to be inspected within the area are treated as nodes, and the traversable paths between different nodes are treated as edges, thus constructing a spatiotemporal task graph. When the edge computing unit 102 receives inspection task information for the corresponding area, it updates the initial spatiotemporal task graph based on the identification information of the targets to be inspected contained in the inspection task information, obtaining a target spatiotemporal task graph. The status data of the unmanned inspection vehicle 103 may include, but is not limited to, remaining battery power, current location, load status (number of assigned tasks), and equipment health status (whether the sensors are functioning properly). Based on the status data of each unmanned inspection vehicle 103 and the target spatiotemporal task graph, tasks are assigned and inspection paths are planned for the unmanned inspection vehicles 103 within the area, resulting in inspection instructions. This embodiment of the application does not limit the content of task assignment and inspection path planning; those skilled in the art can determine this according to their needs.

[0032] The unmanned inspection vehicle 103 is used to receive inspection instructions issued by the edge computing unit 102 connected to it, and to execute inspection tasks according to the inspection instructions. For example, after receiving the inspection instructions, the unmanned inspection vehicle 103 completes the inspection tasks according to the inspection instructions.

[0033] The unmanned vehicle inspection system for hydropower stations provided in this application embodiment utilizes a three-tiered collaborative architecture comprised of a cloud-based data management center, regionalized edge computing units, and multiple unmanned inspection vehicles. This architecture replaces manual inspections in high-risk areas such as the dam's back walkway, characterized by steep slopes, narrow spaces, and dense high-voltage equipment. It directly mitigates safety risks such as falls and electric shocks, significantly improving the safety level of power station operation and maintenance. The cloud-based data management center generates standardized inspection tasks based on the equipment list and inspection cycle, distributing them to corresponding edge computing units by region. The edge computing units combine unmanned vehicle status data with spatiotemporal task maps to perform precise task allocation and path planning. Differential path instructions enable non-overlapping collaborative operation among multiple vehicles, eliminating the problem of insufficient coverage in manual inspections and ensuring unmanned vehicles are unaffected by special operating conditions such as light fluctuations, humidity condensation, equipment obstruction, and signal blind spots. This allows for continuous, 24 / 7 inspections. Furthermore, the multimodal data collected by the unmanned inspection vehicles is objective and traceable, resolving the issues of subjective data acquisition and accuracy fluctuations due to personnel experience and condition.

[0034] In some optional implementations, the unmanned inspection vehicle 103 is pre-equipped with a multimodal data acquisition device, which is used to collect multimodal data of the target to be inspected during the execution of the inspection task.

[0035] For example, the multimodal data acquisition device may include, but is not limited to, a visible light camera, an infrared thermal imager, a gas sensor, and a sound acquisition sensor. The visible light camera can acquire image data of the target to be inspected, the infrared thermal imager can acquire temperature data of the target, the gas sensor can acquire the concentrations of gases such as hydrogen sulfide, carbon monoxide, and methane in the environment where the target is located, and the sound acquisition sensor can acquire sound data of the environment where the target is located. In this embodiment, the gas sensor may include, but is not limited to, an electrochemical probe, and the sound acquisition sensor may include, but is not limited to, a microphone array.

[0036] The unmanned inspection vehicle 103 is also used to send multimodal data of the target to be inspected to the edge computing unit connected to it via a data transmission module.

[0037] For example, the data transmission module can be a pre-configured module with data transmission function in the unmanned inspection vehicle 103. The specific content of the data transmission module is not limited in this application embodiment, as long as it can realize data transmission.

[0038] The edge computing unit 102 is also used to perform anomaly identification on the target to be inspected based on multimodal data, and obtain anomaly identification results. For example, in this embodiment of the application, the edge computing unit 102 first accurately associates data such as visible light images, infrared thermal images, acoustic prints, and gas concentrations with the same part of the same target to be inspected based on timestamps and device location coordinates. Then, it performs single-dimensional screening by channel, filters background interference through visual models, judges whether the temperature rise exceeds the threshold through infrared data, calculates the anomaly degree through acoustic autoencoders, and compares the gas concentration with safety standards. Finally, it uses comprehensive logic to determine the anomaly, and does not draw conclusions based on data from a single channel. Finally, it outputs anomaly identification results containing information such as anomaly type, device ID, and specific location.

[0039] In some optional implementations, the unmanned inspection vehicle 103 has two driving modes: tracked mode and wheeled differential mode. The unmanned inspection vehicle 103 integrates vibration sensors, a lidar, and a main control module. Exemplarily, the vibration sensor is used to collect vibration data during the driving process of the unmanned inspection vehicle 103, and the lidar is used to collect information on the smoothness of the road surface during the driving process. The main control module may be a pre-installed central processing unit in the unmanned inspection vehicle. In this embodiment, the vibration sensor may include, but is not limited to, an inertial measurement unit (IMU). The unmanned inspection vehicle 103 adopts a magnesium-aluminum alloy load-bearing frame, and the chassis integrates a tracked-wheel composite drive mechanism. The tracked mode enhances traction on steep slopes and slippery surfaces, while the wheeled differential mode provides high-speed maneuverability on straight roads. The double-link suspension and the inertial measurement unit form a closed-loop control, continuously adjusting damping during driving to suppress yaw and pitch angles. The power system consists of a hub motor, a drive controller, and a lithium iron phosphate battery pack. The battery management unit monitors the state of charge (SOC), temperature, and cell voltage difference of the battery pack via the CAN-FD bus. When the SOC is lower than the set threshold, the vehicle triggers the automatic charging docking logic, uses visual servoing and electromagnetic alignment to align with the wireless charging coupling device, and starts high-frequency electromagnetic induction charging.

[0040] When the unmanned inspection vehicle performs inspection tasks, the main control module controls the vehicle's driving mode based on vibration data collected by vibration sensors and road surface information collected by lidar. For example, in this embodiment, when the road slope is greater than 15 degrees or the road surface is slippery (determined by vibration frequency detection via IMU), the vehicle switches to tracked mode to enhance traction. When the road slope is less than or equal to 15 degrees and the road surface is straight (road surface smoothness detected by lidar), the vehicle switches to wheeled differential mode to increase driving speed.

[0041] In some optional implementations, when the unmanned inspection vehicle is performing an inspection task, the main control module of the unmanned inspection vehicle is also used to acquire environmental data within the target area, perform local route planning for the unmanned inspection vehicle based on the environmental data, and control the unmanned inspection vehicle to travel according to the planned local route if the local route planning is successful.

[0042] For example, the target range is determined based on the location and driving direction of the unmanned inspection vehicle. The specific content of the target range is not limited in this application embodiment, and those skilled in the art can determine it according to their needs. In this application embodiment, the global path is planned by the edge computing unit 102. The global path includes the sequence of nodes to be inspected, road segment attributes (slope, road surface grade), time window constraints and other information, which serve as the basic framework for local planning. The vehicle-mounted multi-sensor collects environmental data in real time and fuses it to generate three types of core constraint inputs, which provide a basis for local planning: (1) Occupied grid map: generated by 16-line lidar point cloud data, marking the location and outline of obstacles; (2) Semantic mask: output by YOLOv8-Seg semantic segmentation model, marking "pass / no entry / danger" areas; (3) Dynamic obstacle map: fused with data collected by visible light camera and lidar, updating the location of dynamic targets such as personnel and other vehicles in real time.

[0043] Subsequently, a rolling temporal window strategy is adopted, setting a distance horizon of 20–40 meters or a time horizon of 2–3 seconds along the global path with the vehicle's current position as the center. The window range is adaptively adjusted with vehicle speed, and only the path within the window is replanned. Then, the planning result of the previous frame or the current global road segment is used as a warm-start initialization algorithm, and nonholonomic kinematic constraints on the vehicle's minimum turning radius and safety corridor constraints composed of channel boundaries and restricted areas are applied. By integrating the cost function of global path deviation, obstacle distance, path curvature change, passable width and time window deviation, incremental path search is performed within the window to generate the optimal local alternative path. When a new obstacle enters the safety radius, the current trajectory becomes infeasible, the positioning uncertainty increases, or the communication lag exceeds the threshold, replanning is triggered. After successful planning, the main control module controls the vehicle to execute the path, and at the same time, the differential information of the path change is uploaded to the edge node to update the spatiotemporal task graph. If continuous planning fails, a road segment reallocation is requested from the edge or a temporary avoidance strategy is executed.

[0044] The unmanned vehicle inspection system for hydropower stations provided by the present invention will be specifically described below through a specific embodiment.

[0045] Example: In this embodiment, the unmanned vehicle inspection system for hydropower stations is deployed in a medium-sized hydropower station with 2×350MW units in a valley. It is constructed according to the three-level collaborative architecture of "cloud data management center - plant edge computing unit - mobile unmanned inspection vehicle". The hardware structure, electrical connection relationship and operation method of each component are as follows.

[0046] The cloud server is located in a dedicated data center on the fourth floor of the operations and maintenance building, equipped with two 42U racks. Each rack houses six 2U learning servers, interconnected in a leaf-spine structure using a 100GbE fiber optic backplane switch, and aggregated to the factory's core switch via dual-link 40GbE fiber optic cables. All servers are installed with a KubeSphere-based container orchestration platform; the data lake uses a Ceph+MinIO solution, powered by a dual-input 60kVA modular UPS, with power supplied to the two rows of servers via power distribution units (PDUs) to ensure N+1 redundancy. The cloud server utilizes GPU acceleration for incremental training of multimodal tensors, generating Open Neural Network Exchange Format (ONNX) models; after training, the models are converted into TensorRT engine files using a self-developed hot-swap script and pushed to edge nodes via an MQTT / TLS encrypted tunnel. At this point, a TLS 1.3 two-way certificate ensures the model's identity integrity and data confidentiality during transmission.

[0047] The edge computing unit is installed in an IP65 industrial cabinet in a corner of the duty room on the ground floor of the main plant. Its core is a dual-board redundant architecture consisting of a development board and an edge security unit, directly connected via PCIeGen3×4. The AGX board handles high-speed inference, while the i.MX8M board runs a whitelist firewall and a time synchronization daemon. An 8-port Gigabit PoE+ switch is integrated at the top of the cabinet: ports 1 and 2 are connected to two ceiling-mounted Wi-Fi 6APs (access points), ports 3 and 4 are connected to the core switch via photoelectric conversion, and the remaining ports are reserved for a programmable logic controller (PLC) and an OPC-UA server. To improve the reliability of the edge-to-vehicle link, a 5G NR industrial router is installed simultaneously. When the Wi-Fi signal RSSI falls below the threshold, the i.MX8M board uses the SD-WAN module to switch links for MQTT messages, ensuring uninterrupted service.

[0048] The edge-side software stack is loaded with TensorRT-8, OpenVINO-2023.2, and Redis-7 using Docker. When the protobuf stream uploaded by the autonomous vehicle enters the AGX board's network port, the Zero-Copy mechanism directly maps the frame buffer to the GPU memory, achieving 40ms-level inference using tensor rearrangement. If the temperature rise in the inference result exceeds 85℃ or the combustible gas concentration exceeds the limit, the i.MX8M board immediately writes a boolean variable to the plant control system via the OPC-UA client, and simultaneously inserts the event index into the InfluxDB time series database. The average latency of the above edge-cloud pipeline, measured by OTDR (Optical Time Domain Reflectometry), is 176ms.

[0049] The unmanned inspection vehicle is 780mm long, 540mm wide, and weighs 38kg. The chassis is a T6-6061 magnesium-aluminum die-cast unibody component. Two 65mm wide rubber tracks are mounted on the inner side of the chassis, while four hub-drive motors (rated 150W) are located on the outer side. Track-wheel switching is achieved via an electromagnetic sliding key coupling, driven by a 24V DC electromagnetic coil. A centrally mounted "W-type" double-link adjustable damping shock absorber is installed. The IMU feeds real-time attitude feedback to the STM32F4 main control MCU via the CAN-FD bus, adjusting the PWM duty cycle of the damping coil in a 200Hz closed-loop manner. The main power supply is a 48V / 25Ah lithium iron phosphate battery pack. The BMS collects 15 channels of individual cell voltage, 3 channels of temperature, and bus current, sending them to the main control MCU via CAN-FD messages every 100ms. When the SOC drops to 15%, the MCU calls ROS2Action to trigger a back-to-base action.

[0050] A 16-line rotating LiDAR with a scanning frequency of 20Hz is mounted at the center of the vehicle's top. A D435iRGB-D binocular camera is fixed to the front gimbal, driven by an MG996R servo motor, achieving ±45° pitch. An infrared thermal imager and a gas probe are positioned side-by-side on the left side of the chassis, while a six-element MEMS microphone and a bidirectional fisheye camera are located on the right. An ultrasonic ranging module is embedded in each of the four corners of the chassis. All the above sensors are connected to the "P1 (signal) / P2 (power)" connector via two 8-core shielded cable connectors: P1 is a 1GbE + 2×I²C + 1×CAN-FD signal bus, and P2 is a 48V main power output and a 12VDC-DC branch. The vehicle-mounted edge computing board JetsonOrin-NX is directly connected to P1 via the 1GbE port, the I²C bus connects to the microphone array and the gas probe, and the CAN-FD bus connects to the BMS, IMU, and motor driver, achieving data synchronization in a unified clock domain.

[0051] Vehicle-to-side baseline communication uses an Intel AX210 Wi-Fi 6 module in the 5.180-5.825GHz band; if RSSI < -75dBm, the NetworkManager service triggers 802.11s Mesh mode, allowing multiple vehicles to act as relays. Emergency links are handled by a Quectel RG520N-GL5G modem, compatible with eSIM industrial cards; the Wi-Fi and 5G modules are connected to the same Mini-PCIe expansion board via an M.2 interface, sharing a USB-3.1 bus. The wireless charging device operates at an 85kHz resonant frequency, with the primary coil fixed inside the parking pile and the secondary coil embedded in the vehicle chassis. An NXPTEA2017 resonant controller drives the full-bridge MOSFETs. The vehicle detects pitch attitude via a lower-mounted IMU, combining the reversing camera image with ArUco markings to achieve a ±5mm alignment accuracy; when the MCU is in I... 2The TEA2017 “PowerGood” signal was detected on the C bus and the communication frame verification passed. The BMS then started the constant current-constant voltage charging curve.

[0052] According to an embodiment of the present invention, an embodiment of an unmanned vehicle inspection method for hydropower stations is also provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0053] This embodiment provides a method for inspecting unmanned vehicles in hydropower stations, which can be used in any edge computing unit of the unmanned vehicle inspection system in the above embodiments. Figure 2 This is a flowchart of an unmanned vehicle inspection method for hydropower stations according to an embodiment of the present invention, such as... Figure 2 As shown, the process includes the following steps: Step S201: Obtain inspection task information for the target area and status data of multiple unmanned inspection vehicles. The inspection task information includes the identification information and priority information of multiple targets to be inspected.

[0054] For example, the target area refers to the specific hydropower station area managed by the edge computing unit, which is a geographical partition corresponding one-to-one with the edge computing unit. Inspection task information is a set of instructions generated and issued by the cloud data management center based on the hydropower station equipment list and preset inspection cycles, serving as the basis for task planning on the edge side. The identification information of the target to be inspected is the unique identification data for each key point of the equipment or channel to be inspected, which may include, but is not limited to, equipment ID, type, and spatial coordinates, used to accurately distinguish different inspection objects and avoid task omissions or duplications. The priority information of the target to be inspected is the inspection sequence set according to factors such as the importance of the equipment, the level of operational risk, and the existence of historical faults. For example, core equipment such as high-voltage switchgear and generators have a higher priority than ordinary pipelines and auxiliary components such as supports. The status data of multiple unmanned inspection vehicles are the real-time operating parameters of all unmanned inspection vehicles connected to the edge computing unit, covering the vehicle's remaining battery power, current location, assigned tasks, sensor operating status, and load status, serving as an important constraint for the edge side to rationally allocate tasks.

[0055] Step S202: Update the spatiotemporal task map of the target area based on the identification information of multiple targets to be inspected, and obtain the target spatiotemporal task map.

[0056] For example, the initial spatiotemporal task map before the update contains the usual nodes to be inspected in the area (corresponding to equipment and key points of the passage) and accessible path edges. The identification information of the targets to be inspected includes key data such as the unique ID, location coordinates, and equipment type of each target. The edge computing unit will make targeted adjustments to the initial map based on this identification information. If there are new targets to be inspected that are not included in the initial map, they will be added to the node set of the map as new nodes. If some existing nodes of the initial map are missing from the identification information, it means that the corresponding target does not need to be included in this inspection, and it will be removed from the node set. At the same time, the basic attributes of the relevant nodes will be updated synchronously according to the identification information to ensure that the nodes in the map completely correspond to the current inspection task. The target spatiotemporal task map obtained after this adjustment eliminates redundant or missing nodes that do not match the current task, providing a precise and practical digital carrier for subsequent multi-vehicle task allocation and path planning based on this map.

[0057] Step S203: Based on the target spatiotemporal task map, the priority information of multiple targets to be inspected, and the status data of multiple unmanned inspection vehicles, a genetic algorithm is used to assign tasks to multiple targets to be inspected, and the set of targets to be inspected for each unmanned inspection vehicle in the target area is determined.

[0058] For example, in the embodiments of this application, the "task-vehicle-access sequence" triple is encoded as a chromosome (e.g., "vehicle 1-device A-device C-device E" is a chromosome); the initial chromosome group is constructed according to heuristic rules (high task priority → priority allocation, adjacent device space → grouped into the same subgraph, high event risk density → priority coverage).

[0059] The fitness function is shown in the following formula:

[0060] in, Indicates suitability; , , , as well as This represents the weighting coefficient, used to adjust the importance of each indicator in the evaluation; This represents the total cost of travel, taking into account the combined travel distance, gradient, and road surface grade. This represents the total penalty for the priority level of the targets to be inspected; if high-risk targets are not prioritized, points are added. This represents the total penalty for breach of the time window; penalties are incurred for arriving early or late at the inspection node. This represents the total penalty for insufficient vehicle battery charge (SOC) constraints; points are awarded if the task cannot be completed due to insufficient battery charge. This represents the total penalty for multiple vehicle collisions; additional points are awarded if vehicles occupy the same road segment or node simultaneously.

[0061] The genetic algorithm employs tournament selection (screening chromosomes with low fitness values), sequential crossover (OX), or partial mapping crossover (PMX), combined with neighborhood search methods such as swapping and two-beam optimization (2-opt) to obtain a candidate node sequence for each vehicle. Each sequence is then refined using an ant colony algorithm, with path optimization guided by pheromones. Pheromones and edge weights are jointly updated based on factors such as road congestion, gradient, risk, vehicle load, and battery level. Simultaneously, time windows and minimum distance constraints are applied to simultaneous occupancy on the same or adjacent road segments, ultimately generating a subgraph V_k for each inspection vehicle (the set of targets to be inspected for each vehicle).

[0062] Step S204: Determine the inspection path of the corresponding unmanned inspection vehicle based on the set of targets to be inspected for each unmanned inspection vehicle and the spatiotemporal task map of the targets.

[0063] For example, in this embodiment of the application, the inspection path of the unmanned inspection vehicle is optimized using the ant colony algorithm. The specific optimization process is as follows: Let n be the total number of target nodes to be inspected by a single unmanned inspection vehicle, where i represents the current node and j represents the next node to be visited; the actual travel cost between any two nodes i and j is denoted by n. express, It's not just about distance, but a comprehensive cost that fits the scenario of the horse trail behind the dam = physical distance between two points + road slope loss + road surface slippery risk value (the larger the value, the more difficult and time-consuming the section from i to j is to travel). Let t represent the pheromone concentration on the path from node i to node j at time t. The higher the concentration, the greater the probability that this path will be selected later. The degree of heuristic from node i to node j represents the ant's innate tendency to choose this path; each ant has its own taboo list. The tabu list is recorded to ensure that each inspected node is visited only once and there are no duplicate inspections. The tabu list is cleared after all inspected nodes have been traversed. The degree of heuristic can be calculated using the following formula:

[0064] in, This represents the distance from node i to node j; , This represents a weighting coefficient, which allows ants to prioritize paths that are close, have a gentle slope, and are low-risk, thus adapting to the complex environment of the horse trail behind the dam. Indicates the slope of the road section. This indicates the node's risk level, which is determined based on the node's priority information.

[0065] A strategy of "local evaporation + global optimal deposition" is adopted, and a congestion suppression factor is superimposed. Pheromones are updated (to guide optimal path convergence). Minimum safety intervals are set for road segments. (e.g., 3 seconds), requiring that the time difference between multiple vehicles passing through the same road segment is not less than [amount missing]. At the same time, time window constraints are imposed, requiring arrival at the node within a specified time to ensure the timeliness and safety of the inspection.

[0066] After multiple rounds of iterative convergence, the optimal node access sequence for each unmanned inspection vehicle is obtained. And the corresponding time constraints, thus obtaining the inspection path.

[0067] Step S205: Determine the inspection instructions for the corresponding unmanned inspection vehicle based on the set of targets to be inspected and the inspection path of each unmanned inspection vehicle.

[0068] For example, in this embodiment of the application, the newly generated The route issued for this vehicle in the previous round Perform sequence comparisons to find the minimum edit script. The final result is a differential path instruction package containing "path change + time window update". Only lightweight changes are issued to save communication bandwidth, while supporting breakpoint resumption and adapting to signal fluctuation scenarios. If a temporary obstacle or task priority change occurs during execution, the edge side will recalculate and issue a minimum differential patch within the rolling time domain T. If local conflicts cannot be resolved, the task will be reassigned back to the genetic layer to ensure that the overall inspection task is completed on time.

[0069] The unmanned vehicle inspection method for hydropower stations provided in this embodiment relies on edge computing units to intelligently allocate and plan the path of inspection tasks for the target area locally. By acquiring inspection task information and unmanned inspection vehicle status data, and combining the updated spatiotemporal task map with the identifiers of the targets to be inspected, the task planning can be made to perfectly match the actual inspection needs of the target area, ensuring that no targets are missed. Based on the target spatiotemporal task map, task priority, and vehicle status data, a genetic algorithm is used to allocate tasks, which can prioritize the inspection needs of high-risk and high-priority equipment. At the same time, the set of targets to be inspected for each vehicle is reasonably defined by combining the remaining battery power and current location of the vehicles, avoiding task interruptions due to mismatched vehicle status, and improving the feasibility and rationality of task execution. Based on this, the inspection paths and instructions for each vehicle are further determined, enabling collaborative operation of multiple unmanned inspection vehicles. This effectively avoids problems such as overlapping vehicle paths and regional conflicts, significantly improving inspection efficiency and reducing blind spots. At the same time, the planning calculations are completed locally on the edge side, without relying on a large amount of computing power from the cloud, significantly reducing cloud-edge communication bandwidth consumption, improving the response speed of instruction issuance, adapting to the real-time inspection needs of the complex environment of the dam trail at hydropower stations, providing precise guidance for unmanned vehicles to efficiently execute inspection tasks, and further ensuring the intelligent and efficient operation of the entire inspection system.

[0070] In some optional implementations, the above method further includes the following steps: Step a1: Obtain multimodal data of multiple targets to be inspected from the feedback of the unmanned inspection vehicle. The multimodal data includes visible light image data collected by the visible light camera, thermal image data collected by the infrared thermal imager, acoustic fingerprint data collected by the sound acquisition sensor, and target gas concentration collected by the gas sensor.

[0071] For example, please refer to the description of the relevant content in the system embodiment, which will not be repeated here.

[0072] Step a2: Use a pre-built semantic segmentation model to identify key monitoring areas in the visible light image data of each target to be inspected, and determine at least one region of interest in the corresponding visible light image data.

[0073] For example, in the application embodiment, when pre-constructing the semantic segmentation model, a large number of visible light image samples of typical targets to be inspected in hydropower stations (such as switchgear, generators, valves, busbars, etc.) are collected. Pixel-level annotations are performed on key monitoring areas (such as the instrument display area of ​​the switchgear, the valve stem connection of the valve, and the wiring terminals of the generator) and background areas (such as walls, ground, and unrelated supports) in each sample image to form an annotated dataset. This dataset is then input into a network model suitable for small target segmentation in industrial equipment, such as U-Net or its improved versions, for training. Through iterative optimization of model parameters, the model is made capable of accurately distinguishing between "the target body to be inspected - key monitoring areas - background" in the image. After training, the model is deployed to the edge computing unit. When the unmanned inspection vehicle uploads the collected visible light image data of the target to be inspected to the edge computing unit, the edge computing unit... The computational unit first preprocesses the image, including denoising, illumination equalization, and size normalization, to eliminate interference with image quality caused by factors such as uneven lighting at the hydropower station and dust covering the equipment surface. Then, the preprocessed image is input into the deployed semantic segmentation model, which performs pixel-level classification calculations and outputs a classification mask map with the same size as the original image. Different pixel values ​​in the mask map correspond to key monitoring areas, non-key equipment areas, and background areas, respectively. Finally, the edge computing unit extracts the pixel set of all corresponding key monitoring areas based on the pixel classification results of the mask map and defines it as the region of interest of the target to be inspected. Since a single device may have multiple parts that need to be monitored, at least one region of interest will be determined in the end, providing accurate spatial positioning basis for matching the corresponding area in the infrared thermal image data and carrying out targeted anomaly detection.

[0074] Step a3: Based on the region of interest of each target to be inspected, determine the target thermal image data in the corresponding thermal image data. The target thermal image data is the data corresponding to the region of interest in the thermal image data.

[0075] For example, the unmanned inspection vehicle simultaneously collects visible light images and thermal image data of the target to be inspected (associated data bound to the same target through timestamps and device locations). The onboard visible light camera and infrared thermal imager have been pre-calibrated spatially and have a fixed pixel coordinate mapping relationship. The edge computing unit first associates the visible light image with the region of interest mask with the corresponding thermal image data of the target to be inspected through the device ID and timestamp. Then, using the preset coordinate mapping relationship, the pixel coordinates of the "region of interest" in the visible light image are converted into the pixel coordinates of the thermal image data. Finally, the area corresponding to the coordinates is extracted from the thermal image data, and the target thermal image data is obtained.

[0076] Step a4: Based on the target thermal image data, perform anomaly detection on the region of interest of the target to be inspected to obtain the first anomaly detection result for the corresponding target.

[0077] For example, the edge computing unit first retrieves the preset normal temperature threshold (different equipment parts have specific thresholds) for the area of ​​interest of the target to be inspected, then analyzes the target thermal image data, extracts the core temperature parameters such as the highest temperature and average temperature of the area, compares these parameters with the preset threshold, and if the temperature exceeds the threshold range (such as the temperature of the switch cabinet terminal exceeding 80°C), it is determined to be an abnormal temperature rise, and the abnormal temperature, area location and other information are recorded simultaneously, and finally a first abnormality detection result containing the abnormality type and confidence level is generated.

[0078] Step a5: Based on the voiceprint data, perform anomaly detection on the target to be inspected to obtain the second anomaly detection result for the target to be inspected.

[0079] For example, in this embodiment of the application, the edge computing unit first converts the voiceprint data into frequency domain spectral features, inputs them into a spectral autoencoder that has been pre-trained with the normal voiceprint of the device, and obtains the reconstructed spectrum; calculates the error between the original spectrum and the reconstructed spectrum, i.e. the anomaly index, and compares it with a preset normal error threshold. If the error exceeds the threshold, it is determined that there are acoustic anomalies such as bearing squealing or valve leakage, and finally generates a second anomaly detection result containing the anomaly type.

[0080] Step a6: Based on the target gas concentration, perform anomaly detection on the environment where the target to be inspected is located to obtain the third anomaly detection result for the target to be inspected.

[0081] For example, the edge computing unit first retrieves the preset safe concentration threshold of the target gas in the area where the target to be inspected is located. The preset safe concentration thresholds are different for different equipment areas. For example, the combustible gas thresholds in the high-voltage cabinet area and the ordinary pipeline area are different. Then, the real-time target gas concentration data collected by the gas sensor is compared with the threshold. If the real-time concentration exceeds the threshold or the concentration change rate exceeds the normal range, it is determined that the environmental gas is abnormal. Finally, a third abnormality detection result containing the abnormal gas type, concentration value, and degree of exceedance is generated.

[0082] This embodiment also provides an unmanned vehicle inspection device for hydropower stations, which is used to implement the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0083] This embodiment provides an unmanned vehicle inspection device for hydropower stations, such as... Figure 3 As shown, it includes: The first acquisition module 301 is used to acquire inspection task information of the target area and status data of multiple unmanned inspection vehicles. The inspection task information includes the identification information and priority information of multiple targets to be inspected. The update module 302 is used to update the spatiotemporal task map of the target area based on the identification information of multiple targets to be inspected, so as to obtain the target spatiotemporal task map; The task allocation module 303 is used to allocate tasks to multiple targets to be inspected based on the target spatiotemporal task map, the priority information of multiple targets to be inspected, and the status data of multiple unmanned inspection vehicles, and to determine the set of targets to be inspected for each unmanned inspection vehicle in the target area. The first determining module 304 is used to determine the inspection path of the corresponding unmanned inspection vehicle based on the set of targets to be inspected by each unmanned inspection vehicle and the spatiotemporal task map of the targets. The second determining module 305 is used to determine the inspection instructions of the corresponding unmanned inspection vehicle based on the set of targets to be inspected and the inspection path of each unmanned inspection vehicle.

[0084] In some alternative embodiments, the above-described apparatus further includes: The second acquisition module is used to acquire multimodal data of multiple targets to be inspected fed back by the unmanned inspection vehicle. The multimodal data includes visible light image data collected by the visible light camera, thermal image data collected by the infrared thermal imager, voiceprint data collected by the sound acquisition sensor, and target gas concentration collected by the gas sensor. The identification module is used to identify key monitoring areas of the visible light image data of each target to be inspected using a pre-built semantic segmentation model, and to determine at least one region of interest in the corresponding visible light image data. The third determination module is used to determine the target thermal image data in the corresponding thermal image data based on the region of interest of each target to be inspected. The target thermal image data is the data corresponding to the region of interest in the thermal image data. The first detection module is used to perform anomaly detection on the region of interest of the target to be inspected based on the target thermal imaging data, and obtain the first anomaly detection result of the target to be inspected. The second detection module is used to perform anomaly detection on the target to be inspected based on voiceprint data, and obtain the second anomaly detection result for the target to be inspected. The third detection module is used to perform anomaly detection on the environment where the target to be inspected is located based on the target gas concentration, and obtain the third anomaly detection result for the target to be inspected.

[0085] The unmanned vehicle inspection device for hydropower stations provided in this embodiment of the invention can execute the unmanned vehicle inspection method for hydropower stations provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the above modules and units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0086] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

[0087] The following is a detailed reference. Figure 4 This diagram illustrates a structural schematic suitable for implementing an electronic device according to embodiments of the present invention. The electronic device may include a processor (e.g., a central processing unit, graphics processor, etc.) 401, which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 402 or a program loaded from memory 408 into random access memory (RAM) 403. The RAM 403 also stores various programs and data required for the operation of the electronic device. The processor 401, ROM 402, and RAM 403 are interconnected via a bus 404. An input / output (I / O) interface 405 is also connected to the bus 404.

[0088] Typically, the following devices can be connected to I / O interface 405: input devices 406 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 407 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; memory devices 408 including, for example, magnetic tapes, hard disks, etc.; and communication devices 409. Communication device 409 allows electronic devices to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 4 Electronic devices with various devices are shown, but it should be understood that it is not required to implement or have all of the devices shown, and more or fewer devices may be implemented or have instead.

[0089] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 409, or installed from a memory 408, or installed from a ROM 402. When the computer program is executed by the processor 401, it performs the functions defined in the unmanned vehicle inspection method for hydropower stations according to embodiments of the present invention.

[0090] Figure 4The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of the present invention.

[0091] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code. When the software or computer code is accessed and executed by the computer, processor, or hardware, the unmanned vehicle inspection method for hydropower stations shown in the above embodiments is implemented.

[0092] A portion of this invention can be applied as a computer program product, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0093] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A hydropower station unmanned vehicle inspection system, characterized in that, The system includes: a cloud data management center, multiple edge computing units, and multiple unmanned inspection vehicles; The cloud data management center is used to generate inspection task information for multiple areas of the hydropower station based on the equipment list and inspection cycle of the hydropower station, and to distribute the inspection task information of each area to the corresponding edge computing unit. The multiple edge computing units correspond one-to-one with the multiple areas. Each edge computing unit is connected to at least one unmanned inspection vehicle in the corresponding area, and is used to update the spatiotemporal task map of the corresponding area according to the inspection task information to obtain the target spatiotemporal task map. Based on the status data of the unmanned inspection vehicles in the corresponding area and the target spatiotemporal task map, the unmanned inspection vehicles in the area are assigned tasks and inspection paths are planned to obtain inspection instructions. The nodes in the spatiotemporal task map represent the targets to be inspected, and the edges represent the passable paths between different nodes. The unmanned inspection vehicle is used to receive inspection instructions issued by the edge computing unit connected to it, and to perform inspection tasks according to the inspection instructions.

2. The system according to claim 1, characterized in that, The unmanned inspection vehicle is pre-equipped with a multimodal data acquisition device, which is used to collect multimodal data of the target to be inspected during the execution of the inspection task. The unmanned inspection vehicle is also used to send the multimodal data of the target to be inspected to the edge computing unit connected to it via a data transmission module; The edge computing unit is also used to identify anomalies in the target to be inspected based on the multimodal data, and obtain anomaly identification results.

3. The system according to claim 1 or 2, characterized in that, The unmanned inspection vehicle has two driving modes: tracked mode and wheeled differential mode. The unmanned inspection vehicle is equipped with vibration sensors, lidar and main control module. When the unmanned inspection vehicle is performing an inspection task, the main control module is used to control the driving mode of the unmanned inspection vehicle based on the vibration data collected by the vibration sensor and the road surface information collected by the lidar.

4. The system according to claim 3, characterized in that, When the unmanned inspection vehicle performs an inspection task, the main control module of the unmanned inspection vehicle is also used to acquire environmental data within the target area, and to perform local route planning for the unmanned inspection vehicle based on the environmental data. If the local route planning is successful, the unmanned inspection vehicle is controlled to travel according to the planned local route. The target area is determined based on the position and direction of travel of the unmanned inspection vehicle.

5. A method for unmanned vehicle inspection of a hydropower station, characterized in that, The method, applied to any edge computing unit of the unmanned vehicle inspection system for hydropower stations according to any one of claims 1 to 4, comprises: The system acquires inspection task information for the target area and status data for multiple unmanned inspection vehicles. The inspection task information includes identification information and priority information for multiple targets to be inspected. The spatiotemporal task map of the target area is updated based on the identification information of the multiple targets to be inspected, and the target spatiotemporal task map is obtained. Based on the target spatiotemporal task map, the priority information of multiple targets to be inspected, and the status data of multiple unmanned inspection vehicles, a genetic algorithm is used to assign tasks to multiple targets to be inspected, and to determine the set of targets to be inspected for each unmanned inspection vehicle in the target area. The inspection path of the corresponding unmanned inspection vehicle is determined based on the set of targets to be inspected and the spatiotemporal task map of the targets of each unmanned inspection vehicle. The inspection instructions for each unmanned inspection vehicle are determined based on the set of targets to be inspected and the inspection path.

6. The method according to claim 5, characterized in that, The method further includes: The system acquires multimodal data of multiple targets to be inspected from feedback by the unmanned inspection vehicle. The multimodal data includes visible light image data acquired by a visible light camera, thermal image data acquired by an infrared thermal imager, voiceprint data acquired by a sound acquisition sensor, and target gas concentration acquired by a gas sensor. Using a pre-built semantic segmentation model, key monitoring areas are identified in the visible light image data of each target to be inspected, and at least one region of interest is determined for the corresponding visible light image data. Based on the region of interest of each target to be inspected, target thermal image data is determined in the corresponding thermal image data, wherein the target thermal image data is the data corresponding to the region of interest in the thermal image data; Based on the target thermal imaging data, anomaly detection is performed on the region of interest of the target to be inspected, and the first anomaly detection result of the corresponding target to be inspected is obtained. Based on the voiceprint data, anomaly detection is performed on the target to be inspected, and a second anomaly detection result for the target to be inspected is obtained. Anomaly detection is performed on the environment where the target to be inspected is located based on the target gas concentration, and the third anomaly detection result corresponding to the target to be inspected is obtained.

7. A hydropower station unmanned vehicle inspection device, characterized in that, The device, applied to any edge computing unit of the unmanned vehicle inspection system for hydropower stations according to any one of claims 1 to 4, comprises: The first acquisition module is used to acquire inspection task information of the target area and status data of multiple unmanned inspection vehicles. The inspection task information includes the identification information and priority information of multiple targets to be inspected. The update module is used to update the spatiotemporal task map of the target area based on the identification information of the multiple targets to be inspected, so as to obtain the target spatiotemporal task map; The task allocation module is used to allocate tasks to multiple targets to be inspected based on the target spatiotemporal task map, the priority information of multiple targets to be inspected, and the status data of multiple unmanned inspection vehicles, using a genetic algorithm to determine the set of targets to be inspected for each unmanned inspection vehicle in the target area. The first determining module is used to determine the inspection path of the corresponding unmanned inspection vehicle based on the set of targets to be inspected and the spatiotemporal task map of each unmanned inspection vehicle. The second determining module is used to determine the inspection instructions for the corresponding unmanned inspection vehicle based on the set of targets to be inspected and the inspection path of each unmanned inspection vehicle.

8. An electronic device, characterized in that, include: The system includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the unmanned vehicle inspection method for hydropower stations as described in any one of claims 5 or 6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the unmanned vehicle inspection method for hydropower stations as described in any one of claims 5 or 6.

10. A computer program product, characterized in that, It includes computer instructions, which are used to cause the computer to execute the unmanned vehicle inspection method for hydropower stations as described in any one of claims 5 or 6.

Images