A pumped storage power station robot construction inspection method and system of a fusion monitoring system

Abstract

The application discloses a pumped storage power station robot construction inspection method and system fusing a monitoring system, and belongs to the technical field of inspection. The method comprises the following steps: firstly, an edge fusion device arranged in a robot is started, and online multiplexing calibration is performed on the robot and a macro camera; then, based on any device selected in a macro image, an edge fusion device acquires a target device image, and the target device image is cropped to acquire a target region mask; finally, based on the target region mask, the robot shoots a target device at a preset distance, the edge fusion device performs defect detection on the target device image shot by the robot, and outputs a detection result. The above method establishes a macro-micro fusion inspection system for a pumped storage construction site, fully breaks through the data island between fixed monitoring and robots, and significantly improves the intelligence, refinement and safety of pumped storage power station construction inspection.

Description

Technical Field

[0001] This application belongs to the field of inspection technology, and specifically relates to a method and system for robot construction inspection of pumped storage power stations that integrates a monitoring system. Background Technology

[0002] In the construction areas of pumped-storage power stations, including the underground powerhouse, upper and lower dam shoulders, and spiral casing, 4K / 8K (kilo, meaning the number of pixels in the horizontal direction of an image / video) fixed cameras have been widely deployed for safety, progress, and metering monitoring. Simultaneously, inspection robots (ground-based or drone-based) equipped with 4K binocular + TOF (Time of Flight) cameras provide close-up, first-person view inspections of welds, bolt tightness, and equipment readings. The two systems operate independently: camera streams are directly stored in an NVR (Network Video Recorder) or uploaded to the cloud, while the robots use local AI (Artificial Intelligence) for recognition. There is no spatial connection between the two systems, resulting in significant data silos.

[0003] In existing technologies, on the macroscopic monitoring side, cameras are fixed to scaffolding or high poles with explosion-proof housings, and extrinsic parameters are manually calibrated once. The H.265 bitstream (High Efficiency Video Coding) is uploaded in full at 4K@25 fps (4K resolution, 25 frames per second) to a cloud-based GPU (Graphics Processing Unit) server for CNN (Convolutional Neural Network) recognition of safety helmets, tower crane postures, etc. On the microscopic inspection side, the robot relies on RTK (Real-Time Kinematic) + IMU (Inertial Measurement Unit) for positioning, and local YOLOv5 (You Only Look Once (version 5) object detection algorithm) is used for defect detection. The results are transmitted back via Wi-Fi, and the operator views them frame by frame on the robot's image transmission interface. Then coordinate association is performed. If "macro guidance micro" is required, the pixel coordinates are estimated by a person on the monitoring screen and the robot is verbally directed to move there; or the calibration board is temporarily held in the camera's field of view and the homography matrix is ​​calculated offline once, and no further updates are made.

[0004] However, existing technologies have the following problems: 1) One-time calibration failure: After the scaffolding is removed and the high-pole camera is hit by a crane, the external parameters shift, and the error of the original calibration matrix is ​​rapidly amplified to >30 cm, making it impossible to continue guiding the robot.

[0005] 2) Full 4K cloud bandwidth explosion: 25 Mbps per channel, 250 Mbps for ten channels, far exceeding the ≤100 Mbps dedicated ring network of pumped storage plant area, which can only reduce the frame rate or the resolution, and lose detailed features.

[0006] 3) The robot's narrow field of view makes it easy to miss inspections: The single-path first-view cannot cover large-span branch pipes, requiring back-and-forth trips, which is inefficient; in addition, the robot's image transmission and macro view do not have a unified numbering system, making it difficult for operators to determine "inspected / uninspected".

[0007] 4) Macro-micro data fragmentation: The cloud-based CNN results are only stored as a JSON list (JavaScript ObjectNotation, a lightweight data exchange format), while the robot's local video is only stored as an MP4 (a common digital multimedia file format). The timestamps and coordinate systems of the two are inconsistent, making it impossible to automatically generate a "single graph" report. Later follow-up requires manual alignment, which takes more than 10 minutes.

[0008] 5) Self-identification of the construction phase without construction methods: The macro video and robot action log are separated, and the supervisor needs to manually review and compare them every day to determine whether the current stage is "civil engineering" or "installation". The misjudgment rate is about 20% (percentage), which affects the progress measurement. Summary of the Invention

[0009] To address the aforementioned issues, this application provides a robotic construction inspection method and system for pumped storage power stations that integrates a monitoring system, significantly improving the intelligence, precision, and safety of pumped storage power station construction inspections.

[0010] The purpose of this disclosure is to provide a robotic construction and inspection method for pumped-storage power stations that integrates a monitoring system, including: Once the edge blending device installed in the robot is activated, it performs online multiplexing calibration of the robot and the macro camera; Based on any device selected in the macroscopic image, the edge blending device acquires the target device image and crops the target device image to obtain the target region mask; Based on the target area mask, the robot takes a picture of the target device at a preset distance. The edge blending device performs defect detection on the target device image taken by the robot and outputs the detection results.

[0011] Optionally, online multiplexing calibration of the robot and the macro camera includes, The robot acquires video frames from multiple macroscopic cameras and its own trajectory. The robot moves within the common field of view of the macro camera and acquires pairs of binocular images through the binocular camera; A 2D-3D association is established based on the acquired paired binocular images and video frames from multiple macroscopic cameras; Based on the established 2D-3D association, the extrinsic parameters of the macroscopic camera are solved, and the robot's trajectory parameters are used as the standard for scale normalization. At preset intervals, the robot's trajectory parameters within the predetermined time are re-bound to the macro camera, and the macro camera's extrinsic parameters are updated.

[0012] Optionally, establishing a 2D-3D association based on the acquired paired binocular images and video frames from multiple macroscopic cameras includes: The robot walks a predetermined distance in a zigzag pattern within the public field of view of the macro camera and acquires paired binocular images through the binocular camera. Based on the acquired paired stereo images, 3D point clouds are generated and ORB features are extracted. Based on the extracted ORB features, the 2D pixels with the same names as the video frames of the acquired paired binocular images and multi-channel macro cameras are obtained. The obtained 2D pixels are bound one by one to the generated 3D point cloud to form a stable correspondence between 2D pixels and 3D spatial points that can be used to solve camera extrinsic parameters.

[0013] Optionally, generating a 3D point cloud based on the acquired paired stereo images includes, A stereo matching algorithm is used to perform stereo matching on pairs of binocular images to find the correspondence of each pixel and calculate the disparity. Based on the baseline distance, focal length, and intrinsic parameter matrix of the binocular camera, the disparity of each pixel is calculated and converted into the true depth value Z using the principle of triangulation. Using the robot's body coordinate system as a reference, the coordinates (X, Y, Z) of each pixel are calculated to form a 3D point cloud.

[0014] Optionally, it also includes determining whether the reprojection error is greater than a preset error value based on the macroscopic camera extrinsic parameter T_macro, wherein the determination includes, After projecting 3D points back onto the image plane based on the macroscopic camera extrinsic parameter T_macro, the deviation between the projected pixels and the actual pixels is obtained. Determine whether the acquired error is greater than the preset error value. If it is, recalibrate; otherwise, filter the target perception data based on the target perception data trigger conditions.

[0015] Optionally, it also includes edge fusion devices filtering target perception data based on target perception data trigger conditions, specifically including: The edge fusion device detects images acquired by each macro camera to obtain the object bounding box B_macro and category C_macro for any target; Based on the obtained object bounding box B_macro and category C_macro of any target, determine whether the corresponding target meets the target perception data triggering condition. If it does, obtain the target perception data of the target and upload it to the cloud. The target perception data includes image keyframes, object bounding boxes, 3D centroid X_w obtained by backprojection from camera extrinsic parameters, timestamps, and the ID of the macro camera.

[0016] Optionally, the target perception data triggering condition includes satisfying any of the following conditions: If the area change rate of a target in the acquired image exceeds a preset percentage, or if a new target or a new target category is detected, the target has entered a danger zone.

[0017] Optionally, based on any device selected in the macroscopic image, the edge blending device acquires a target device image and crops the target device image to obtain a target region mask, including... Calculate world coordinates X_w based on the pixel p_macro and depth map D_macro of any selected device; The calculated world coordinates X_w are transformed into the robot body coordinate system to obtain the relative pose X_r of the target device image; Based on X_r and real-time laser point cloud, a robot trajectory is generated, and the robot moves towards the target device based on the trajectory. As the robot moves, it generates a 3D search box centered on X_r and with a preset value multiplied by the length of the target device as the side length. This box is then projected onto the left image of the binocular view to obtain the attention mask M_macro2micro.

[0018] Optionally, based on a target region mask, the robot captures images of the target device at a preset distance. An edge blending device then performs defect detection on the captured image of the target device and outputs the detection results, including... The robot acquires images of the target device within a preset distance, and also acquires depth d_micro and binocular stitched images; The defect box B_micro and the reading V_micro are output from the image stitched by binoculars. The four corner points of the defect box B_micro are back-projected to the world coordinate system, and then projected back to the corresponding image frame under the macro camera according to the camera extrinsic parameters. A semi-transparent overlay layer is drawn based on the obtained image frame, and numbered bubble labels are generated simultaneously to finally obtain a micro-cropped image and interpretation results.

[0019] Optionally, it also includes a combined macroscopic and microscopic understanding of the construction methods, specifically including: Images acquired by the robot and the macroscopic camera are sampled at a preset frame rate, and macroscopic object state vectors and microscopic motion embedding vectors are obtained respectively. Based on the acquired macroscopic object state vectors and microscopic action embedding vectors, a heterogeneous graph G=(V 总 , E), where V 总 Nodes include macroscopic object nodes V and microscopic action nodes V'. The macroscopic object node V is the macroscopic object state vector S_macro; the microscopic action node V' is the microscopic action embedding vector S_micro; E is an edge. If the robot performs an action on any macroscopic object at time t, a directed edge V'→V is established, where the weight is the reciprocal of the three-dimensional distance between the two nodes. Based on the constructed heterogeneous graph G=(V 总 (E), aggregate neighbor features and evolve along the time axis, finally outputting the construction method stage.

[0020] Optionally, based on heterogeneous graph G=(V 总 The process involves aggregating neighbor features using a two-layer GCN-LSTM algorithm, evolving them along the time axis, and finally outputting the final output.

[0021] Optionally, it also includes local model updates by the edge blending device, specifically including, Each edge fusion device calculates its local model gradient based on local data at a preset interval and uploads it to the federated server; After receiving the calculated local model gradient, the federated server performs a weighted average to obtain the global model weights and then sends them to the edge fusion device. The edge fusion device updates online based on the received global model weights.

[0022] Another objective of this disclosure is to provide a robotic construction and inspection system for pumped-storage power stations that integrates a monitoring system, including, Macro cameras are used to acquire macroscopic images of the construction site; The robot is equipped with a camera to acquire microscopic images of the target. An edge blending device, mounted on a robot, is used for online multiplexing and calibration of the robot and a macroscopic camera. Based on any device selected in the macroscopic image, it acquires the device image, crops the device image to obtain a target area mask, and performs defect detection on the target based on the acquired target area mask and the target microscopic image, and outputs the detection results.

[0023] Optionally, it also includes, The drone collaboration module is used to acquire on-site images; The edge backup module is used to save the acquired images; Laser projection marking device is used by robots to project cross lasers at defect locations; The cloud-based visualization module is used to overlay macroscopic camera footage, robot trajectories, and defect points onto the BIM model in real time for viewing.

[0024] Compared with the prior art, this application has the following advantages: The above method establishes a macro-micro integrated inspection system for pumped storage construction sites, fully breaking down data silos between fixed monitoring and robots, significantly improving the intelligence, precision, and safety of pumped storage power station construction inspections, and laying a technical foundation for subsequent federated learning multi-station collaboration.

[0025] Furthermore, through online reuse calibration, event-driven uploading, cross-scale ROI cropping, and GCN-LSTM construction method understanding, this disclosure solves the problem of one-time calibration failure caused by scaffolding changes, realizes automatic updating of macroscopic camera pose at preset intervals (e.g., 10 minutes), reducing errors; solves the bandwidth bottleneck caused by full 4K cloud upload, greatly reducing the traffic of the dedicated ring network; solves the problem of narrow field of view of the robot's first-person perspective and easy omission of large-span equipment, realizing "clicking on the macroscopic screen to command the robot to autonomously approach for inspection"; solves the pain point of macro-micro coordinate system fragmentation and inability to trace uniformly, automatically generating a "single image" report with coordinates and defect overlay, greatly reducing traceability time and improving inspection efficiency; solves the inefficiency problem of relying on manual judgment during the construction phase, improving the accuracy of identification in construction, installation, commissioning, and acceptance phases.

[0026] Other features and advantages of this application will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the application. The objectives and other advantages of this application may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A schematic diagram of a construction inspection method for a pumped storage power station using a robot based on an embodiment of this disclosure is shown. Figure 2 A schematic diagram of an online multiplexing calibration and event-driven process according to an embodiment of this disclosure is shown. Figure 3This illustration shows a schematic diagram of the implementation process of cross-scale fusion and "one-map" reporting function in an embodiment of this disclosure; Figure 4 A schematic diagram of the structure of a pumped-storage power station robot construction and inspection system according to an embodiment of the present disclosure is shown. Figure 5 A schematic diagram of the structure of a pumped-storage power station robot construction and inspection system, representing another embodiment of the integrated monitoring system in this disclosure, is shown. Detailed Implementation

[0029] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0030] like Figure 1 As shown in the embodiments of this disclosure, a construction inspection method for a pumped-storage power station robot in a fusion monitoring system is disclosed. The method includes: first, after the edge fusion device installed in the robot is activated, online multiplexing calibration is performed on the robot and the macroscopic camera; then, based on any device selected in the macroscopic image, the edge fusion device acquires an image of the target device and crops the target device image to obtain a target area mask. Finally, based on the target area mask, the robot takes a picture of the target device at a preset distance, and the edge fusion device performs defect detection on the target device image taken by the robot and outputs the detection results. The above method is a macro-micro multi-scale perspective analysis and understanding and construction inspection method for pumped-storage power station robots in a fusion monitoring system, which significantly improves the intelligence, precision and safety of construction inspection of pumped-storage power stations, and lays a technical foundation for subsequent federated learning multi-station collaboration.

[0031] Specifically, the above method is a macro-micro multi-scale perspective analysis and understanding method for pumped storage power station robots that integrates monitoring systems. Specifically, the 4K / 8K construction cameras already deployed extensively at pumped storage power station sites are directly used as the "macro layer" (hereinafter referred to as macro cameras), and the inspection robot equipped with 4K binoculars + TOF is used as the "micro layer." The two are integrated through an "edge fusion box" (referred to as the edge fusion device in this embodiment) placed inside the robot's explosion-proof enclosure to complete the above method. Specifically, as shown... Figure 2 As shown, after the edge fusion device is powered on, online multiplexing calibration of the robot and the macroscopic camera includes: First, the robot acquires video frames from multiple macroscopic cameras and its own trajectory. Specifically, it pulls N H.265 4K (4K resolution) streams (pre-buffered at 1fps by default) via RTSP (Real-Time Streaming Protocol), where N≥1 and is an integer. At the same time, it reads the trajectory {L_robot} (10Hz, positioning accuracy ±5cm) given by the robot's UWB (Ultra Wideband)-magnetic fingerprint coupling positioning module.

[0032] Then, the robot moves within the common field of view of the macro camera and acquires paired binocular images through the binocular camera; the robot enters the field of view and walks in a zigzag pattern for a distance greater than or equal to a preset distance within the common field of view of the macro camera. The preset distance may be, but is not limited to, 15m, and the trajectory is collected. The binocular camera also simultaneously captures a preset number of images. Specifically, the preset number is: binocular simultaneous shooting > 30 frames (more than 30 images are recorded per second during shooting).

[0033] Based on the acquired paired binocular images and video frames from multiple macroscopic cameras, a 2D-3D association is established. First, based on the acquired paired binocular images, 3D point clouds are generated. Specifically, this includes: 1) using stereo matching to calculate the disparity map, performing stereo matching on the left and right images to find the correspondence of each pixel and calculate the disparity; 2) converting the disparity into depth using the binocular camera intrinsic parameters. Specifically, based on the baseline distance, focal length, and intrinsic parameter matrix of the binocular camera, the disparity of each pixel is converted into the true depth value Z using the principle of triangulation; 3) generating 3D point clouds pixel-by-pixel. Using the robot's body coordinate system as a reference, the (X, Y, Z) coordinates of each pixel are calculated (and converted to the robot's body coordinate system coordinates) to form the 3D point cloud. Then, the ORB algorithm is used to extract ORB (Oriented FAST and Rotated BRIEF) features from paired stereo images and match them to the corresponding macroscopic frames to establish a 2D-3D association. Specifically, oriented FAST corner points are first used to detect and locate feature points, and then rotation-invariant BRIEF binary descriptors are used to generate feature descriptions. This approach is characterized by fast computation, rotation invariance, and suitability for real-time operation at edge devices, enabling rapid extraction of stable feature points in the robot's stereo images, providing a foundation for subsequent cross-screen matching. Finally, ORB feature matching is used to find corresponding 2D pixels in the robot's stereo images and macroscopic camera frames. Combined with the 3D point cloud output from the robot's stereo images, the 2D pixels in the macroscopic images are bound one-to-one with real-world 3D coordinates, forming 2D pixels that can be used for camera extrinsic parameter solving. Stable correspondence between points in 3D space.

[0034] Based on the established 2D-3D association, the extrinsic parameters of the macroscopic camera are solved, and scale normalization is performed using the robot's trajectory parameters as a standard. Specifically, incremental SfM is used to solve the extrinsic parameters T_macro (rotation R + translation t) of the macroscopic camera (also known as the macroscopic camera), and scale normalization is performed using L_robot (trajectory) as the ground truth. Preferably, the initial calibration error is, but is not limited to, <2cm. Incremental SfM is an incremental motion reconstruction structure. During the motion reconstruction structure solution process, new viewpoint images are added incrementally, and feature matching and bundle adjustment optimization are performed sequentially to gradually reconstruct the 3D structure and camera pose, rather than a one-time global optimization.

[0035] At preset time intervals, the robot's trajectory parameters within a predetermined time period are re-bound to the macroscopic camera, updating the macroscopic camera's extrinsic parameters. This re-binding is performed using a sliding window BA (Bundle Adjustment) method. Specifically, the preset time interval can be, but is not limited to, 10 minutes, and may include, but is not limited to, 1 minute. For example, every 10 minutes, the robot's trajectory within the latest minute is re-bound and optimized with the macroscopic frame, updating the macroscopic camera's extrinsic parameter T_macro. Further, it is determined whether the reprojection error exceeds a preset error value. If it does, recalibration is performed; otherwise, subsequent event-driven edge filtering and uploading are executed. For example, if the reprojection error exceeds 3 cm due to scaffolding removal or camera collision, recalibration is automatically triggered without manual calibration. The reprojection error is the deviation between the projected 3D spatial point and the actual pixel point, used to quantify the accuracy of the macroscopic camera's extrinsic parameter T_macro. A larger error indicates a more severe extrinsic parameter offset. The macroscopic camera's extrinsic parameter T_macro is the parameter to be optimized.

[0036] The edge fusion device operates online with SfM+ sliding window BA, automatically updating the macroscopic camera pose within 10 minutes of scaffolding movement. This completely bridges the macro and micro coordinate systems, enabling a closed-loop operation of "clicking on the macroscopic level and guiding the microscopic level within seconds." Furthermore, the problem of one-time calibration failure is eliminated, saving labor and downtime costs.

[0037] like Figure 2 As shown, after calibration, the method further includes event-driven edge filtering and uploading, that is, the edge fusion device filters the target perception data based on the target perception data trigger conditions, specifically including... Deploying YOLOv8-N (YOLOv8-Nano, the eighth generation YOLO algorithm) in edge fusion devices Nanoscale lightweight model; The edge fusion device detects the 4K raw stream (images acquired by a macro camera) based on YOLOv8-N to obtain the object bounding box B_macro and category C_macro for any target; among them, 2 / 4 frame skip detection can be performed, and finally the object bounding box B_macro and category C_macro for any target are obtained; among them, the identified objects (targets) include: (1) construction equipment or facilities, such as tower cranes (tower crane hooks), pump trucks, formwork trolleys, volutes, bolt buckets, hoppers, and temperature probes. (2) construction personnel or safety personnel, such as construction personnel and safety helmets. (3) defects or anomalies, such as cracks and surface cracks. (4) scene extensions, such as other detectable equipment, personnel, and dangerous area targets at the construction site. Thus, B_macro is the bounding box coordinate of the above objects in the macro image; C_macro is the category label corresponding to the above objects.

[0038] Based on the obtained object bounding box B_macro and category C_macro of any target, determine whether the corresponding target meets the target perception data triggering conditions. If it does, acquire the target perception data and upload it to the cloud. Specifically, the target perception data triggering conditions include: The rate of change of the target area is: ΔA = |A_t–A_t-1| / A_t-1; where A_t and A_t-1 represent the area at time t and time t-1, respectively. The target area is obtained based on the object bounding box (i.e., the known target location and size information).

[0039] A new target or new target category is detected, obtained based on the category box, i.e., a new object appearance flag: New = C_t {C_t-1, C_t-2…C_t-m}, where m represents the total number of consecutive frames (m is a positive integer), C_t is the object category detected in the current frame (time t), and C_t-1, C_t-2…C_t-m are the sets of all object categories that have appeared in the past m consecutive frames (m can take the value 30). The flag for a new object is: New = C_t. {C_t-1, C_t-2…C_t-30}, where C_t is the object category detected in the current frame (time t) (e.g., tower crane, personnel, bolt bucket, crack, etc.), and C_t-1, C_t-2…C_t-30 are the sets of all object categories that have appeared in the past 30 consecutive frames. New = 1: A new object category has appeared in the current frame that has not appeared in the past 30 frames; this is considered a new object. New = 0: The current object category has already appeared in the past 30 frames and is not considered a new object. When a target enters a dangerous area, an intrusion flag is determined based on the location information within the object frame: Intrude = IoU(B_macro, ROI_danger)>0 (meaning a Boolean operation result of true indicates that the intrusion condition for the dangerous area is met (IoU>0)). ROI_danger refers to the dangerous area, which is a high-risk area pre-defined in the macro monitoring screen where personnel or equipment are prohibited from entering at will. It is used to perform an IoU (Intersection over Union) calculation with the detected object frame B_macro to determine whether the target has entered the dangerous area. When ΔA > a preset percentage or either New=1 or Intrude=1, it is determined to be target perception data; the preset percentage can be, but is not limited to, 20%, and can be adaptively changed according to the application environment. Target perception data can be "event frames".

[0040] Target perception data is uploaded to the cloud. This data includes image keyframes, object bounding boxes, 3D centroid X_w obtained by backprojection from camera extrinsic parameters, timestamps, and the macro camera ID. Event frames are immediately packaged as follows: JPEG (Joint Photographic Experts Group) keyframe + B_macro + 3D centroid X_w (backprojected from T_macro) + timestamp + camera ID (i.e., macro camera ID), and uploaded to the cloud via MQTT-TLS (MQTT (Message Queuing Telemetry Transport) secure transmission protocol based on TLS (Transport Layer Security) encryption, with a bandwidth consumption of <5 Mbps). Non-event frames are only stored locally and are overwritten cyclically after 7 days.

[0041] On the edge side, only the "keyframe + object frame + 3D coordinates" are uploaded to the cloud as needed, while the 4K raw stream is stored locally. A dedicated ring network of ≤100Mbps in the factory area can meet the needs of multiple concurrent connections without the need to expand fiber optic cables.

[0042] like Figure 3 As shown, the method includes cross-scale ROI (Region of Interest) cropping and attention guidance. Specifically, based on any device selected in the macroscopic image, an edge blending device acquires a target device image, and the target device image is cropped to obtain a target region mask. Based on the pixel p_macro of the selected arbitrary device and the depth map D_macro (interpolated from the SfM sparse point cloud), calculate the world coordinates X_w; where the depth map D_macro is generated by interpolating the sparse points (3D point cloud output during the 2D-3D association process) into a dense depth map D_macro through an interpolation algorithm.

[0043] Transform the calculated world coordinates X_w into the robot's body coordinate system to obtain the relative pose X_r of the target device image; that is... Figure 3 The ROI (Region of Interest) projection in the model. Relative pose includes the position (coordinates) and orientation (or rotation angle) of the target device image in the robot's body coordinate system.

[0044] The robot path planning module is based on X_r and real-time laser point cloud (data from the robot's built-in LiDAR), and adopts an improved A... (An algorithm for finding the shortest path) The path planning algorithm generates a collision-free trajectory for the robot and moves it toward the target device based on the trajectory; As the robot moves, a 3D search box is generated centered on X_r, with a preset value multiplied by the device's longest side as the side length. This search box is then projected onto the left-side binocular image, resulting in an attention mask M_macro2micro. The preset value can be, but is not limited to, 2. A 3D bounding box construction and perspective projection algorithm is used to generate the 3D search box centered on the target's relative pose, and then projected onto the left-side binocular image using camera intrinsic and extrinsic parameters. The subsequent defect detection network only performs a sliding window operation on the region within M_macro2micro, reducing computation by 70% and lowering false background detections.

[0045] After unifying the world coordinate system, operators can click / select equipment on the macro screen, and the system can calculate the 3D target in seconds and command the robot to approach autonomously, eliminating the need for manual distance estimation and verbal instructions, and significantly reducing the missed detection rate.

[0046] like Figure 3 The diagram also includes the feedback of microscopic detection results and the generation of a single image. Specifically, based on a target region mask, the robot photographs the target device at a preset distance. An edge blending device then performs defect detection on the robot-photographed image of the target device and outputs the detection results, including... The robot acquires images of the target device within a preset distance and uses Time-of-Flight (TOF) to obtain depth d_micro (microscopic depth) and a binocular stitched image. Specifically, the robot's binocular cameras simultaneously capture two images, left and right, which are then stitched together to form a single wide-format image – the binocular stitched image. Simultaneously, the robot's onboard TOF module independently measures the distance to the target, obtaining d_micro (microscopic depth), within a preset distance that can be, but is not limited to, 0.5–1.5 m.

[0047] The YOLOv8-P (You Only Look Once Version 8 – Pose, YOLOv8 pose estimation model) model is used to detect defect boxes B_micro and readings V_micro in the binocular stitched images. The four corner points of the defect box B_micro are back-projected to the world coordinate system, and then projected back to the corresponding image frame from the macro camera according to the camera's extrinsic parameters; that is... Figure 3 The inverse mapping in.

[0048] A semi-transparent overlay is drawn based on the obtained image frames, and numbered bubble labels are generated simultaneously, ultimately yielding a microscopic cropped image and interpretation results. The semi-transparent overlay and labels are drawn using graphics rendering; clicking on them unfolds the microscopic 4K cropped image and AI interpretation results. Furthermore, the system automatically generates a PDF report on the web (browser-based web interface): the cover is a macroscopic panoramic thumbnail, and the inner pages list "macroscopic positioning map + microscopic defect close-up + 3D coordinates + confidence level" for each device, eliminating the need for manual screenshots. Macroscopic ROI cropping generates an attention mask, reducing the robot's detection range by 70% and lowering background false alarms; microscopic results are overlaid back onto the macroscopic frames in real time, automatically generating a "one-image" report, eliminating the need for manual image stitching and reducing traceability time from hours to seconds.

[0049] In this embodiment of the disclosure, the method further includes performing a combined macroscopic and microscopic understanding of the construction process, specifically including: Images acquired by the robot and a macroscopic camera are sampled at a preset frame rate, and macroscopic object state vectors and microscopic action embedding vectors are obtained separately. The macroscopic object state vector uses YOLOv8 N object detection to obtain the bounding box and category, combines this with backprojection of camera extrinsic parameters to obtain the 3D position, calculates the velocity using Kalman filtering, and obtains the pose from point cloud or pose estimation. These are then concatenated to form the state vector: S_macro = position + velocity + pose. The microscopic action embedding vector uses the SlowFast video action recognition network to extract action features from the microscopic inspection video, resulting in the microscopic action embedding vector S_micro.

[0050] Based on the acquired macroscopic object state vectors and microscopic action embedding vectors, a heterogeneous graph G=(V 总 E); where the preset frame rate can be, but is not limited to, 1 fps (frames per second). Wherein, the heterogeneous graph G = (V 总 E) satisfies: Heterogeneous graph G=(V 总 E) contains two types of nodes: V 总= V (macroscopic object node) + V' (microscopic action node), where V' is the actual node that exists in the image.

[0051] Macro-object node V: Consists of the construction equipment state vector S_macro detected by the macro camera. Furthermore, macro-objects include, but are not limited to, construction equipment such as tower cranes, pump trucks, and formwork trolleys.

[0052] Micro-action node V': It consists of the action embedding vector S_micro extracted from the robot's micro-video by SlowFast 8 (SlowFast 8×8, SlowFast video recognition network). The micro-actions include, but are not limited to, construction actions such as welding, bolt tightening or grinding.

[0053] Edge E: The directed edge V'→V established when the robot performs an action. That is, at time t, if the robot performs an action on any macroscopic object, a directed edge E is established from the microscopic action node V' to the macroscopic object node V. The weight of the edge is set to the reciprocal of the three-dimensional spatial distance between the two nodes, thus completing the construction of the heterogeneous graph G=(V total, E). Based on the constructed heterogeneous graph G=(V,E), the system first aggregates neighbor features, then evolves along the time axis, and finally outputs the construction stage. This is achieved using a two-layer GCN-LSTM algorithm: the GCN (Graph Convolutional Network) first aggregates neighbor features (i.e., a two-layer GCN graph convolutional network spatially aggregates the neighbor features of each node in the heterogeneous graph), and the LSTM (Long Short-Term Memory) then evolves along the time axis (i.e., the aggregated features are input into the LSTM long short-term memory network for temporal evolution modeling). Finally, the Softmax (normalized exponential function) outputs the construction stage {civil engineering, installation, commissioning, acceptance}. This method was trained on a dataset of 50,000 pumped-storage construction videos, achieving an accuracy of 96% (percentage). Furthermore, when an operator clicks on any device, the system retrieves all associated micro-segments based on the graph index and uses the timestamp primary key of the edge-fusion device's local SQLite (embedded lightweight local database) to replay the corresponding robot's historical video within 2 seconds.

[0054] The self-identification of construction phases makes the construction progress controllable. Among them, the heterogeneous graph GCN-LSTM integrates the macroscopic object state and microscopic action sequence to automatically determine the civil engineering, installation, commissioning and acceptance phases with an accuracy rate of >96%. Supervisors do not need to manually replay and compare every day, and the progress measurement is timely and accurate.

[0055] The method also includes federated learning and a secure closed loop, where edge fusion devices perform local model updates, specifically including... Each edge fusion device calculates its local model gradient ΔW based on local data at a preset period and uploads it to the federated server. The local model gradient ΔW is calculated using the standard deep learning backpropagation algorithm. Specifically, the edge fusion device uses local inspection images, defect detection results, and construction method identification data as training samples. It uses a stochastic gradient descent (SGD) or Adam (Adaptive Moment Estimation) optimizer to calculate the loss function through forward inference, and then obtains the gradient ΔW of the local model parameters through backpropagation. This gradient ΔW characterizes the direction of model updates based on local data. Furthermore, the preset period can be weekly, but is not limited to this; other periods are also applicable to this disclosure.

[0056] After receiving the calculated local model gradients, the federated server of each pumped storage power station's edge fusion device performs a weighted average to obtain the global model weights, and then distributes them to the edge fusion devices. The calculation process using the FedAvg (federated averaging algorithm) includes: (1) The federated server collects the local gradients ΔW1, ΔW2…ΔW uploaded by all edge fusion devices. n n represents n edge fusion devices, where n is an integer and > 0; (2) Assign weights based on the amount of local data on each device; (3) Perform a weighted average over all local gradients: W_global = (n1·ΔW1+ n2·ΔW2+ … + n n ·ΔW n ) / (n1+n2+…+n n ), where n1~n n This is a weighting coefficient for the amount of local sample data on each edge device; Finally, the global model weights W_global are obtained and distributed.

[0057] The server uses FedAvg (Federated Averaging) to aggregate and distribute the global weight W_global, enabling hot updates of edge fusion devices without downtime.

[0058] The edge fusion device updates online based on the received global model weights W_global.

[0059] The method continuously evolves through federated learning, enabling shared use across multiple stations (multiple pumped storage power stations). The edge gradients are uploaded after differential privacy processing, aggregated in the cloud using FedAvg, and then distributed. The model is incrementally updated weekly, allowing for rapid adaptation to new cracks and equipment without the need for extensive data recollection, thus achieving knowledge sharing across the group-level pumped storage construction sites.

[0060] In this embodiment, the original 4K video is stored locally on a RAID1 (disk mirroring array) SSD (solid-state drive) for 7 days, automatically overwriting upon expiration. The upload link uses MQTT over TLS1.3 (MQTT communication based on TLS1.3 encrypted tunnel transmission), and the certificate has built-in TPM2.0 (Trusted Platform Module 2.0) to prevent man-in-the-middle tampering. Explosion-proof requirements: The edge fusion device is an edge box, and the edge box shell is made of 316L stainless steel Exd IICT6 (explosion-proof rating). All antennas are introduced through explosion-proof feedthroughs. Utilizing the existing explosion-proof shells and stainless steel flexible tubes of the on-site cameras, no additional explosion-proof pan-tilt units are required, saving costs >30%.

[0061] Example 1: Macro-micro collaborative inspection of the roof of the underground powerhouse of a pumped storage power station during construction. Scenario overview: The underground powerhouse of a pumped storage power station in East China is 220m long and 26m wide. Twelve 4K explosion-proof cameras have been installed on the roof for safety monitoring. Unit 4 is entering the spiral casing installation stage, requiring initial tightening, final tightening, and weld sampling inspection of 24 M72 (metric coarse thread, nominal diameter 72mm) high-strength bolts at a height of 30m. Traditional methods rely on manual climbing and binoculars, resulting in a high rate of missed inspections and requiring secondary scaffolding erection, taking 5 days.

[0062] The application of the above method includes the following process: 1) System startup and calibration The edge blending box (Ex d IIC T6) is powered on along with the ground-based orbital robot and pulls 12 channels of 4K@25fps H.265 streams via RTSP, pre-buffering 1 fps keyframes.

[0063] The robot moves at a constant speed along the longitudinal track of the factory for 40m. The UWB-magnetic fingerprint module outputs the trajectory {L_robot} at 10Hz, with a positioning error of ≤4cm.

[0064] The robot simultaneously captures over 30 frames of data from the top camera's common field of view using both eyes. The edge fusion box runs incremental SfM to solve for 12 macroscopic camera extrinsic parameters T_macro, with an initial error of 1.6 cm.

[0065] The sliding window BA is enabled and automatically updated every 10 minutes. When the tower crane moves and the reprojection error of camera 2 increases to 3.2cm, the system immediately recalibrates and the error drops back to 1.4cm. No manual calibration plate needs to be set up throughout the process.

[0066] 2) Event-driven upload The lightweight YOLOv8-N performs 2 / 4 frame skipping detection on 12 streams, identifying objects such as tower crane hooks, volutes, bolt barrels, and construction workers.

[0067] Set event rules: ① Bolt barrel area change > 20%; ② New "final tightening wrench" category appears; ③ Personnel enter the 2m restricted area of ​​the volute.

[0068] At 09:21 on the first day of operation, the tower crane hook lifted the final tightening wrench into the No. 4 machine pit, causing a 35% change in area, triggering an event. The edge box captured the frame as a JPEG (200kB) + object frame + 3D centroid X_w, uploaded via MQTT-TLS, taking 180 ms and using only 4.8Mbps of bandwidth, saving 82% compared to the original 4K full stream.

[0069] 3) Macro-level clicks - Micro-level convergence When the supervisor clicks on the area of ​​the bolt to be inspected in the macro view on the web interface, the edge box will back-calculate the pixel coordinates p_macro into world coordinates X_w (error <2cm) and then convert them into robot orbit coordinates X_r.

[0070] After receiving X_r, the robot starts the improved A. The path was planned to avoid the ground ventilation pipes, and the vehicle traveled 12m to reach the area directly below the volute; the gimbal was raised by 28°, the TOF distance measurement was 1.1m, and the autofocus was activated.

[0071] Edge fusion box generates a 512×512 attention mask M_macro2micro, and the defect detection network slides a window only in this area, reducing inference latency from 120ms to 35ms and background false positive rate by 70%.

[0072] 4) Microscopic detection and feedback The robot captures 4K images with its binoculars, and AI identifies two bolts lacking anti-loosening lines and one weld pore. The output frame is B_micro and the depth is d_micro.

[0073] The edge box backprojects the four corners of B_micro onto the macro frame, generating a semi-transparent overlay image and automatically annotating "4-15# and 4-16# bolts are not drawn; 4-23# weld has 2mm×3mm pores".

[0074] The system generates a PDF report within 1 second, including a macroscopic positioning map, microscopic close-up, 3D coordinates, and a confidence level of 0.92. The supervisor's mobile phone receives the message push from the client in real time.

[0075] 5) Self-identification of construction method stages After each bolt inspection is completed, the graph database is written with "node = bolt, edge = final tightening action".

[0076] At 17:00 that day, the GCN-LSTM model, based on 126 macro nodes and 312 micro edges, determined that Unit 4 was in the "installation phase - final tightening phase" with an accuracy of 96%, and automatically synchronized it to the owner's ERP (Enterprise Resource Planning) progress module.

[0077] 6) Results and Benefits The entire process of installing 24 bolts took 6 hours, which is 4.5 days shorter than the traditional manual climbing method. Three defects were found, but the project passed the acceptance test on the first attempt with zero rework. The 4K raw stream was stored locally for 7 days, with a peak upload traffic of 5.2 Mbps, which meets the requirements of the factory's ≤100 Mbps dedicated ring network. The edge fusion box operated at 28W throughout the process without triggering the explosion-proof enclosure temperature rise alarm.

[0078] Example 2: Multi-machine collaborative inspection during the backfilling period of the upper reservoir abutment branch pipe. Scenario overview: A pumped storage power station in Southwest China has a 510m drop between the upper and lower reservoirs, a 6.4m diameter branch pipe, and 2m thick concrete backfill. Three high-pole 4K cameras have been deployed on the dam abutment, but the view is top-down and fixed, making it impossible to see the details of the angle between the steel lining and the rebar cage. The project team has added one ground-based tracked robot and one quadcopter drone (to acquire macroscopic images), requiring the completion of 120m long backfill layer delamination, temperature monitoring, and construction method phase confirmation without adding new cameras or expanding fiber optic cables.

[0079] The application of the above method includes the following process: 1) Rapid transformation Only one edge fusion box is added inside the high-pole electrical box, and three cameras are connected using the existing explosion-proof fiber optic transceiver; robots and drones are connected to the box via 5G SA dual-SIM CPE (Customer Premises Equipment) to form an integrated air-ground macro-micro network.

[0080] 2) Establishment of unified coordinates After takeoff, the drone hovered for 15m and captured 50 frames from the field of view of the high-pole camera. The edge box ran SfM to align the drone's world coordinate system with the extrinsic parameters of the three fixed cameras with an error of 1.8cm.

[0081] The ground robot walks along the track at the top of the branch pipe. UWB provides a 3cm-level trajectory, and further BA optimization completes the unification of macro-micro-aerial three-dimensional benchmarks.

[0082] 3) Event-driven approach saves bandwidth The main risks during the backfilling stage are: interruption of concrete supply, surface cracks >1mm, and temperature >28℃ (degrees Celsius).

[0083] Edge YOLOv8-N identifies three types of objects: "hopper", "crack" and "temperature probe"; when the hopper leaves the ROI or the crack area changes by more than 20%, an upload is triggered.

[0084] 184 event frames were generated within 24 hours, totaling 36 MB (megabytes), saving 86% of bandwidth compared to a full 4K upload.

[0085] 4) Air-ground collaborative microscopic re-inspection 09:40 Event: Camera #3 detected a 12m x 3cm crack. The supervisor selected the crack area on the web interface, calculated the 3D coordinates using edgeboxes, and simultaneously issued tasks to the drone and ground robot: The drone arrived directly overhead in 40 seconds, with a TOF distance of 2.5m, and the visible crack length was 12.3cm. The ground robot traveled 18 meters and looked up to photograph the side wall of the steel cage, discovering two early micro-cracks of 0.5 mm in diameter and 8 mm in depth.

[0086] The edgebox overlays the detection results of the drone and robot onto the same macro frame, automatically generating a "crack unfolding map" with annotations of length, width, depth, and coordinates, with an accuracy of ±1.5mm.

[0087] 5) Identification of construction method stages On that day, a total of 94 "concrete supply" nodes, 187 "vibration" nodes, and 52 "finishing" nodes were recorded.

[0088] GCN-LSTM determined that the current stage is "backfilling stage - second layer finishing stage" and predicted that it will enter the "third layer pouring" stage in 6 hours, with an error of 0.8 hours from the actual progress.

[0089] 6) Results and Benefits The backfilling cycle for the 120m branch pipe was 14 days, completed 1.5 days ahead of schedule; five early micro-cracks were found, and grouting was carried out in time, avoiding the need to remove 60m³ (square meters) of concrete later, saving direct costs of 420,000 yuan; the peak bandwidth throughout was 7.3 Mbps, far below the 100Mbps limit of the plant ring network; after two weeks of federated learning, the crack identification mAP (mean Average Precision) increased from 0.81 to 0.88, and there were no more missed detections.

[0090] The aforementioned method directly utilizes the numerous 4K / 8K construction cameras already deployed at pumped-storage power stations as the "macro layer," and inspection robots equipped with 4K binoculars and TOF sensors as the "micro layer." The two layers are aligned spatiotemporally, filtered for events, fused across scales, and controlled via an "edge fusion box" housed within the robot's explosion-proof enclosure. The cloud only receives keyframes, object frames, and event clippings, and uses federated learning to distribute incremental weights weekly, achieving macro-micro multi-scale collaborative inspection through "local real-time processing, lightweight cloud evolution, macro-guiding micro-level, and micro-level feedback to macro-level." This significantly improves the intelligence, precision, and safety of pumped-storage power station construction inspections while reusing existing cameras and operating on ≤100 Mbps networks, laying a technical foundation for subsequent multi-station collaborative inspections using federated learning.

[0091] like Figure 4 As shown in the embodiments of this disclosure, a pumped-storage power station robotic construction inspection system capable of executing the above-described fusion monitoring system is also introduced. The system includes a macroscopic camera, a robot, and an edge fusion device mounted on the robot. The macroscopic camera is used to acquire macroscopic images of the construction site; the robot is equipped with a camera to acquire microscopic images of the target; the edge fusion device is used to perform online reuse calibration of the robot and the macroscopic camera, acquire device images based on any device selected in the macroscopic image, crop the device images to obtain a target area mask, and perform defect detection on the target based on the acquired target area mask and the target microscopic image, and output the detection results. The system achieves the core functions of "reusing existing cameras, reducing bandwidth by 80%, macroscopic guidance of microscopic observation, and microscopic feedback to macroscopic observation."

[0092] like Figure 5As shown, the macroscopic cameras consist of a 4K camera array, and the edge fusion device is an edge fusion box. The edge fusion box includes a core computing unit, which can utilize the NVIDIA Jetson AGX Orin (a high-performance, low-power embedded AI computing module). The edge fusion box also integrates a UWB-magnetic fingerprint positioning module, a gigabit Ethernet port, a 4G / 5G module, a TPM (Trusted Platform Module) security chip, an online calibration module, an event filtering module, and a cross-scale projection module to complete the aforementioned online calibration, event filtering, cross-scale projection, and federated learning. The robot itself is also equipped with a 4K binocular + TOF camera to provide a microscopic field of view and high-precision depth, serving as the data source for defect detection and 3D coordinate feedback. The macroscopic cameras have an H.265 stream interface: at least one 4K@25 fps RTSP input for SfM calibration and ROI guidance; if an analog camera is used on-site, an HDMI-IP (High-Definition Video Interface - Internet / LAN Protocol) encoder must be added. The robot is also equipped with an explosion-proof enclosure and power supply, meeting the Ex d IIC T6 requirements of the pumped storage underground plant, ensuring continuous operation in an environment with a hydrogen concentration of <10%. The robot uploads relevant data to the federal server and / or reporting center via an MQTT Broker (MQTT (a lightweight messaging protocol for the Internet of Things) message relay station).

[0093] In this embodiment, the system further includes a drone collaboration module, an edge backup module, a laser projection line marker, and a cloud visualization module. The drone collaboration module acquires on-site images. Furthermore, the drone collaboration module connects to the same MQTT entity via a 5G SA (Standalone) network to achieve aerial overhead and facade inspection, improving coverage of large-span branch pipes. Even with a drone, the system can still function normally, but blind spots on the facade require the robot to navigate around them. The edge backup module stores the acquired images. Furthermore, the edge backup module is a dual-disk RAID1 (Redundant Array of Independent Disks Level 1) module: extending the original 4K storage period to 30 days for convenient high-frame-rate retrospective recording. If cost is a constraint, it can be changed to a single disk recording every 7 days. The laser projection line marker projects a cross laser beam onto the defect location using the robot. This allows workers to verify the image on-site, enhancing visualization. The cloud visualization module overlays the macro-camera image, robot trajectory, and defect points onto the BIM (Building Information Model) model in real time for browsing. The cloud-based visualization module can be used with the cloud-based digital twin visualization engine (Cesium-3D Tiles): macroscopic cameras, robot trajectories, and defect points are overlaid onto the BIM model in real time, providing an immersive viewing experience; if only a two-dimensional report is required, this engine can be omitted. This improves coverage, visualization experience, and data retention time.

[0094] The aforementioned system establishes a macro-micro fusion inspection system for pumped storage construction sites with "zero new cameras," comprehensively breaking down data silos between fixed monitoring and robots. Through online reuse calibration, event-driven uploading, cross-scale ROI clipping, and GCN-LSTM construction method understanding, this disclosure solves the problem of one-time calibration failure caused by scaffolding changes, achieves automatic updates of macro camera pose at preset intervals (e.g., 10 minutes), with an error of <2 cm; solves the bandwidth bottleneck caused by full 4K cloud upload, reducing dedicated ring network traffic by more than 80%; solves the problem of narrow field of view and easy omission of large-span equipment in the robot's first-person perspective, realizing "clicking on the macro screen to command the robot to autonomously approach for inspection"; solves the pain point of fragmented macro-micro coordinate systems and inability to trace uniformly, automatically generating a "single image" report with coordinates and defect overlay, reducing traceability time from 10 minutes to 2 seconds; and solves the inefficiency of relying on manual judgment during the construction phase, improving the identification accuracy of construction, installation, commissioning, and acceptance phases to 96%.

[0095] Although this application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of this application.

Claims

1. A robotic construction and inspection method for pumped-storage power stations integrating a monitoring system, characterized in that, include, Once the edge blending device installed in the robot is activated, it performs online multiplexing calibration of the robot and the macro camera; Based on any device selected in the macroscopic image, the edge blending device acquires the target device image and crops the target device image to obtain the target region mask; Based on the target area mask, the robot takes a picture of the target device at a preset distance. The edge blending device performs defect detection on the target device image taken by the robot and outputs the detection results.

2. The method for robotic construction and inspection of pumped-storage power stations using a fusion monitoring system according to claim 1, characterized in that, Online multiplexing calibration of robots and macro cameras includes, The robot acquires video frames from multiple macroscopic cameras and its own trajectory. The robot moves within the common field of view of the macro camera and acquires pairs of binocular images through the binocular camera; A 2D-3D association is established based on the acquired paired binocular images and video frames from multiple macroscopic cameras; Based on the established 2D-3D association, the extrinsic parameters of the macroscopic camera are solved, and the robot's trajectory parameters are used as the standard for scale normalization. At preset intervals, the robot's trajectory parameters within the predetermined time are re-bound to the macro camera, and the macro camera's extrinsic parameters are updated.

3. The method for robotic construction and inspection of pumped-storage power stations using a fusion monitoring system according to claim 2, characterized in that, Based on the acquired paired binocular images and video frames from multiple macroscopic cameras, a 2D-3D correlation is established, including: The robot walks a predetermined distance in a zigzag pattern within the public field of view of the macro camera and acquires paired binocular images through the binocular camera. Based on the acquired paired stereo images, 3D point clouds are generated and ORB features are extracted. Based on the extracted ORB features, the 2D pixels with the same names as the video frames of the acquired paired binocular images and multi-channel macro cameras are obtained. The obtained 2D pixels are bound one by one to the generated 3D point cloud to form a stable correspondence between 2D pixels and 3D spatial points that can be used to solve camera extrinsic parameters.

4. The method for robotic construction and inspection of pumped-storage power stations using a fusion monitoring system according to claim 3, characterized in that, Based on the acquired paired stereo images, 3D point clouds are generated, including: A stereo matching algorithm is used to perform stereo matching on pairs of binocular images to find the correspondence of each pixel and calculate the disparity. Based on the baseline distance, focal length, and intrinsic parameter matrix of the binocular camera, the disparity of each pixel is calculated and converted into the true depth value Z using the principle of triangulation. Using the robot's body coordinate system as a reference, the coordinates (X, Y, Z) of each pixel are calculated to form a 3D point cloud.

5. The method for robotic construction and inspection of pumped-storage power stations using a fusion monitoring system according to claim 3, characterized in that, It also includes determining whether the reprojection error exceeds a preset value based on the macroscopic camera extrinsic parameter T_macro, which includes... After projecting 3D points back onto the image plane based on the macroscopic camera extrinsic parameter T_macro, the deviation between the projected pixels and the actual pixels is obtained. Determine whether the acquired error is greater than the preset error value. If it is, recalibrate; otherwise, filter the target perception data based on the target perception data trigger conditions.

6. The method for robotic construction and inspection of pumped-storage power stations using a fusion monitoring system according to any one of claims 2-5, characterized in that, It also includes edge fusion devices filtering target perception data based on target perception data trigger conditions, specifically including... The edge fusion device detects images acquired by each macro camera to obtain the object bounding box B_macro and category C_macro for any target; Based on the obtained object bounding box B_macro and category C_macro of any target, determine whether the corresponding target meets the target perception data triggering condition. If it does, obtain the target perception data of the target and upload it to the cloud. The target perception data includes image keyframes, object bounding boxes, 3D centroid X_w obtained by backprojection from camera extrinsic parameters, timestamps, and the ID of the macro camera.

7. The method for robotic construction and inspection of pumped-storage power stations using a fusion monitoring system according to claim 6, characterized in that, Target perception data triggering conditions include satisfying any of the following conditions: If the area change rate of a target in the acquired image exceeds a preset percentage, or if a new target or a new target category is detected, the target has entered a danger zone.

8. The method for robotic construction and inspection of pumped-storage power stations using a fusion monitoring system according to claim 6, characterized in that, Based on any device selected within the macroscopic image, the edge blending device acquires an image of the target device and crops the target device image to obtain a target region mask, including... Calculate world coordinates X_w based on the pixel p_macro and depth map D_macro of any selected device; The calculated world coordinates X_w are transformed into the robot body coordinate system to obtain the relative pose X_r of the target device image; Based on X_r and real-time laser point cloud, a robot trajectory is generated, and the robot moves towards the target device based on the trajectory. As the robot moves, it generates a 3D search box centered on X_r and with a preset value multiplied by the length of the target device as the side length. This box is then projected onto the left image of the binocular view to obtain the attention mask M_macro2micro.

9. The method for robotic construction and inspection of pumped-storage power stations using a fusion monitoring system according to claim 7, characterized in that, Based on a target region mask, the robot captures images of the target device at a preset distance. An edge blending device then performs defect detection on the robot-captured images of the target device and outputs the detection results, including... The robot acquires images of the target device within a preset distance, and also acquires depth d_micro and binocular stitched images; The defect box B_micro and the reading V_micro are output from the image stitched by binoculars. The four corner points of the defect box B_micro are back-projected to the world coordinate system, and then projected back to the corresponding image frame under the macro camera according to the camera extrinsic parameters. A semi-transparent overlay layer is drawn based on the obtained image frame, and numbered bubble labels are generated simultaneously to finally obtain a micro-cropped image and interpretation results.

10. The method for robotic construction and inspection of pumped-storage power stations using a fusion monitoring system according to claim 9, characterized in that, It also includes a combined macroscopic and microscopic understanding of construction methods, specifically including, Images acquired by the robot and the macroscopic camera are sampled at a preset frame rate, and macroscopic object state vectors and microscopic motion embedding vectors are obtained respectively. Based on the acquired macroscopic object state vectors and microscopic action embedding vectors, a heterogeneous graph G=(V 总 , E), where V 总 Nodes include macroscopic object nodes V and microscopic action nodes V'. The macroscopic object node V is the macroscopic object state vector S_macro; the microscopic action node V' is the microscopic action embedding vector S_micro; E is an edge. If the robot performs an action on any macroscopic object at time t, a directed edge V'→V is established, where the weight is the reciprocal of the three-dimensional distance between the two nodes. Based on the constructed heterogeneous graph G=(V 总 (E), aggregate neighbor features and evolve along the time axis, finally outputting the construction method stage.

11. The method for robotic construction and inspection of pumped-storage power stations using a fusion monitoring system according to claim 10, characterized in that, Based on heterogeneous graph G=(V 总 The process involves aggregating neighbor features using a two-layer GCN-LSTM algorithm, evolving them along the time axis, and finally outputting the final output.

12. The method for robotic construction and inspection of pumped-storage power stations using a fusion monitoring system according to claim 11, characterized in that, This also includes edge fusion devices performing local model updates, specifically including, Each edge fusion device calculates its local model gradient based on local data at a preset interval and uploads it to the federated server; After receiving the calculated local model gradient, the federated server performs a weighted average to obtain the global model weights and then sends them to the edge fusion device. The edge fusion device updates online based on the received global model weights.

13. A robotic construction and inspection system for pumped-storage power stations integrating a monitoring system, characterized in that, include, Macro cameras are used to acquire macroscopic images of the construction site; The robot is equipped with a camera to acquire microscopic images of the target. An edge blending device, mounted on a robot, is used for online multiplexing and calibration of the robot and a macroscopic camera. Based on any device selected in the macroscopic image, it acquires the device image, crops the device image to obtain a target area mask, and performs defect detection on the target based on the acquired target area mask and the target microscopic image, and outputs the detection results.

14. The pumped-storage power station robotic construction and inspection system of the integrated monitoring system according to claim 13, characterized in that, It also includes, The drone collaboration module is used to acquire on-site images; The edge backup module is used to save the acquired images; Laser projection marking device is used by robots to project cross lasers at defect locations; The cloud-based visualization module is used to overlay macroscopic camera footage, robot trajectories, and defect points onto the BIM model in real time for viewing.

Images