A Hydropower Station Generator Rotor Auxiliary Positioning System Based on Deep Learning, Multi-view 3D Reconstruction, and Radar Ranging

Abstract

This invention discloses a hydropower station generator rotor-assisted positioning system based on deep learning, multi-view 3D reconstruction, and radar ranging. The perception layer acquires multi-view images and vertical height data of the rotor; the transmission layer achieves microsecond-level time synchronization of multi-source data; the processing layer implements closed-loop feedback adaptively adjusting constraint weights and volumetric rendering sampling density based on ICP registration residuals to generate a high-precision rotor point cloud model; and then calculates 3D offset, axis deviation angle, and shortest edge distance through a two-stage positioning process. Simultaneously, complementary filtering is used to decompose the height sequence across multiple time scales to separate the slowly decreasing trend component and the rapidly oscillating component. The application layer presents the positioning information and provides multi-level safety alarms through digital twin rendering and augmented reality guidance. This invention solves the problems of low vertical accuracy, lack of global 3D spatial perception, and inability to monitor abnormal vibrations in existing single-sensor positioning schemes.

Description

Technical Field

[0001] This invention relates to the field of hydropower station generator installation and maintenance technology, and in particular to a hydropower station generator rotor auxiliary positioning system based on deep learning multi-view three-dimensional reconstruction and radar ranging. Background Technology

[0002] Hydroelectric generator sets are the core energy conversion equipment of hydropower stations. The generator rotor, as a rotating component, can weigh hundreds of tons and have a diameter of over ten meters. During major overhauls, the rotor needs to be lifted out of the stator bore for inspection and then repositioned. The single-sided gap between the rotor and the stator bore is typically only a few millimeters to a dozen millimeters. If the rotor's placement angle or horizontal offset is too large during hoisting, the outer edge of the rotor will rub against and collide with the stator bore wall, leading to deformation of the core laminations, damage to the winding insulation, and even serious accidents such as rotor jamming and breakage of the hoisting wire rope. Therefore, during the rotor repositioning process, high-precision real-time measurement of the rotor's three-dimensional spatial position relative to the stator bore, and providing intuitive and quantitative position guidance to the hoisting operators, is a critical technical problem that urgently needs to be solved in hydropower station maintenance operations.

[0003] Currently, the hoisting and positioning of generator rotors in hydropower stations mainly relies on manual visual judgment combined with simple measuring tools. Operators stand on top of the rotor or above the stator base, visually observing changes in the gap between the rotor's outer edge and the stator's inner wall. They then use a measuring tape or handheld laser rangefinder to measure the gap width point by point, judging the rotor's current spatial attitude based on experience, and transmitting adjustment instructions to the crane operator via hand gestures or walkie-talkies. Some hydropower stations have introduced laser trackers or total stations for auxiliary measurements, installing target balls or prisms on the rotor surface to indirectly calculate the rotor's attitude by tracking the spatial coordinates of the target. In addition, a few solutions attempt to use a single industrial camera for visual 3D reconstruction or utilize millimeter-wave radar for single-point ranging in the height direction. In the field of 3D reconstruction technology, multi-view stereo matching methods can recover dense point clouds of a scene from multiple calibration images, while neural radiation field-based methods can achieve highly realistic geometric and appearance reconstruction through volume rendering optimization.

[0004] The aforementioned existing technologies all have significant shortcomings. The method of manual visual judgment combined with simple tools has low positioning accuracy; the error in operator visual judgment can reach the centimeter level, failing to meet the millimeter-level installation accuracy requirements. Furthermore, manual calculation and adjustment instructions are required after each measurement, resulting in low hoisting efficiency. Uneven lighting distribution inside hydropower stations and severe specular reflection on metal surfaces further reduce the accuracy of visual judgment. This method can only acquire single-point distance data and cannot intuitively present the three-dimensional relative positional relationship between the rotor and stator inner bores. Operators lack global spatial awareness, leading to a high risk of misjudgment. While laser trackers offer high accuracy, the equipment is expensive, requires multiple targets for collaborative sampling, has a complex data processing flow, and its real-time performance cannot meet the dynamic operational needs of hoisting sites. Single-view 3D reconstruction schemes are greatly affected by weak texture and reflection on metal surfaces, resulting in insufficient reconstruction accuracy in the vertical direction. Single radar ranging can only measure vertical height and cannot obtain horizontal offset and axis deviation, which still cannot solve the safety hazards of rotor tilting collisions. Therefore, it is necessary to design a multi-view 3D reconstruction and radar ranging auxiliary positioning system for hydropower station generator rotors based on deep learning to solve the above problems. Summary of the Invention

[0005] The technical problem to be solved by this invention is to provide a hydropower station generator rotor auxiliary positioning system based on deep learning multi-view 3D reconstruction and radar ranging. It aims to overcome the problems of low positioning accuracy, lack of global 3D spatial perception, and unreliable vertical reconstruction that exist in hydropower station generator rotor hoisting and positioning due to reliance on manual visual judgment and single sensor solutions. It features the characteristics of integrating multi-view vision and radar ranging, adaptively optimizing 3D reconstruction accuracy through closed-loop feedback, and ensuring operational safety through multi-timescale motion decomposition.

[0006] To achieve the above technical solution, the technical solution adopted by the present invention is as follows: A hydropower station generator rotor auxiliary positioning system based on deep learning multi-view 3D reconstruction and radar ranging includes: The perception layer includes a visual acquisition unit and a radar ranging unit. The visual acquisition unit includes multiple industrial cameras deployed at different angles to acquire rotor images. The radar ranging unit includes a vertically mounted millimeter-wave radar to measure the vertical height of the rotor top. The transmission layer, whose input end is connected to the output end of the perception layer, is used to synchronize the data collected by the vision acquisition unit and the radar ranging unit in time, and transmit the synchronized data to the processing layer. The processing layer, with its input end connected to the output end of the transmission layer, includes a data preprocessing module, a deep learning 3D reconstruction engine, and a real-time positioning analysis module. The data preprocessing module preprocesses the received video stream and radar ranging data, outputting the preprocessed video stream to the deep learning 3D reconstruction engine and the preprocessed radar altitude data to both the deep learning 3D reconstruction engine and the real-time positioning analysis module. The deep learning 3D reconstruction engine performs 3D reconstruction based on the preprocessed multi-view images and radar altitude data, generating a rotor point cloud model and outputting it to the real-time positioning analysis module. The deep learning 3D reconstruction engine also receives the registration residuals from the real-time positioning analysis module and adjusts the reconstruction parameters accordingly. The real-time positioning analysis module calculates the rotor's positioning parameters relative to the hole based on the rotor point cloud model and performs multi-timescale decomposition on the radar altitude data, separating the slowly decreasing trend component from the rapidly oscillating component. The application layer, whose input is connected to the output of the processing layer, is used to display the three-dimensional relative position of the rotor and the hole according to the positioning parameters, and to provide operation guidance and vibration warning according to the slow-changing downward trend component and the fast-changing oscillation component, respectively.

[0007] Furthermore, the visual acquisition unit includes three industrial-grade panoramic shutter cameras with a resolution of 2K or higher, a frame rate of no less than 30fps, and microsecond-level exposure control. Each camera is equipped with an IP67-rated anti-fog and dustproof fixed-focus lens, a working distance of 5m to 30m, and a ring-shaped LED fill light array to eliminate interference from metal surface reflections. The three cameras are symmetrically distributed at 120° in the horizontal plane, with a pitch angle of 45°.

[0008] Preferably, the radar ranging unit uses a medium-to-high frequency millimeter-wave radar with a frequency of 76GHz to 81GHz, which is vertically installed at the bottom of the hoisting equipment; the ranging accuracy of the millimeter-wave radar is at the millimeter level, the maximum range is 20m, and the sampling frequency is not less than 50Hz; the millimeter-wave radar has a built-in temperature sensor and dynamic calibration algorithm to compensate for measurement errors caused by the thermal expansion of metal, and the outer shell adopts a fully sealed metal structure to shield electromagnetic interference.

[0009] Preferably, the transport layer includes: The hardware synchronization module uses the IEEE 1588PTP precision clock protocol to achieve microsecond-level time synchronization between the industrial camera and the millimeter-wave radar. The TSN-based industrial Ethernet ring network has a backbone bandwidth of no less than 1Gbps and a single-node fault switching time of less than 50ms. A circular memory data buffer queue is used to align video frames and radar data packets by timestamp and to configure a frame loss retransmission mechanism.

[0010] Preferably, the data preprocessing module preprocesses the video stream including: Adaptive gamma correction, the correction formula is: ; in, These are the original image pixel values. To correct the pixel values ​​of the image, The adaptive gamma coefficients are calculated from the cumulative distribution function of the image gray-level histogram. Background subtraction method for segmenting moving targets, current frame With background model The difference result is ,in, For the current number Frame image in coordinates Pixel value at that location, For the background model; if Then the pixel is determined to be the foreground. As the segmentation threshold, Otsu's method is used to segment the difference map. Obtained through adaptive calculation; ORB feature point extraction: ORB feature points are extracted in the rotor region. Lucas-Kanade optical flow tracking establishes cross-frame feature point associations to track rotor motion trajectories.

[0011] Preferably, the data preprocessing module preprocesses the radar ranging data including: Median filtering is used to filter the original radar ranging sequence. Perform 3-window filtering: ; Kalman filtering, establishing a linear motion model, state vector ,in For height, Vertical velocity; state transition matrix Control input matrix Observation matrix , The radar sampling interval is given; the prediction equation is: ; ; The update equation is: ; ; ; in, To control the input, Let covariance matrix be the variance matrix. For process noise covariance, To measure the noise covariance, For Kalman gain, Radar measurement; nominal process noise covariance. Initialize based on the statistical variance of the ranging data of the millimeter-wave radar in a stationary state. The value is the square of the nominal ranging accuracy of the millimeter-wave radar.

[0012] Preferably, the deep learning 3D reconstruction engine adopts a hybrid reconstruction architecture of MVS and lightweight NeRF, including: The MVS base layer performs multi-view stereo matching on synchronized frames from three different perspectives using the PatchMatch algorithm. It calculates the depth value of each pixel and generates an initial dense point cloud. The matching cost function is: ; in, For reference image pixels, For pixels The corresponding depth estimate, For The center's neighborhood window, For the first Image For pixels In depth Next Projected coordinates in the image; The radar height constraint module measures the average height of the rotor top as measured by the millimeter-wave radar. As a constraint term added to the loss function, the total loss function is: ; in, For the loss of MVS reconstruction, To reconstruct the average height of the rotor top in the point cloud, To constrain weights; A lightweight NeRF enhancement layer employs a lightweight MLP network to implicitly express scene radiosity and density, performing volume rendering completion for weakly textured areas on the metal surface. The volume rendering formula is as follows: ; ; in, For light pixel color, Light transmittance, For volume density, For radiance, This is the sampling step size; The closed-loop feedback module is used to adaptively adjust the constraint weights based on the registration residuals after each ICP fine registration. The volume rendering sampling density distribution of the lightweight NeRF enhancement layer.

[0013] Preferably, the closed-loop feedback module performs feedback adjustments in the following manner: Calculate the average registration residual: ; in, For the number of registration points, The rotation matrix for ICP solution. The translation vector obtained by ICP. For points in the rotor point cloud, for Nearest neighbor point in the point cloud of the standard CAD model of the hole; Calculate the vertical residual components: ; in, The translation vector The vertical component, For the transformed point cloud, the first The vertical coordinates of each point The perpendicular coordinates of its nearest neighbor; Update the constraint weights based on the vertical residual components: ; in, The initial constraint weights range from 0.1 to 0.5. This is the feedback gain coefficient, with a value ranging from 1.0 to 5.0; Map the registration residuals to a volume rendering sampling density field: for residuals exceeding a threshold In the spatial region, the number of ray sampling points in that region during NeRF volume rendering is increased to the original number. times, The value can be between 2 and 4; The feedback adjustment is performed iteratively in each reconstruction-registration loop until... Converging to a preset threshold the following, The hole installation tolerance is set to one-tenth of the tolerance.

[0014] Furthermore, the MLP network for the lightweight NeRF enhancement layer adopts the following structure: The input features are processed by a multi-resolution hash grid coding layer, and the hash table size is [size missing]. The resolution has 8 levels, and the feature dimension of each level is 2. The encoded features are input into a two-layer MLP network, with 64 hidden units per layer and ReLU activation function. The output layer outputs volume density. And a 256-dimensional radiance feature vector, which is combined with the line-of-sight direction through a fully connected layer. Decode into RGB color values ; The MLP network is trained using the Adam optimizer, with an initial learning rate of... Every 200 iterations, the value decreases to 0.5 times its original value. The MVS depth map is used to initialize the NeRF sampling range along each ray: for pixels The corresponding light rays, using MVS depth estimates Centered on the sample, the sampling interval is selected: ; in The standard deviation of the MVS depth estimate is obtained from the curvature estimate of the matching cost function.

[0015] Furthermore, the deep learning 3D reconstruction engine employs a dynamic computation scheduling strategy: using the vertical velocity component in the Kalman filter state vector... The absolute value is used as the rotor speed. ,when When the full-precision reconstruction mode is activated, the model is updated every 5 frames; when When switching to key point tracking mode, only the rotor edge contour point cloud is updated, and the position change is quickly registered through the ICP algorithm.

[0016] Preferably, the real-time positioning analysis module adopts a two-stage positioning process: The first stage is coarse-grained positioning, which uses smooth height data output by Kalman filtering to determine whether the rotor has entered the effective working range of ±10cm from the target height. If it has not entered, a height guidance command is displayed on the GUI interface. If it has entered, the second stage is started. The second stage is fine-grained localization, which will reconstruct the rotor point cloud. Point cloud of the pre-set standard CAD model of holes Perform ICP iterative nearest point registration, with the objective function being: ; Solving the optimal rotation matrix using Singular Value Decomposition (SVD) Translation vector .

[0017] Preferably, the real-time positioning analysis module calculates the following positioning parameters based on the ICP registration results: 3D offset , , ; Axis deflection angle: ; in, The direction vector of the rotor's central axis. The direction vector of the standard axis of the hole; Shortest edge distance The minimum value is obtained by taking the shortest distance from all points in the rotor point cloud to the surface of the hole model.

[0018] Preferably, the real-time positioning analysis module decomposes the radar altitude data into multiple time scales in the following way: Smooth height sequence of Kalman filter output Perform complementary filtering: Slowly decreasing trend component ; fast oscillatory components ; Among them, the smoothing coefficient , The time constant is determined based on the current rate of descent. Confirmed: When hour ,when hour ; This refers to the radar sampling interval.

[0019] Preferably, the real-time positioning analysis module is also used for: Based on the slowly decreasing trend component Calculate the current rate of descent And predict the rotor's target height. Remaining time: ; The remaining time is displayed in the GUI interface of the application layer; In length of The root mean square amplitude of the rapidly varying oscillation component is calculated within the sliding window: ; when Exceeding the oscillation safety threshold The vibration warning is triggered when the oscillation safety threshold is set. It is determined based on 0.2 times the minimum permissible clearance between the rotor and the hole.

[0020] Preferably, when the root mean square amplitude of the rapidly varying oscillation component is... Exceeding the oscillation safety threshold Simultaneously, the process noise covariance matrix of the Kalman filter is adjusted: ; in, The nominal process noise covariance matrix is... This is the magnification factor, with a value ranging from 0.5 to 2.0.

[0021] Preferably, the application layer builds a cross-platform 3D visualization GUI interface based on the Unity engine, including: The digital twin rendering window displays the 3D reconstructed model of the rotor and holes in real time, and supports viewpoint rotation, scaling and translation. The parameter dashboard displays the three-dimensional offset in real time. , , axial deflection angle and shortest edge distance ; An AR guidance layer overlays dynamic directional arrows and distance scales onto a 3D model, prompting the operator to adjust the direction and magnitude. Multi-level alarm module, set when or When a yellow alert is triggered, or A red alarm is triggered and the hoisting operation is forcibly interrupted.

[0022] Furthermore, it also includes redundancy and fault tolerance mechanisms: Sensor redundancy: when any industrial camera fails, the system automatically switches to dual-view reconstruction mode and triggers an accuracy alarm. The processing layer employs a distributed computing architecture to balance the computational load, separating and deploying video stream preprocessing and 3D reconstruction tasks to ensure that the single-frame processing latency is less than 100ms. Abnormal circuit breaker: If 3D reconstruction fails for 5 consecutive seconds, the system will trigger a self-check and stop the hoisting operation. Power redundancy, supporting dual AC power supply and backup battery, maintaining system operation for no less than 30 minutes after power failure.

[0023] Preferably, a method for assisted positioning of a hydropower station generator rotor based on deep learning-based multi-view 3D reconstruction and radar ranging includes the following steps: Rotor images are acquired using multiple industrial cameras deployed at different angles, while vertical height data of the top of the rotor is acquired using a vertically mounted millimeter-wave radar. The data acquired by industrial cameras and millimeter-wave radar are synchronized in time, and the synchronized data is transmitted to the processing unit. In the processing unit, the received video stream and radar ranging data are preprocessed respectively; Three-dimensional reconstruction is performed based on preprocessed multi-view images and radar height data to generate a rotor point cloud model. The average height of the rotor top measured by radar is added as a constraint term to the loss function of the three-dimensional reconstruction. The weight of the constraint term in the loss function and the sampling density of the three-dimensional reconstruction are adjusted according to the residual feedback of subsequent positioning and registration. Based on the rotor point cloud model, the positioning parameters of the rotor relative to the hole are calculated. The positioning parameters include three-dimensional offset, axis deflection angle and shortest edge distance. Multi-timescale decomposition of radar altitude data is performed to separate the slowly varying descent trend component from the rapidly varying oscillation component; Based on the positioning parameters, the three-dimensional relative position of the rotor and the hole is displayed on the screen, and operation guidance and vibration warning are provided according to the slow downward trend component and the fast oscillation component, respectively.

[0024] Preferably, adjusting the weights of the constraint terms in the loss function and the sampling density of the 3D reconstruction based on the residual feedback from the localization and registration specifically includes: After each ICP fine registration is completed, the average registration residual is calculated: ; Calculate the vertical residual components: ; Update constraint weights for: ; For residuals exceeding the threshold In the spatial region, increase the number of ray sampling points for NeRF volume rendering to the original level. times, The value can be between 2 and 4; Iteratively execute the above feedback adjustments until... Converging to a preset threshold the following.

[0025] Preferably, the radar altitude data is decomposed into multiple time scales, specifically including: height sequence of Kalman filter output Perform complementary filtering to extract the slowly decreasing trend component: ; And the fast oscillation component: ; smoothness coefficient , The time constant is the current rate of descent. hour ,when hour , The sampling interval; The current descent rate is calculated based on the slowly varying descent trend component, and the remaining time to reach the target altitude is predicted. The root mean square amplitude within the sliding window is calculated based on the rapidly changing oscillation component. When the root mean square amplitude exceeds the safety threshold, a vibration warning is triggered, and the process noise covariance matrix of the Kalman filter is adjusted synchronously.

[0026] Preferably, a computer device includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the aforementioned deep learning-based multi-view 3D reconstruction and radar ranging hydropower station generator rotor assisted positioning method.

[0027] Preferably, a computer-readable storage medium stores computer instructions, which are used to cause a computer to execute the aforementioned deep learning-based multi-view 3D reconstruction and radar ranging method for assisted positioning of hydropower station generator rotors.

[0028] The beneficial effects of this invention are as follows: 1. This invention constructs a four-layer modular system architecture comprising a perception layer, a transmission layer, a processing layer, and an application layer. The perception layer synchronously acquires multi-view images of the rotor and vertical height data under the condition of coordinated deployment of multi-view industrial cameras and millimeter-wave radar. The transmission layer achieves microsecond-level multi-source data time synchronization through a precise clock protocol and an industrial Ethernet ring network. The processing layer adopts a hybrid reconstruction architecture of MVS and lightweight NeRF and embeds a radar height constraint term in the loss function for 3D reconstruction. At the same time, it implements closed-loop feedback adjustment of constraint weights and volume rendering sampling density based on ICP registration residuals, achieving millimeter-level 3D positioning accuracy and axis deflection angle measurement accuracy better than 0.1 degrees. This solves the technical problems of low positioning accuracy of single sensors and unreliable vertical reconstruction in the prior art.

[0029] 2. This invention separates the hoisting motion of the rotor into a slow-declining trend component and a fast-declining oscillation component by performing complementary filtering decomposition on the height sequence output by the Kalman filter. The trend component is used to calculate the current descent rate and predict the remaining time to reach the target height, while the oscillation component is used to calculate the vibration amplitude in real time and trigger an early warning when the vibration exceeds the safety threshold. At the same time, the process noise covariance matrix of the Kalman filter is adaptively adjusted, achieving a balance between positioning accuracy and operational safety. This solves the problem in the prior art where the mixed estimation of two types of motion with different time scales leads to lag in height estimation and the inability to independently monitor abnormal vibrations.

[0030] 3. This invention constructs a cross-platform 3D visualization interface at the application layer, renders and reconstructs the 3D model of the rotor and stator inner hole in real time in the form of a digital twin, and superimposes dynamic directional arrows and distance scales on the model to form an augmented reality guidance layer. The parameter dashboard is composed of 3D offset, axis deviation angle and shortest edge distance. With the help of a multi-level alarm mechanism, abnormal deviation is given graded warnings and forced interruption. This solves the problem of lack of global 3D spatial perception and quantitative operation guidance in existing manual operations and reduces the dependence on the operator's experience. Attached Figure Description

[0031] Figure 1 This is a schematic diagram of the overall system architecture according to an embodiment of the present invention; Figure 2 This is a schematic diagram of device deployment according to an embodiment of the present invention; Figure 3 This is a flowchart illustrating the core algorithm processing in an embodiment of the present invention. Figure 4 This is a diagram of the hybrid 3D reconstruction engine architecture according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the two-stage localization algorithm according to an embodiment of the present invention; Figure 6 This is a schematic diagram of the system GUI interface according to an embodiment of the present invention; Figure 7 This is a schematic diagram illustrating the comparative advantages of the method in ICP registration accuracy compared to existing technologies in this embodiment of the invention; Figure 8 A schematic diagram illustrating the separation effect of the height signal during rotor hoisting in this embodiment of the invention; Figure 9 This is a structural diagram of a computer device in an embodiment of the present invention. Detailed Implementation

[0032] Example 1: like Figure 1 As shown, a hydropower station generator rotor auxiliary positioning system based on deep learning multi-view 3D reconstruction and radar ranging includes: The perception layer includes a visual acquisition unit and a radar ranging unit. The visual acquisition unit includes multiple industrial cameras deployed at different angles to acquire rotor images. The radar ranging unit includes a vertically mounted millimeter-wave radar to measure the vertical height of the rotor top. The transmission layer, whose input end is connected to the output end of the perception layer, is used to synchronize the data collected by the vision acquisition unit and the radar ranging unit in time, and transmit the synchronized data to the processing layer. The processing layer, with its input end connected to the output end of the transmission layer, includes a data preprocessing module, a deep learning 3D reconstruction engine, and a real-time positioning analysis module. The data preprocessing module preprocesses the received video stream and radar ranging data, outputting the preprocessed video stream to the deep learning 3D reconstruction engine and the preprocessed radar altitude data to both the deep learning 3D reconstruction engine and the real-time positioning analysis module. The deep learning 3D reconstruction engine performs 3D reconstruction based on the preprocessed multi-view images and radar altitude data, generating a rotor point cloud model and outputting it to the real-time positioning analysis module. The deep learning 3D reconstruction engine also receives the registration residuals from the real-time positioning analysis module and adjusts the reconstruction parameters accordingly. The real-time positioning analysis module calculates the rotor's positioning parameters relative to the hole based on the rotor point cloud model and performs multi-timescale decomposition on the radar altitude data, separating the slowly decreasing trend component from the rapidly oscillating component. The application layer, whose input is connected to the output of the processing layer, is used to display the three-dimensional relative position of the rotor and the hole according to the positioning parameters, and to provide operation guidance and vibration warning according to the slow-changing downward trend component and the fast-changing oscillation component, respectively.

[0033] like Figure 2 As shown, the visual acquisition unit includes three industrial-grade panoramic shutter cameras with a resolution of 2K or higher, a frame rate of no less than 30fps, and microsecond-level exposure control. Each camera is equipped with an IP67-rated anti-fog and dustproof fixed-focus lens, a working distance of 5m to 30m, and a ring-shaped LED fill light array to eliminate interference from metal surface reflections. The three cameras are symmetrically distributed at 120° in the horizontal plane, with a pitch angle of 45°.

[0034] Preferably, the radar ranging unit uses a medium-to-high frequency millimeter-wave radar with a frequency of 76GHz to 81GHz, which is vertically installed at the bottom of the hoisting equipment; the ranging accuracy of the millimeter-wave radar is at the millimeter level, the maximum range is 20m, and the sampling frequency is not less than 50Hz; the millimeter-wave radar has a built-in temperature sensor and dynamic calibration algorithm to compensate for measurement errors caused by the thermal expansion of metal, and the outer shell adopts a fully sealed metal structure to shield electromagnetic interference.

[0035] Preferably, the transport layer includes: The hardware synchronization module uses the IEEE 1588PTP precision clock protocol to achieve microsecond-level time synchronization between the industrial camera and the millimeter-wave radar. The TSN-based industrial Ethernet ring network has a backbone bandwidth of no less than 1Gbps and a single-node fault switching time of less than 50ms. A circular memory data buffer queue is used to align video frames and radar data packets by timestamp and to configure a frame loss retransmission mechanism.

[0036] Preferably, the data preprocessing module preprocesses the video stream including: Adaptive gamma correction, the correction formula is: ; in, These are the original image pixel values. To correct the pixel values ​​of the image, The adaptive gamma coefficients are calculated from the cumulative distribution function of the image gray-level histogram. Background subtraction method for segmenting moving targets, current frame With background model The difference result is ,in, For the current number Frame image in coordinates Pixel value at that location, For the background model; if Then the pixel is determined to be the foreground. As the segmentation threshold, Otsu's method is used to segment the difference map. Obtained through adaptive calculation; ORB feature point extraction: ORB feature points are extracted in the rotor region. Lucas-Kanade optical flow tracking establishes cross-frame feature point associations to track rotor motion trajectories.

[0037] Furthermore, the specific implementation of ORB feature point extraction is as follows: The FAST corner detection algorithm is used for feature point detection, with a threshold of 20. If the absolute value of the grayscale difference between a pixel and at least 12 consecutive pixels within a circular neighborhood of radius 3 exceeds this threshold, the pixel is determined to be a candidate corner point. Non-maximum suppression is applied to the detected candidate corner points, retaining only the top 500 corner points with the largest response values ​​as the final feature points. A binary description vector is constructed for each feature point using the BRIEF descriptor, with the descriptor dimension set to 256 bits. In the rotor image, the feature point extraction region is limited to the foreground mask region segmented by background subtraction. Feature point detection is performed only within the rotor region, excluding invalid feature points in the background region.

[0038] Furthermore, the specific implementation of Lucas-Kanade optical flow tracking is as follows: An image pyramid is used to perform multi-scale optical flow calculations. The pyramid has three layers, with the resolution decreasing by a factor of 0.5 from the top to the bottom. In each layer's Lucas-Kanade optical flow calculation, the neighborhood window size is set to 21 pixels multiplied by 21 pixels. The Newton-Raphson method is used for iterative optical flow calculation, with the iteration termination condition set as the residual vector magnitude being less than 0.03 pixels or the number of iterations reaching 30. Optical flow is calculated layer by layer starting from the top of the pyramid. The calculation result of each layer is used as the initial value for the next layer for projection mapping. Finally, the cross-frame pixel displacement vector of each feature point is output at the original image resolution layer. If the optical flow tracking residual of a certain feature point exceeds 3.0 pixels, the feature point is considered to have failed tracking and is discarded to avoid incorrect matching affecting subsequent calculations.

[0039] Preferably, the data preprocessing module preprocesses the radar ranging data including: Median filtering is used to filter the original radar ranging sequence. Perform 3-window filtering: ; Kalman filtering, establishing a linear motion model, state vector ,in For height, Vertical velocity; state transition matrix Control input matrix Observation matrix , The radar sampling interval is given; the prediction equation is: ; ; The update equation is: ; ; ; in, To control the input, Let covariance matrix be the variance matrix. For process noise covariance, To measure the noise covariance, For Kalman gain, Radar measurement; nominal process noise covariance. Initialize based on the statistical variance of the ranging data of the millimeter-wave radar in a stationary state. The value is the square of the nominal ranging accuracy of the millimeter-wave radar.

[0040] like Figure 3and Figure 4 As shown, preferably, the deep learning 3D reconstruction engine adopts a hybrid reconstruction architecture of MVS and lightweight NeRF, including: The MVS base layer performs multi-view stereo matching on synchronized frames from three different perspectives using the PatchMatch algorithm. It calculates the depth value of each pixel and generates an initial dense point cloud. The matching cost function is: ; in, For reference image pixels, For pixels The corresponding depth estimate, For The center's neighborhood window, For the first Image For pixels In depth Next Projected coordinates in the image; The radar height constraint module measures the average height of the rotor top as measured by the millimeter-wave radar. As a constraint term added to the loss function, the total loss function is: ; in, For the loss of MVS reconstruction, To reconstruct the average height of the rotor top in the point cloud, To constrain weights; A lightweight NeRF enhancement layer employs a lightweight MLP network to implicitly express scene radiosity and density, performing volume rendering completion for weakly textured areas on the metal surface. The volume rendering formula is as follows: ; ; in, For light pixel color, Light transmittance, For volume density, For radiance, This is the sampling step size; The closed-loop feedback module is used to adaptively adjust the constraint weights based on the registration residuals after each ICP fine registration. The volume rendering sampling density distribution of the lightweight NeRF enhancement layer.

[0041] Furthermore, when the system switches to key point tracking mode, the specific implementation of ICP fast registration is as follows: During the initial registration, the rotation matrix and translation vector are initialized using the rotor height data output from the coarse positioning stage. The initial value of the rotation matrix is ​​set to the identity matrix, and the initial value of the vertical component of the translation vector is set to the coarse positioning height difference. The convergence condition for ICP iteration is set as follows: the change in the root mean square registration residual between two consecutive iterations is less than 0.01 mm or the number of iterations reaches 50. In each iteration, a KD tree is used to accelerate the nearest neighbor search, and the leaf node capacity of the KD tree is set to 10 points. When the root mean square value of the registration residual exceeds 5.0 mm, a registration anomaly is determined, and a restart of full-precision 3D reconstruction is triggered to update the point cloud model.

[0042] Preferably, the closed-loop feedback module performs feedback adjustments in the following manner: Calculate the average registration residual: ; in, For the number of registration points, The rotation matrix for ICP solution. The translation vector obtained by ICP. For points in the rotor point cloud, for Nearest neighbor point in the point cloud of the standard CAD model of the hole; Calculate the vertical residual components: ; in, The translation vector The vertical component, For the transformed point cloud, the first The vertical coordinates of each point The perpendicular coordinates of its nearest neighbor; Update the constraint weights based on the vertical residual components: ; in, The initial constraint weights range from 0.1 to 0.5. This is the feedback gain coefficient, with a value ranging from 1.0 to 5.0; Map the registration residuals to a volume rendering sampling density field: for residuals exceeding a threshold In the spatial region, the number of ray sampling points in that region during NeRF volume rendering is increased to the original number. times, The value can be between 2 and 4; The feedback adjustment is performed iteratively in each reconstruction-registration loop until... Converging to a preset threshold the following, The hole installation tolerance is set to one-tenth of the tolerance.

[0043] Furthermore, the MLP network for the lightweight NeRF enhancement layer adopts the following structure: The input features are processed by a multi-resolution hash grid coding layer, and the hash table size is [size missing]. The resolution has 8 levels, and the feature dimension of each level is 2. The encoded features are input into a two-layer MLP network, with 64 hidden units per layer and ReLU activation function. The output layer outputs volume density. And a 256-dimensional radiance feature vector, which is combined with the line-of-sight direction through a fully connected layer. Decode into RGB color values ; The MLP network is trained using the Adam optimizer, with an initial learning rate of... Every 200 iterations, the value decreases to 0.5 times its original value. The MVS depth map is used to initialize the NeRF sampling range along each ray: for pixels The corresponding light rays, using MVS depth estimates Centered on the sample, the sampling interval is selected: ; in The standard deviation of the MVS depth estimate is obtained from the curvature estimate of the matching cost function.

[0044] Furthermore, the deep learning 3D reconstruction engine employs a dynamic computation scheduling strategy: using the vertical velocity component in the Kalman filter state vector... The absolute value is used as the rotor speed. ,when When the full-precision reconstruction mode is activated, the model is updated every 5 frames; when When switching to key point tracking mode, only the rotor edge contour point cloud is updated, and the position change is quickly registered through the ICP algorithm.

[0045] like Figure 5 As shown, preferably, the real-time positioning analysis module adopts a two-stage positioning process: The first stage is coarse-grained positioning, which uses smooth height data output by Kalman filtering to determine whether the rotor has entered the effective working range of ±10cm from the target height. If it has not entered, a height guidance command is displayed on the GUI interface. If it has entered, the second stage is started. The second stage is fine-grained localization, which will reconstruct the rotor point cloud. Point cloud of the pre-set standard CAD model of holes Perform ICP iterative nearest point registration, with the objective function being: ; Solving the optimal rotation matrix using Singular Value Decomposition (SVD) Translation vector .

[0046] Preferably, the real-time positioning analysis module calculates the following positioning parameters based on the ICP registration results: 3D offset , , ; Axis deflection angle: ; in, The direction vector of the rotor's central axis. The direction vector of the standard axis of the hole; Shortest edge distance The minimum value is obtained by taking the shortest distance from all points in the rotor point cloud to the surface of the hole model.

[0047] Preferably, the real-time positioning analysis module decomposes the radar altitude data into multiple time scales in the following way: Smooth height sequence of Kalman filter output Perform complementary filtering: Slowly decreasing trend component ; fast oscillatory components ; Among them, the smoothing coefficient , The time constant is determined based on the current rate of descent. Confirmed: When hour ,when hour ; This refers to the radar sampling interval.

[0048] Preferably, the real-time positioning analysis module is also used for: Based on the slowly decreasing trend component Calculate the current rate of descent And predict the rotor's target height. Remaining time: ; The remaining time is displayed in the GUI interface of the application layer; In length of The root mean square amplitude of the rapidly varying oscillation component is calculated within the sliding window: ; when Exceeding the oscillation safety threshold The vibration warning is triggered when the oscillation safety threshold is set. It is determined based on 0.2 times the minimum permissible clearance between the rotor and the hole.

[0049] Preferably, when the root mean square amplitude of the rapidly varying oscillation component is... Exceeding the oscillation safety threshold Simultaneously, the process noise covariance matrix of the Kalman filter is adjusted: ; in, The nominal process noise covariance matrix is... This is the magnification factor, with a value ranging from 0.5 to 2.0.

[0050] like Figure 6 As shown, preferably, the application layer builds a cross-platform 3D visualization GUI interface based on the Unity engine, including: The digital twin rendering window displays the 3D reconstructed model of the rotor and holes in real time, and supports viewpoint rotation, scaling and translation. The parameter dashboard displays the three-dimensional offset in real time. , , axial deflection angle and shortest edge distance ; An AR guidance layer overlays dynamic directional arrows and distance scales onto a 3D model, prompting the operator to adjust the direction and magnitude. Multi-level alarm module, set when or When a yellow alert is triggered, or A red alarm is triggered and the hoisting operation is forcibly interrupted.

[0051] Furthermore, it also includes redundancy and fault tolerance mechanisms: Sensor redundancy: when any industrial camera fails, the system automatically switches to dual-view reconstruction mode and triggers an accuracy alarm. The processing layer employs a distributed computing architecture to balance the computational load, separating and deploying video stream preprocessing and 3D reconstruction tasks to ensure that the single-frame processing latency is less than 100ms. Abnormal circuit breaker: If 3D reconstruction fails for 5 consecutive seconds, the system will trigger a self-check and stop the hoisting operation. Power redundancy, supporting dual AC power supply and backup battery, maintaining system operation for no less than 30 minutes after power failure.

[0052] Preferably, a method for assisted positioning of a hydropower station generator rotor based on deep learning-based multi-view 3D reconstruction and radar ranging includes the following steps: Rotor images are acquired using multiple industrial cameras deployed at different angles, while vertical height data of the top of the rotor is acquired using a vertically mounted millimeter-wave radar. The data acquired by industrial cameras and millimeter-wave radar are synchronized in time, and the synchronized data is transmitted to the processing unit. In the processing unit, the received video stream and radar ranging data are preprocessed respectively; Three-dimensional reconstruction is performed based on preprocessed multi-view images and radar height data to generate a rotor point cloud model. The average height of the rotor top measured by radar is added as a constraint term to the loss function of the three-dimensional reconstruction. The weight of the constraint term in the loss function and the sampling density of the three-dimensional reconstruction are adjusted according to the residual feedback of subsequent positioning and registration. Based on the rotor point cloud model, the positioning parameters of the rotor relative to the hole are calculated. The positioning parameters include three-dimensional offset, axis deflection angle and shortest edge distance. Multi-timescale decomposition of radar altitude data is performed to separate the slowly varying descent trend component from the rapidly varying oscillation component; Based on the positioning parameters, the three-dimensional relative position of the rotor and the hole is displayed on the screen, and operation guidance and vibration warning are provided according to the slow downward trend component and the fast oscillation component, respectively.

[0053] Preferably, adjusting the weights of the constraint terms in the loss function and the sampling density of the 3D reconstruction based on the residual feedback from the localization and registration specifically includes: After each ICP fine registration is completed, the average registration residual is calculated: ; Calculate the vertical residual components: ; Update constraint weights for: ; For residuals exceeding the threshold In the spatial region, increase the number of ray sampling points for NeRF volume rendering to the original level. times, The value can be between 2 and 4; Iteratively execute the above feedback adjustments until... Converging to a preset threshold the following.

[0054] Preferably, the radar altitude data is decomposed into multiple time scales, specifically including: height sequence of Kalman filter output Perform complementary filtering to extract the slowly decreasing trend component: ; And the fast oscillation component: ; smoothness coefficient , The time constant is the current rate of descent. hour ,when hour , The sampling interval; The current descent rate is calculated based on the slowly varying descent trend component, and the remaining time to reach the target altitude is predicted. The root mean square amplitude within the sliding window is calculated based on the rapidly changing oscillation component. When the root mean square amplitude exceeds the safety threshold, a vibration warning is triggered, and the process noise covariance matrix of the Kalman filter is adjusted synchronously.

[0055] Preferably, a computer device includes a memory and a processor, which are interconnected. The memory stores computer instructions, and the processor executes the computer instructions to perform the aforementioned deep learning-based multi-view 3D reconstruction and radar ranging hydropower station generator rotor assisted positioning method.

[0056] Preferably, a computer-readable storage medium stores computer instructions, which are used to cause a computer to execute the aforementioned deep learning-based multi-view 3D reconstruction and radar ranging method for assisted positioning of hydropower station generator rotors.

[0057] Example 2: This embodiment uses the reinstallation of a generator rotor during a major overhaul of a hydropower station unit as an application scenario. The rotor has a diameter of 8.5m, a mass of 180t, a stator inner diameter of 8.54m, and a single-sided clearance of 20mm. A bridge crane equipped with a wire rope winding system is used for hoisting. The experimental platform is built according to the S1 simulation scenario. Three industrial cameras are used with 2K resolution, 30fps frame rate, and microsecond-level exposure control. They are symmetrically distributed at 120 degrees in the horizontal plane and fixed at a 45-degree elevation angle. Each camera is equipped with a ring-shaped LED fill light array. The millimeter-wave radar uses a 76-81GHz mid-to-high frequency radar, vertically mounted at the bottom of the lifting device, with a sampling frequency set to 50Hz and a PTP clock synchronization accuracy of microseconds. The processing layer runs on distributed edge computing nodes, with video preprocessing and 3D reconstruction tasks deployed separately.

[0058] The key parameter is set as follows: FAST corner detection threshold in ORB feature point extraction. The value is set to 20, the upper limit for non-maximum suppression to retain feature points is 500, and the BRIEF descriptor dimension is 256 bits. The Lucas-Kanade optical flow pyramid is set to 3 layers, the neighborhood window is 21 pixels by 21 pixels, and the iteration termination residual threshold is 0.03 pixels. Segmentation threshold The Kalman filter state vector is calculated online adaptively using the Otsu method. Sampling interval The nominal process noise covariance is 0.02s. Initialize the ranging variance based on the radar stationary state, and measure the noise covariance. The value is taken as the square of the radar's nominal accuracy of 1 mm. Initial weight for radar altitude constraint. The value is 0.3, which is the feedback gain coefficient. The value is 2.0, which represents the sampling density amplification factor. The value is 3. Complementary filter time constant. The value is 3.0s when the descent rate is below 2cm / s and 1.0s when it is not below 2cm / s. Oscillation safety threshold. The value is 4mm, which is the length of the sliding window. Take 100 sampling points. ICP registration convergence threshold. The value is 1mm. The yellow warning threshold for the centerline deviation angle of the multi-level alarm is 1 degree, and the red alarm threshold is 2 degrees. The yellow warning threshold for the shortest edge distance is 3cm, and the red alarm threshold is 1cm.

[0059] The comparative test employed two existing methods. Method 1 is a single-view visual reconstruction combined with a multi-radar ranging scheme. It uses one industrial camera of the same specifications to photograph the rotor from directly above, and three millimeter-wave radars to triangulate the horizontal offset. It does not use deep learning 3D reconstruction or radar height constraints. Method 2 is a standard MVS reconstruction scheme with fixed-constraint weights (NeRF). It uses the same three cameras as this invention, but with different radar height constraint weights. The value is fixed at 0.3, and the closed-loop feedback adaptive adjustment and multi-timescale motion decoupling mechanism are not enabled.

[0060] The complete process of the embodiment is as follows: The rotor starts from an initial height of 15m and is slowly lowered under the control of the crane operator. The system starts recording positioning data from the moment the rotor enters the range of 1m above the hole and continues to record until the rotor reaches the target height and completes its landing. The entire process takes about 120 seconds. Each working condition is repeated 3 times, and the average value is taken as the final result. The operating data is shown in Tables 1 and 2 below.

[0061] Table 1: Positioning accuracy data of the method of the present invention in different height ranges;

[0062] Table 2: Comparison of positioning accuracy of the three schemes within ±10cm of the target height;

[0063] Based on the comparative data above, the method of this invention achieves significant improvements over the two existing schemes in several key indicators. Regarding positioning accuracy, the average ICP registration residual of this method within a target height range of ±10cm is 0.7mm, far superior to the 4.8mm of Comparative Method 1 and the 1.8mm of Comparative Method 2. This indicates that the closed-loop feedback adaptive reconstruction mechanism effectively improves the 3D reconstruction quality, thereby enhancing positioning accuracy. The vertical deviation in the 3D offset is only 0.7mm, demonstrating that the radar height constraint plays a crucial role in compensating for insufficient vertical accuracy in visual reconstruction. Comparative Method 1, lacking this constraint, suffers a vertical deviation of 1.8mm. In terms of axis deflection measurement, this method achieves an accuracy of 0.07 degrees, effectively preventing edge collisions caused by rotor tilting. Comparative Method 1 suffers an angle error of 0.65 degrees due to severe occlusion during single-view reconstruction. Regarding convergence efficiency, this method requires only 7 iterations to converge, while Comparative Method 2 requires 12. The accelerated convergence effect comes from the adaptive driving of the reconstruction parameters by the ICP residual feedback. The vibration warning positive report rate reached 95%, verifying the effective monitoring capability of the multi-timescale motion decoupling mechanism for abnormal vibrations during hoisting—a feature not possessed by the two comparative methods. The single-frame processing latency was consistently controlled within 100ms, meeting the real-time operational feedback requirements of the hoisting site. In summary, this invention achieves synergistic optimization in positioning accuracy, convergence speed, and safety warning across three core technological innovations: deep fusion of multi-view vision and millimeter-wave radar, closed-loop feedback adaptive optimization, and multi-timescale motion decoupling.

[0064] Figure 7 This demonstrates the comparative advantages of the closed-loop feedback adaptive reconstruction mechanism of this invention over the standard MVS plus NeRF reconstruction mode with fixed constraint weights in ICP registration accuracy. In the standard mode, the radar altitude constraint weights... The value is fixed at 0.3 and does not adjust with changes in reconstruction quality. The ICP registration residual decreases slowly during the iteration process, converging to a root mean square residual of approximately 2.8 mm after about 12 reconstruction registration cycles. In the closed-loop feedback mode of this invention, the vertical residual component of ICP registration... Real-time driving constraint weights The adaptive adjustment mechanism increases the weights when the initial residual is high to enhance the correction effect of radar constraints. As the residual gradually decreases, the weights slowly decay to avoid over-constraint. Simultaneously, NeRF volume rendering automatically increases the sampling density in spatial regions with high residuals to supplement local geometric details. This mechanism enables the ICP registration residual in the closed-loop feedback mode to converge to approximately 1.2 mm after about 7 iterations, improving the convergence speed by about 40% and reducing the steady-state residual by about 57%. This improved registration accuracy directly enhances the calculation accuracy of 3D offset and axis deviation in subsequent positioning parameters, showing a significant advantage, especially in estimating vertical offset.

[0065] Figure 8 This paper demonstrates the effectiveness of the multi-timescale motion decoupling method of this invention in separating height signals during rotor hoisting. The original radar height signal contains two types of motion components with different physical sources: controlled descent motion and environmental vibration. After decomposition using complementary filtering, the slowly varying descent trend component accurately reflects the stable descent rate of the rotor controlled by the operator, while the rapidly varying oscillation component obtained after deducting this trend component clearly presents the vibration characteristics caused by the elastic swing of the hoisting cable and external wind loads. The trend component accounts for approximately 95% of the energy in the signal, but its frequency is mainly concentrated in the low-frequency range of 0 to 0.5 Hz; the oscillation component accounts for approximately 5% of the energy, with a frequency distribution in the range of 0.5 to 5 Hz and an amplitude between 2 and 8 mm. Through the above decomposition, the system eliminates vibration interference when calculating the rotor descent rate and predicting the remaining time to reach the target height, and eliminates the occlusion effect of the descent trend during vibration early warning monitoring. This separation effect cannot be achieved by a single Kalman filter because the Kalman filter estimates both types of motion in the same state vector, resulting in a descent rate estimation lag of approximately 0.3 seconds and the inability to output independent vibration amplitude indicators.

[0066] Example 3: like Figure 9 As shown, this embodiment of the invention also provides a computer device, which includes one or more processors 10, a memory 20, and interfaces for connecting the various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory for displaying graphical information of a GUI on external input / output devices, such as display devices coupled to the interfaces. In some alternative embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations, for example, as a server array, a group of blade servers, or a multiprocessor system.

[0067] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GDA), or any combination thereof.

[0068] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0069] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0070] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0071] The computer device also includes a communication interface 30 for communicating with other devices or communication networks.

[0072] This invention also provides a computer-readable storage medium. The methods described above according to embodiments of the invention can be implemented in hardware or firmware, or implemented as computer code that can be recorded on a storage medium, or implemented as computer code downloaded via a network and originally stored on a remote storage medium or a non-transitory machine-readable storage medium and then stored on a local storage medium. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium can be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium can also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code, which, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0073] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A hydropower station generator rotor auxiliary positioning system based on deep learning multi-view 3D reconstruction and radar ranging, characterized in that the system... include: The perception layer includes a visual acquisition unit and a radar ranging unit. The visual acquisition unit includes multiple industrial cameras deployed at different angles to acquire rotor images. The radar ranging unit includes a vertically mounted millimeter-wave radar to measure the vertical height of the rotor top. The transmission layer, whose input end is connected to the output end of the perception layer, is used to synchronize the data collected by the vision acquisition unit and the radar ranging unit in time, and transmit the synchronized data to the processing layer. The processing layer, with its input end connected to the output end of the transmission layer, includes a data preprocessing module, a deep learning 3D reconstruction engine, and a real-time positioning analysis module. The data preprocessing module preprocesses the received video stream and radar ranging data, outputting the preprocessed video stream to the deep learning 3D reconstruction engine and the preprocessed radar altitude data to both the deep learning 3D reconstruction engine and the real-time positioning analysis module. The deep learning 3D reconstruction engine performs 3D reconstruction based on the preprocessed multi-view images and radar altitude data, generating a rotor point cloud model and outputting it to the real-time positioning analysis module. The deep learning 3D reconstruction engine also receives the registration residuals from the real-time positioning analysis module and adjusts the reconstruction parameters accordingly. The real-time positioning analysis module calculates the rotor's positioning parameters relative to the hole based on the rotor point cloud model and performs multi-timescale decomposition on the radar altitude data, separating the slowly decreasing trend component from the rapidly oscillating component. The application layer, whose input is connected to the output of the processing layer, is used to display the three-dimensional relative position of the rotor and the hole according to the positioning parameters, and to provide operation guidance and vibration warning according to the slow-changing downward trend component and the fast-changing oscillation component, respectively.

2. The hydropower station generator rotor auxiliary positioning system based on deep learning multi-view 3D reconstruction and radar ranging according to claim 1, characterized in that, The radar ranging unit adopts a medium-to-high frequency millimeter-wave radar, which is vertically installed at the bottom of the hoisting equipment. The ranging accuracy of the millimeter-wave radar is at the millimeter level. The millimeter-wave radar has a built-in temperature sensor and dynamic calibration algorithm to compensate for measurement errors caused by the thermal expansion of metal. The outer shell adopts a fully sealed metal structure to shield electromagnetic interference.

3. The hydropower station generator rotor auxiliary positioning system based on deep learning multi-view 3D reconstruction and radar ranging according to claim 1, characterized in that, The transport layer includes: The hardware synchronization module uses the IEEE 1588PTP precision clock protocol to achieve microsecond-level time synchronization between the industrial camera and the millimeter-wave radar. TSN-based industrial Ethernet ring network; A circular memory data buffer queue is used to align video frames and radar data packets by timestamp and to configure a frame loss retransmission mechanism.

4. The hydropower station generator rotor auxiliary positioning system based on deep learning multi-view 3D reconstruction and radar ranging according to claim 1, characterized in that, The data preprocessing module performs the following preprocessing on the video stream: Adaptive gamma correction, the correction formula is: ； in, These are the original image pixel values. To correct the pixel values ​​of the image, The adaptive gamma coefficients are calculated from the cumulative distribution function of the image gray-level histogram. Background subtraction method for segmenting moving targets, current frame With background model The difference result is ,in, For the current number Frame image in coordinates Pixel value at that location, For the background model; if Then the pixel is determined to be the foreground. As the segmentation threshold, Otsu's method is used to segment the difference map. Obtained through adaptive calculation; ORB feature point extraction: ORB feature points are extracted in the rotor region. Lucas-Kanade optical flow tracking establishes cross-frame feature point associations to track rotor motion trajectories.

5. The hydropower station generator rotor auxiliary positioning system based on deep learning multi-view 3D reconstruction and radar ranging according to claim 1, characterized in that, The data preprocessing module preprocesses the radar ranging data, including: Median filtering is used to filter the original radar ranging sequence. Perform 3-window filtering: ； Kalman filtering, establishing a linear motion model, state vector ,in For height, Vertical velocity; state transition matrix Control input matrix Observation matrix , The radar sampling interval is given; the prediction equation is: ； ； The update equation is: ； ； ； in, To control the input, Let covariance matrix be the variance matrix. For process noise covariance, To measure the noise covariance, For Kalman gain, Radar measurement; nominal process noise covariance. Initialize based on the statistical variance of the ranging data of the millimeter-wave radar in a stationary state. The value is the square of the nominal ranging accuracy of the millimeter-wave radar.

6. The hydropower station generator rotor auxiliary positioning system based on deep learning multi-view 3D reconstruction and radar ranging according to claim 1, characterized in that, The deep learning-based 3D reconstruction engine employs a hybrid reconstruction architecture combining MVS and lightweight NeRF, including: The MVS base layer performs multi-view stereo matching on synchronized frames from three different perspectives using the PatchMatch algorithm. It calculates the depth value of each pixel and generates an initial dense point cloud. The matching cost function is: ； in, For reference image pixels, For pixels The corresponding depth estimate, For The center's neighborhood window, For the first Image For pixels In depth Next Projected coordinates in the image; The radar height constraint module measures the average height of the rotor top as measured by the millimeter-wave radar. As a constraint term added to the loss function, the total loss function is: ； in, For the loss of MVS reconstruction, To reconstruct the average height of the rotor top in the point cloud, To constrain weights; A lightweight NeRF enhancement layer employs a lightweight MLP network to implicitly express scene radiosity and density, performing volume rendering completion for weakly textured areas on the metal surface. The volume rendering formula is as follows: ； ； in, For light pixel color, Light transmittance, For volume density, For radiance, This is the sampling step size; The closed-loop feedback module is used to adaptively adjust the constraint weights based on the registration residuals after each ICP fine registration. The volume rendering sampling density distribution of the lightweight NeRF enhancement layer.

7. The hydropower station generator rotor auxiliary positioning system based on deep learning multi-view 3D reconstruction and radar ranging according to claim 6, characterized in that, The closed-loop feedback module performs feedback adjustments as follows: Calculate the average registration residual: ； in, For the number of registration points, The rotation matrix for ICP solution. The translation vector obtained by ICP. For points in the rotor point cloud, for Nearest neighbor point in the point cloud of the standard CAD model of the hole; Calculate the vertical residual components: ； in, The translation vector The vertical component, For the transformed point cloud, the first The vertical coordinates of each point The perpendicular coordinates of its nearest neighbor; Update the constraint weights based on the vertical residual components: ； in, These are the initial constraint weights. This is the feedback gain coefficient; Map the registration residuals to a volume rendering sampling density field: for residuals exceeding a threshold In the spatial region, the number of ray sampling points in that region during NeRF volume rendering is increased to the original number. times; Feedback adjustments are performed iteratively in each reconstruction-registration loop until... Converging to a preset threshold the following, The hole installation tolerance is set to one-tenth of the tolerance.

8. The hydropower station generator rotor auxiliary positioning system based on deep learning multi-view 3D reconstruction and radar ranging according to claim 1, characterized in that, The real-time positioning analysis module adopts a two-stage positioning process: The first stage is coarse-grained positioning, which uses smoothed height data output by Kalman filtering to determine whether the rotor has entered the effective working area within the threshold range of the target height. If it has not entered, a height guidance command is displayed on the GUI interface. If it has entered, the second stage is started. The second stage is fine-grained localization, which will reconstruct the rotor point cloud. Point cloud of the pre-set standard CAD model of holes Perform ICP iterative nearest point registration, with the objective function being: ； Solving the optimal rotation matrix using Singular Value Decomposition (SVD) Translation vector .

9. The hydropower station generator rotor auxiliary positioning system based on deep learning multi-view 3D reconstruction and radar ranging according to claim 8, characterized in that, The real-time positioning analysis module calculates the following positioning parameters based on the ICP registration results: 3D offset , , ; Axis deflection angle: ； in, The direction vector of the rotor's central axis. Vector of the standard axis direction of the hole; shortest edge distance The minimum value is obtained by taking the shortest distance from all points in the rotor point cloud to the surface of the hole model.

10. The hydropower station generator rotor auxiliary positioning system based on deep learning multi-view 3D reconstruction and radar ranging according to claim 1, characterized in that, The real-time positioning analysis module decomposes radar altitude data into multiple time scales as follows: Smooth height sequence of Kalman filter output Perform complementary filtering: Slowly decreasing trend component ; fast oscillatory components ; Among them, the smoothing coefficient , The time constant is determined based on the current rate of descent. Sure, This refers to the radar sampling interval.

11. The hydropower station generator rotor auxiliary positioning system based on deep learning multi-view 3D reconstruction and radar ranging according to claim 12, characterized in that, The real-time location analysis module is also used for: Based on the slowly decreasing trend component Calculate the current rate of descent And predict the rotor's target height. Remaining time: ； The remaining time is displayed in the GUI interface of the application layer; In length of The root mean square amplitude of the rapidly varying oscillation component is calculated within the sliding window: ； when Exceeding the oscillation safety threshold The vibration warning is triggered when the oscillation safety threshold is set. Determined based on the minimum permissible clearance between the rotor and the hole.

12. The hydropower station generator rotor auxiliary positioning system based on deep learning multi-view 3D reconstruction and radar ranging according to claim 11, characterized in that, When the root mean square amplitude of the fast oscillation component Exceeding the oscillation safety threshold Simultaneously, the process noise covariance matrix of the Kalman filter is adjusted: ； in, The nominal process noise covariance matrix is... This is the magnification factor.

13. The hydropower station generator rotor auxiliary positioning system based on deep learning multi-view 3D reconstruction and radar ranging according to claim 1, characterized in that, The application layer is built on the Unity engine to create a cross-platform 3D visualization GUI interface, including: The digital twin rendering window displays the 3D reconstructed model of the rotor and holes in real time, and supports viewpoint rotation, scaling and translation. The parameter dashboard displays the three-dimensional offset in real time. , , axial deflection angle and shortest edge distance ; An AR guidance layer overlays dynamic directional arrows and distance scales onto a 3D model, prompting the operator to adjust the direction and magnitude. Multi-level alarm module, set when or When a yellow alert is triggered, or A red alarm is triggered and the hoisting operation is forcibly interrupted.

14. A method for assisted positioning of a hydropower station generator rotor based on deep learning multi-view 3D reconstruction and radar ranging, as described in any one of claims 1-13, characterized in that, Includes the following steps: Rotor images are acquired using multiple industrial cameras deployed at different angles, while vertical height data of the top of the rotor is acquired using a vertically mounted millimeter-wave radar. The data acquired by industrial cameras and millimeter-wave radar are synchronized in time, and the synchronized data is transmitted to the processing unit. In the processing unit, the received video stream and radar ranging data are preprocessed respectively; Three-dimensional reconstruction is performed based on preprocessed multi-view images and radar height data to generate a rotor point cloud model. The average height of the rotor top measured by radar is added as a constraint term to the loss function of the three-dimensional reconstruction. The weight of the constraint term in the loss function and the sampling density of the three-dimensional reconstruction are adjusted according to the residual feedback of subsequent positioning and registration. Based on the rotor point cloud model, the positioning parameters of the rotor relative to the hole are calculated. The positioning parameters include three-dimensional offset, axis deflection angle and shortest edge distance. Multi-timescale decomposition of radar altitude data is performed to separate the slowly varying descent trend component from the rapidly varying oscillation component; Based on the positioning parameters, the three-dimensional relative position of the rotor and the hole is displayed on the screen, and operation guidance and vibration warning are provided according to the slow downward trend component and the fast oscillation component, respectively.

15. The method for auxiliary positioning of a hydropower station generator rotor based on deep learning multi-view 3D reconstruction and radar ranging according to claim 14, characterized in that, The weights of the constraint terms in the loss function and the sampling density of the 3D reconstruction are adjusted based on the residual feedback from the localization and registration, specifically including: After each ICP fine registration is completed, the average registration residual is calculated: ； Calculate the vertical residual components: ； Update constraint weights for: ； For residuals exceeding the threshold In the spatial region, increase the number of ray sampling points for NeRF volume rendering to the original level. Iterate through the above feedback adjustments until... Converging to a preset threshold the following.

16. The method for auxiliary positioning of a hydropower station generator rotor based on deep learning multi-view 3D reconstruction and radar ranging according to claim 14, characterized in that, The radar altitude data is decomposed into multiple time scales, specifically including: height sequence of Kalman filter output Perform complementary filtering to extract the slowly decreasing trend component: ； And the fast oscillation component: ； smoothness coefficient , The time constant is the current rate of descent. hour ,when hour , The sampling interval; The current descent rate is calculated based on the slowly varying descent trend component, and the remaining time to reach the target altitude is predicted. The root mean square amplitude within the sliding window is calculated based on the rapidly changing oscillation component. When the root mean square amplitude exceeds the safety threshold, a vibration warning is triggered, and the process noise covariance matrix of the Kalman filter is adjusted synchronously.

17. A computer device, characterized in that, It includes a memory and a processor, which are interconnected and communicate with each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the hydropower station generator rotor assisted positioning method based on deep learning for multi-view three-dimensional reconstruction and radar ranging, as described in any one of claims 14 to 15.

18. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing the computer to execute the hydropower station generator rotor-assisted positioning method based on deep learning for multi-view 3D reconstruction and radar ranging, as described in any one of claims 14 to 15.

Images