Nuclear power plant hoisting height measurement method and system based on visual recognition

Abstract

The present application relates to a nuclear power plant hoisting height measurement method and system based on visual recognition, comprising: obtaining the shaft end surface image of the drum rotating shaft of the ring hoist lifting through an image acquisition unit; calculating based on the shaft end surface image using a first calculation method or a second calculation method to obtain the rotation parameters of the drum rotating shaft; the rotation parameters include: the number of rotation circles and the rotation angle; based on the number of rotation circles and the rotation angle, combined with the diameter of the winding rope, the hook height is calculated; the hook height is sent to a terminal device or a control system for real-time display and positioning prompt. The present application can realize full non-contact measurement, eliminate personnel safety risks at the root, have strong anti-interference ability, have no cable or sensor moving with the hook, have no mechanical wear, have extremely low maintenance amount, have high reliability; the measurement is direct and has high precision, the structure is simple and easy to modify.

Description

Technical Field

[0001] This invention relates to the field of intelligent monitoring of heavy lifting equipment in nuclear power plants, and more specifically, to a method and system for measuring the lifting height of nuclear power plants based on visual recognition. Background Technology

[0002] In nuclear power plant overhauls or routine hoisting operations, accurately measuring the real-time height of the hook relative to reference surfaces such as water tanks is crucial for ensuring the safe placement of underwater equipment. Existing manual measurement methods pose high radiation risks to personnel; laser or infrared rangefinders are unreliable due to water surface reflection, steam mist, and hook sway; wire-type displacement sensors are contact-based, with cables prone to wear, knots, and breakage, requiring significant maintenance and posing a risk of sudden failure. Some solutions incorporate UWB wireless ranging technology, achieving non-contact measurement, but in nuclear power plant buildings with numerous metal reflective surfaces and severe multipath effects, complex anti-multipath algorithms and precise calibration are still required, resulting in a relatively complex system configuration. Therefore, there is an urgent need for a hoisting height measurement method that is non-contact, highly interference-resistant, highly reliable, and structurally simplified. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide a method and system for measuring the hoisting height of nuclear power plants based on visual recognition, addressing the problems existing in the prior art.

[0004] The technical solution adopted by this invention to solve its technical problem is: to construct a method for measuring the hoisting height of nuclear power plants based on visual recognition, including the following steps: Step S10: Obtain an image of the shaft end face of the roller shaft being lifted by the image acquisition unit; Step S20: Based on the shaft end face image, calculate using the first calculation method or the second calculation method to obtain the rotation parameters of the roller shaft; the rotation parameters include: number of rotations and rotation angle; Step S30: Based on the number of rotations and the rotation angle, and combined with the rope diameter, calculate to obtain the hook height; Step S40: Send the hook height to the terminal device or control system for real-time display and positioning prompt.

[0005] In the nuclear power plant hoisting height measurement method based on visual recognition described in this invention, step S20, which involves calculating using a first calculation method, includes: Gaussian filtering is applied to each frame of the axis end face image for noise reduction. The calibration parameters are used for correction, and the annular relative coding band region is expanded into a rectangular strip image through polar coordinate transformation. The relative angle decoding and rotation recognition of the rectangular strip image are performed to obtain the rotation parameters of the roller shaft.

[0006] In the vision-based nuclear power plant hoisting height measurement method of the present invention, the step of performing relative angle decoding and rotation recognition on the rectangular strip image to obtain the rotation parameters of the roller shaft includes: The YOLOv5 target detection algorithm is used to perform target detection on the rectangular strip image to obtain the target image; Multi-target tracking is performed using the Hungarian algorithm / greedy matching. The roller center is predicted based on the Hough plot detection method and combined with the least-multiplier circle fitting method. The total rotation is calculated using a polar angle difference accumulation and weighted fusion algorithm, and the number of rotations and the rotation angle are output.

[0007] In the vision-based nuclear power plant hoisting height measurement method of the present invention, the first calculation method further includes: If the current frame of the image is lost, the current rotation angle is predicted using a motion model based on constant angular velocity. If consecutive frame drops exceed the set threshold, the output will maintain a valid height and trigger a cleanup and maintenance prompt.

[0008] In the nuclear power plant hoisting height measurement method based on visual recognition described in this invention, step S20, which involves calculating using a second calculation method, includes: The shaft end face image is input into a deep learning-based shaft pose estimation network for prediction to obtain the rotation angle; The number of rotations is determined based on the rotation angle.

[0009] In the vision-based nuclear power plant hoisting height measurement method described in this invention, the hook height is calculated using the following formula: H = H0+ (C × π·De) + (θ / 360° × π·De); Where H is the hook height; H0 is the initial height; C is the number of rotations; θ is the rotation angle; and De is the rope diameter.

[0010] This invention also provides a vision-based nuclear power plant hoisting height measurement system, applied to the aforementioned vision-based nuclear power plant hoisting height measurement method, comprising: The image acquisition unit is used to acquire images of the shaft end face of the drum shaft used for hoisting the ring crane; Image processing unit, used for: The rotation parameters of the roller shaft are obtained by calculating using either the first or the second calculation method based on the shaft end face image; the rotation parameters include: the number of rotations and the rotation angle. The hook height is calculated based on the number of rotations and the rotation angle, combined with the rope diameter. The hook height is sent to the terminal device or control system for real-time display and positioning prompts; An illumination compensation unit is used to provide illumination compensation for the image acquisition unit.

[0011] The vision-based nuclear power plant hoisting height measurement system of the present invention further includes: A visual marking unit is attached to the end face of the roller shaft.

[0012] In the nuclear power plant hoisting height measurement system based on visual recognition described in this invention, the visual marking unit includes: an annular relative coding strip; The annular relative coding band is divided into N uniquely coded sectors along the circumference, and each sector corresponds to a unique binary pattern.

[0013] In the nuclear power plant hoisting height measurement system based on visual recognition described in this invention, the image acquisition unit adopts a single-camera architecture or a dual-machine redundant architecture. In the dual-machine redundancy architecture, both cameras are equipped with a deep learning-based axis pose estimation network.

[0014] The vision-based method and system for measuring the hoisting height of a nuclear power plant, as described in this invention, has the following advantages: It includes: acquiring an image of the shaft end face of the roller shaft being lifted by an image acquisition unit; calculating the rotation parameters of the roller shaft based on the shaft end face image using a first or second calculation method; the rotation parameters include the number of rotations and the rotation angle; calculating the hook height based on the number of rotations and the rotation angle, combined with the rope diameter; and sending the hook height to a terminal device or control system for real-time display and positioning prompts. This invention enables fully non-contact measurement, eliminating personnel safety risks at the source, has strong anti-interference capabilities, requires no cables or sensors moving with the hook, has no mechanical wear, requires extremely low maintenance, and has high reliability; the measurement is direct and highly accurate, and the structure is simple and easy to modify. Attached Figure Description

[0015] The present invention will be further described below with reference to the accompanying drawings and embodiments. In the accompanying drawings: Figure 1 This is a flowchart illustrating the method for measuring the hoisting height of nuclear power plants based on visual recognition provided by the present invention. Figure 2 This is a logic block diagram of the nuclear power plant hoisting height measurement system based on vision recognition provided by the present invention; Figure 3 This is a measurement schematic diagram of the nuclear power plant hoisting height measurement system based on vision recognition provided by the present invention. Detailed Implementation

[0016] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0017] To address the problems of traditional solutions, this invention provides a vision-based method for measuring the hoisting height in nuclear power plants. This method, based on the existing ring crane structure, eliminates contact-type wiring mechanisms and optical aiming methods susceptible to environmental interference. It utilizes an industrial camera to non-contactly monitor the rotation state of the rolling shaft in the hook lifting mechanism. By visually recognizing preset coded marks or structural features on the shaft, it calculates the cumulative number of rotations and angle in real time. Combined with the rope geometry parameters of the roller, it accurately calculates the current height of the hook. This method completely eliminates mechanical wear, is unaffected by water pool reflections and water mist, and the visual monitoring object is the shaft located on the roller above the cable tray, far from the high-radiation water pool area, facilitating the installation of radiation shielding for the camera.

[0018] refer to Figure 1 , Figure 1 The flowchart of the vision recognition-based method for measuring the hoisting height of nuclear power plants provided by the present invention is shown.

[0019] Specifically, such as Figure 1 As shown, the vision-based method for measuring the hoisting height of nuclear power plants includes the following steps: Step S10: Obtain an image of the shaft end face of the roller shaft being lifted by the image acquisition unit.

[0020] Optionally, in this embodiment of the invention, the image acquisition unit may be a radiation-resistant fixed-focus industrial camera with a resolution of not less than 5 megapixels and a frame rate of not less than 30fps. The lens is equipped with a quartz glass anti-radiation window and a polarizing filter to suppress metallic reflections. The camera is rigidly fixed to a roller bearing seat or bridge structure by a special bracket, with the optical axis perpendicular to the end face of the rotating shaft.

[0021] Step S20: Based on the shaft end face image, use the first calculation method or the second calculation method to calculate and obtain the rotation parameters of the roller shaft; the rotation parameters include: number of rotations and rotation angle.

[0022] In some embodiments, step S20, the calculation using the first calculation method includes: performing Gaussian filtering to denoise each frame of the axis end face image; correcting using calibration parameters and expanding the annular relative coding band region into a rectangular strip image through polar coordinate transformation; performing relative angle decoding and rotation recognition on the rectangular strip image to obtain the rotation parameters of the drum shaft. Specifically, performing relative angle decoding and rotation recognition on the rectangular strip image to obtain the rotation parameters of the drum shaft includes: using the YOLOv5 target detection algorithm to perform target detection on the rectangular strip image to obtain the target image; using the Hungarian algorithm / greedy matching for multi-target tracking; predicting the drum center based on the Hough diagram detection method and combined with the least-multiply-circle fitting method; and calculating the total rotation and outputting the number of rotations and the rotation angle through polar angle difference accumulation and weighted fusion algorithms. Further, the first calculation method also includes: if the current frame image is lost, predicting the current rotation angle using a motion model based on a constant angular velocity; if consecutive frame loss exceeds a set threshold, outputting "maintain effective height" and triggering a cleanup and maintenance prompt.

[0023] Specifically, in this embodiment, a visual marking unit needs to be set on the end face of the rotating shaft. Preferably, the visual marking unit can be a radiation-resistant, high-contrast annular relative coding strip. This annular relative coding strip is attached to the end face of the rotating shaft of the lifting drum. The annular relative coding strip is divided into N uniquely coded sectors (e.g., N=1024) along the circumference. Each sector corresponds to a unique binary pattern, which is made of a radiation-resistant metal reflective film and a black anodized coating to ensure that it remains identifiable in long-term radiation and oily environments. The codes can be unevenly distributed, as long as the corresponding unique code appears in the camera's field of view at all times. The camera's field of view completely covers the entire coding strip.

[0024] Alternatively, in some other embodiments, step S20, the calculation using the second calculation method includes: inputting the shaft end face image into a deep learning-based shaft attitude estimation network for prediction to obtain the rotation angle; and determining the number of rotations based on the rotation angle.

[0025] Based on the hardware of the aforementioned embodiments, this embodiment does not rely on the ring-shaped relative coding tape, but instead uses the structural features of the shaft itself (such as keyways, bolt holes, and machining textures) and a deep learning network to directly estimate the rotational attitude of the shaft. It is suitable for scenarios where it is inconvenient to paste coding tape and improves the robustness of the system.

[0026] Specifically, this embodiment adds a second top-down camera to the hardware of the aforementioned embodiment (without requiring the attachment of a circular encoding tape), creating binocular redundancy. Each camera is equipped with a deep learning-based axis-of-motion (AOP) attitude estimation network. This network inputs the axis-of-motion end-face image and outputs angle values ​​within the range of 0° to 360°. The two AOP networks operate independently. When the angle deviation between the two networks is less than a set threshold (e.g., 0.5°), the average is taken as the final angle; if the deviation is too large, the features from both perspectives are fused for joint inference; if one camera fails, the other camera continues measurement independently. After obtaining the rotation angle, the number of rotations is accumulated by the angle difference between consecutive frames.

[0027] Step S30: Calculate the hook height based on the number of rotations and the rotation angle, combined with the rope diameter.

[0028] Optionally, in this embodiment of the invention, the hook height is calculated using the following formula: H = H0+ (C × π·De) + (θ / 360° × π·De); Where H is the hook height; H0 is the initial height; C is the number of rotations; θ is the rotation angle; and De is the rope diameter.

[0029] Step S40: Send the hook height to the terminal equipment or control system for real-time display and positioning prompts.

[0030] Specifically, the real-time calculated hook height value can be sent to the crane operator's cab display screen (i.e., the terminal equipment) and the core refueling control system via Modbus TCP or a 4-20 mA interface to achieve height display, positioning prompts, and safety interlocks. The real-time hook height value is also simultaneously sent to the tablet via LoRa for height display and positioning prompts.

[0031] refer to Figure 2 , Figure 2 The present invention illustrates a vision-based nuclear power plant hoisting height measurement system, which is used to implement the vision-based nuclear power plant hoisting height measurement method disclosed in this invention.

[0032] Specifically, such as Figure 2 As shown, the vision recognition-based nuclear power plant hoisting height measurement system includes: The image acquisition unit is used to acquire images of the shaft end face of the roller shaft used for hoisting.

[0033] In some embodiments, the image acquisition unit employs a single-camera architecture or a dual-camera redundant architecture. In the dual-camera redundant architecture, both cameras are equipped with a deep learning-based axis pose estimation network. In this invention, the image acquisition unit can use a radiation-resistant fixed-focus industrial camera with a resolution of at least 5 megapixels and a frame rate of at least 30 fps. The lens is fitted with a quartz glass anti-radiation window and a polarizing filter to suppress metallic reflections. The camera is rigidly fixed to a roller bearing housing or bridge structure using a dedicated bracket, with the optical axis perpendicular to the end face of the rotating shaft.

[0034] The image processing unit is used to: calculate the rotation parameters of the drum shaft based on the shaft end face image using a first calculation method or a second calculation method; the rotation parameters include: the number of rotations and the rotation angle; calculate the hook height based on the number of rotations and the rotation angle, combined with the rope diameter; and send the hook height to the terminal device or control system for real-time display and positioning prompts. Specifically, the image processing unit can use an industrial embedded computer equipped with a GPU module to run image decoding algorithms and height calculation programs in real time, receive camera images via Ethernet, and output height data to the crane control system.

[0035] An illumination compensation unit is used to provide illumination compensation for the image acquisition unit. Optionally, the illumination compensation unit can employ a ring-shaped LED light source (wavelength matched to the camera's peak response), positioned in front of the lens to enhance the contrast of the coded strip pattern through low-angle lighting. The light source has a post-irradiation light attenuation compensation function.

[0036] Furthermore, in some embodiments, the vision-based nuclear power plant hoisting height measurement system further includes: a visual marking unit, which is affixed to the end face of the roller shaft. Optionally, the visual marking unit includes: an annular relative coding strip; the annular relative coding strip is divided into N uniquely coded sectors along the circumference, and each sector corresponds to a unique binary pattern.

[0037] Compared with traditional solutions, the present invention has the following advantages: Intrinsic safety and personnel protection: It achieves completely non-contact measurement, eliminating the need for personnel to approach the hook or high-radiation areas, thus eliminating personnel safety risks at the source.

[0038] Strong anti-interference capability: The visual monitoring object is the roller shaft, which is not affected by hook swing, water surface reflection, steam mist, etc. Compared with the UWB solution, there is no need to install active tags on the hook, completely avoiding multipath interference.

[0039] Maintenance-free and long lifespan: There are no cables or sensors that move with the hook, no mechanical wear, extremely low maintenance, and high reliability.

[0040] Direct and high-precision measurement: It directly measures the rotation of the shaft that produces the hook displacement, obtains height information from the source, avoids the cumulative error caused by indirect calculation, and can achieve millimeter-level accuracy through sub-pixel visual processing.

[0041] Simple structure and easy to modify: Only lightweight visual markers and fixed industrial cameras need to be installed at the roller shaft, which requires minimal modification to the original ring crane mechanical structure and is easy to install and calibrate.

[0042] The invention will now be illustrated with specific examples.

[0043] Example 1: Measurement based on the ring-shaped relative coding band (the measurement schematic diagram is shown below) Figure 3 (as shown) 1. System Composition: Visual marking unit: A radiation-resistant, high-contrast annular relative coding strip is affixed to the end face of the lifting drum shaft. The coding strip is divided into N uniquely coded sectors (e.g., N=1024) along the circumference. Each sector corresponds to a unique binary pattern, which is made of a radiation-resistant reflective metal film and a black anodized coating, ensuring legibility even in long-term radiation and oily environments. The codes can be unevenly distributed, as long as the corresponding unique code is always present in the camera's field of view.

[0044] Image acquisition unit: A radiation-resistant fixed-focus industrial camera with a resolution of no less than 5 megapixels and a frame rate of no less than 30 fps is selected. The lens is equipped with a quartz glass anti-radiation window and a polarizing filter to suppress metallic reflections. The camera is rigidly fixed to the roller bearing seat or bridge structure by a special bracket, with the optical axis perpendicular to the end face of the rotating shaft, and the field of view fully covering the entire encoding tape.

[0045] Illumination compensation unit: Employs a ring-shaped LED light source (wavelength matched to the camera's peak response), positioned in front of the lens, to enhance the contrast of the coded strip pattern using low-angle lighting. The light source also features post-irradiation light attenuation compensation.

[0046] Image processing unit: An industrial embedded computer equipped with a GPU module, which runs image decoding algorithms and height calculation programs in real time, receives camera images via Ethernet, and outputs height data to the crane control system.

[0047] 2. Specific implementation steps: Step 1: System Installation and Geometric Calibration With the drum stationary, tightly adhere the annular coded tape to the end face of the shaft, ensuring its center coincides with the shaft's rotation center. Set a zero-position mark in the circumferential direction. Install the camera and light source, adjusting the focus and aperture to ensure clear imaging of the coded tape. Use a calibration board to calibrate the camera's intrinsic parameters and correct distortion. Measure and record the drum's winding parameters: drum diameter D (or groove bottom diameter if present), wire rope diameter d, current layer number n, and calculate the effective winding diameter De = D + (2n-1)d. For a single-layer winding drum, De is a constant.

[0048] Step 2: Image Acquisition and Preprocessing The camera continuously acquires grayscale images of the rotating shaft end face. The image processing unit performs Gaussian filtering to denoise each frame and corrects lens distortion using calibration parameters. The annular relative coding band region is expanded into a rectangular strip image through polar coordinate transformation for easier decoding.

[0049] Step 3: Relative Angle Decoding and Rotation Recognition: The entire system combines YOLOv5 object detection, Hungarian algorithm / greedy matching for multi-object tracking, Hough circle detection + least squares circle fitting to robustly estimate the roller center, and accurately calculates the total rotation through polar angle difference accumulation + weighted fusion, finally giving the number of rotations C (integer) and the corresponding angle θ.

[0050] Specifically, the core problem the system aims to solve is: a rotating roller with multiple digital markers (or symbols) affixed to its surface, and a camera positioned to capture images of the roller. The goal is to calculate in real-time how many revolutions the roller has made and how much line has been unwound from the reel. To achieve this, the system first needs to identify each marker, then continuously track their movement, and finally calculate the total rotation based on the changes in their angles of rotation around the roller's center. The detailed steps are as follows: The first step is to perform object detection using the YOLOv5 object detection algorithm. The model is specifically trained for this scene and can simultaneously find all digit markers in a single frame, providing their bounding box positions, digit (symbol) categories, and confidence scores. Only detection results with confidence scores exceeding a certain threshold are retained to prevent false detections from interfering with subsequent calculations.

[0051] The second step involves using a multi-target tracker to connect these isolated detections into a continuous trajectory. The system maintains a set of tracked targets, each with its own historical center point list, numerical category record, confidence level, and other information. When a new frame's detection results arrive, the tracker needs to determine which detection corresponds to which old target. It constructs a cost matrix where each element represents the Euclidean distance between the last occurrence position of each old target and the position of each new detection. Then, if a scientific computing library is installed in the environment, the Hungarian algorithm is invoked to find a globally optimal matching scheme that minimizes the sum of the distances of all pairs; otherwise, it degenerates into a greedy pairing in ascending order of distance. Only pairings with distances less than a certain threshold are considered valid; detections that are too far apart are treated as newly appearing targets, and new trajectories are created; while old trajectories that cannot be matched for multiple consecutive frames are deleted. In this way, even if individual markers are briefly occluded, as long as their reappearance position is close to their previous position, their identity can be restored, ensuring the continuity of the trajectory.

[0052] The third step, to calculate the rotation angle of the marker, requires knowing the center point of the roller in the image. The system uses a coarse-to-fine estimation method. In the first few dozen frames, it attempts to find the circular outline of the roller directly in the grayscale image using Hough circle detection. If a circle with a radius matching the image size is found, its center is used as the initial roller center. If no suitable circle is found within this time, the system backtracks and temporarily assumes the roller center is in the exact center of the image. This ensures that a usable center is always available.

[0053] To further improve accuracy, the system dynamically corrects the center online. Every fixed number of frames, it collects the position points of all active tracks from the most recent frames and then uses the least squares method to fit an optimal circle. The underlying principle is that if these points are indeed rotating around the same center, their coordinates should satisfy the equation of a circle. By solving an overdetermined system of linear equations, the coordinates of the center and radius can be deduced. Replacing the old center with the newly fitted center automatically adapts to minor camera shakes or slow drifts in the roller mounting position. After updating the center, the reference angles of all tracks are also recalculated to ensure that angle accumulation does not result in abrupt changes. Users can also freeze the center through the interface, locking the current estimate to prevent further changes for stable measurements.

[0054] The fourth step, accurately calculating the total rotation, is the essence of the system. For each trajectory, each frame calculates an polar angle based on the line connecting the roller center and the current frame center. Subtracting this angle from the polar angle of the previous frame yields the angle the marker has rotated in that frame. The key here is handling the angle's loopback—the angle range is -π to π. If the marker crosses this break, direct subtraction will result in an error value close to a full rotation. Therefore, the system standardizes the difference to between -π and π to ensure it reflects the shortest turn. This raw angle difference undergoes a first-order exponential smoothing filter to remove high-frequency noise caused by frame jitter. Simultaneously, the system maintains a "small angle accumulation pool" and a "full rotation counter." Each time the raw angle difference is obtained, it is added to the accumulation pool; once the absolute value of the accumulation pool exceeds one rotation (2π radians), this portion of rotations is moved to the full rotation counter, and the corresponding full rotation radian is subtracted from the accumulation pool. Thus, each trajectory at any given time has an accurate "total rotation angle = number of full rotations × 2π + current small angle". After processing all matches in each frame, the system will extract the total number of new rotations from each trajectory and immediately clear its cumulative pool and number of laps to avoid duplicate calculations.

[0055] Step 5: Weighted Fusion. Not all trajectories are equally reliable. For example, some have low detection confidence, some have small areas and are prone to drift, and some are newly created and not yet stable. The system calculates a weight for each trajectory, which comprehensively considers three aspects: the smoothed detection confidence, the maturity of the trajectory (the more updates, the more stable), and the area of ​​the landmark in the image (the larger the area, the more accurate the localization). Then, the system performs a weighted average of the newly added rotation amounts of all trajectories according to their weights to obtain the most reliable global rotation increment for this frame, and adds it to the total rotation angle of the system.

[0056] Finally, the distance conversion is quite simple: multiply the total rotation angle (in radians) by the radius of the roller (given by the configuration file) to get the linear distance moved by a point on the edge of the roller, which is the length of the released rope or the length of the winding. The entire process displays the number of rotations and distance in real time on the graphical interface and supports one-click reset.

[0057] Through this interconnected design, YOLOv5 provides the target position, the tracker ensures time continuity, the dynamic estimation of the roller center ensures the accuracy of the angle calculation benchmark, the polar angle difference accumulation and full circle counting eliminate the loss of circles, and the weighted fusion maximizes the use of redundant information from multiple markers. Ultimately, the entire system can achieve stable and accurate non-contact rotation and distance measurement under ordinary cameras.

[0058] Step 4. Calculate the hook height: The hook height (i.e., the current height H) is obtained by adding the length of the wire rope (i.e., the winding length) to the initial height H0. The length of each turn of wire rope is π·De, and the cumulative height change is: H = H0+ (C × π·De) + (θ / 360° × π·De); Where H is the hook height; H0 is the initial height (which can be calibrated by the hook touching the bottom or manually entered); C is the number of rotations; θ is the rotation angle; and De is the rope diameter.

[0059] In the case of multi-layer winding, if the number of layers changes with the winding, the current number of layers and the effective diameter De can be automatically calculated based on the cumulative number of turns and the total number of turns on the drum, so as to achieve accurate compensation.

[0060] Step 5. Anti-occlusion and fault tolerance handling: When a frame fails to decode due to oil splatter or sudden strong reflection, the system automatically uses a motion model based on constant angular velocity to predict the current angle and fill in the temporary data loss. If consecutive frame drops exceed a set threshold, the output maintains the most recent valid height and triggers a cleaning and maintenance prompt. In addition, redundant encoding verification in the encoding band can correct bit errors caused by some sector contamination.

[0061] Step 6: Data Output: The real-time hook height value H is transmitted to the crane operator's cab display and the core refueling control system via Modbus TCP or a 4-20 mA interface, enabling height display, positioning prompts, and safety interlocks. The real-time hook height value H is also simultaneously transmitted to the flatbed via LoRa for height display and positioning prompts.

[0062] Through this embodiment, the height measurement accuracy can be controlled within ±1 mm, meeting the stringent requirements of precision hoisting such as nuclear power plant core refueling.

[0063] Example 2: Redundant visual measurements using a deep learning-based axis pose estimation network: Based on the vision hardware of Example 1, this embodiment does not rely on the encoding tape, but directly estimates the rotation posture of the shaft by utilizing the structural features of the shaft itself (such as keyways, bolt holes, and machining textures) and a deep learning network. It is suitable for scenarios where it is inconvenient to attach the encoding tape and improves the robustness of the system.

[0064] 1. System Composition: Based on the camera and computing unit in Example 1, a second top-view camera is added to form a binocular redundancy. Simultaneously, a deep learning-based axis-of-motion pose estimation network (such as an improved ResNet with a fully connected regression head) and an axis-of-motion feature enhancement module are deployed.

[0065] 2. Specific implementation steps: Step 1: Feature Learning and Model Training In the feature-rich region of the roller shaft end face, multi-turn, multi-angle images of the shaft are manually labeled, and the corresponding high-precision ground truth angle values ​​are recorded (provided by a temporary encoder installation). An angle regression dataset is constructed. A lightweight convolutional neural network is designed, taking the corrected shaft end face image patch as input and outputting angle values ​​within the range of 0° to 360°. The loss function uses periodically perceived mean squared error loss (mapping the angle to a unit circle to calculate the deviation). During training, data augmentation is applied to simulate changes in lighting, partial occlusion, and slight dirt to improve the model's generalization ability.

[0066] Step 2: Redundant camera installation and fusion: Two industrial cameras are arranged at a 90° angle, simultaneously pointing towards the end face of the rotating shaft, serving as backups for each other. The image processing unit runs two independent attitude estimation models. When the angle deviation between the two models is less than a set threshold (e.g., 0.5°), the average is taken as the final angle; if the deviation is too large, the features of the two viewpoints are fused for joint inference; if one camera fails, the other camera maintains the measurement independently.

[0067] The core of joint inference is to stop simply trusting the output of any single viewpoint. Instead, it simultaneously feeds the pivot end face images captured by two cameras into a specially designed multi-view fusion network (joint inference network), allowing this network to directly output a unified and reliable angle value. During the training phase, this network learns how to extract features from two complementary viewpoints and automatically determines which viewpoint is more reliable and how to compensate for the deficiencies of the other viewpoint, thus providing a more robust prediction than any single viewpoint.

[0068] The joint inference network is a two-input, single-output convolutional neural network, which can be seen as an extension of the original single-view CNN. The specific structural design is as follows: Shared feature extractor: Both input images are processed by the same feature extractor (i.e., the initial part of the lightweight CNN trained in step 1, up to the convolutional layers before the global pooling layer). The advantage of shared weights is that images from both perspectives can be mapped to the same feature space, facilitating subsequent comparison and fusion.

[0069] Feature fusion layer: This layer fuses the feature maps or feature vectors extracted from two perspectives. Common fusion methods include: concatenation: directly concatenating two feature vectors into a single long vector, followed by a fully connected layer; element-wise addition / subtraction or taking the maximum value: calculating the sum, difference, or maximum value of two feature vectors. This is effective when the features are already aligned, but since the perspectives differ by 90° and the feature maps are not aligned in space, concatenation is more suitable.

[0070] Attention fusion: This allows the network to automatically learn the weights for each feature channel from each viewpoint. For example, attention weights can be predicted using a small subnetwork, and then the two feature vectors can be weighted and merged. This enables the network to learn during training to "reduce the contribution of features from a viewpoint when it is occluded or blurred."

[0071] Angle Regression Head: The fused feature vectors are fed into a fully connected layer, which ultimately outputs an angle value. The output layer structure is the same as the original single-view model, and it still uses a periodic-aware loss function (that is, supervising by mapping the angle to (sin,cos) on the unit circle) to ensure the continuity of 360°.

[0072] Training data and supervision signals: The joint inference network is not designed out of thin air; it requires offline training with a large number of pairs of dual-view images. This data is captured by two cameras at the same instant, and each pair of images corresponds to a high-precision ground truth angle recorded by an encoder. During training, the loss function directly calculates the deviation between the angle output by the fusion network and the ground truth (also expressed as the distance on the unit circle), and updates the parameters of both the feature extractor and the fusion module simultaneously through backpropagation.

[0073] In addition, some auxiliary tasks can be added to improve robustness. For example, the network can be required to output the predicted angles of two single views at the same time and keep them consistent with the CNN prediction results of each image. This is equivalent to a multi-task learning framework to prevent the feature extractor from forgetting the knowledge of single views when fine-tuning.

[0074] The execution flow of joint inference is as follows: During system operation, under normal circumstances, each camera independently runs a single-view CNN, outputting angle values ​​in real time and comparing deviations. Once a deviation exceeds 0.5°, joint inference is immediately triggered. Synchronous capture: Extract the aligned image frame at the current moment from the video streams of the two cameras, and crop the image block of the rotating end face (after the same correction and preprocessing).

[0075] One forward propagation: The two image patches are fed into the joint inference network at the same time, and the network outputs a fused angle value.

[0076] Result adoption: Use this fused angle value as the final measurement result at the current moment, overriding the contradictory outputs of the two single-viewpoints.

[0077] Resume monitoring: Continue running the two single-view CNNs. When their deviation returns to within the threshold, switch back to the lightweight mode of averaging to reduce computational overhead.

[0078] When the predictions from two viewpoints differ significantly, it's usually because one viewpoint has partial occlusion, strong reflections, dirt, or the texture features of that viewpoint are insignificant when the axis of rotation reaches a certain attitude angle. Single-view models are prone to large errors in these situations, while the same end-face area may be clearly visible from another viewpoint. The joint inference network, by fusing features from both perspectives, essentially sees images from two directions simultaneously. It can make judgments based on its learned knowledge. For example, if the feature map of viewpoint A has almost no texture response, while the feature map of viewpoint B is very clear, the attention mechanism will automatically allow the features of viewpoint B to dominate the final prediction. If the features of the two viewpoints have a complementary relationship (e.g., viewpoint A sees landmark A, and viewpoint B sees landmark B, but both belong to the same rigid body), the network can integrate them into a consistent 3D attitude cue, thus deducing the correct angle. In this way, joint inference is no longer a simple post-processing of two independent estimates, but rather a completely new and more comprehensive attitude calculation starting from the original sensor data (two images). Its accuracy and robustness naturally far exceed those of a single viewpoint.

[0079] Step 3: Cumulative lap tracking and altitude calculation: Similar to Example 1, the hook height is calculated by accumulating the number of rotations based on the angle difference between consecutive frames and combining it with the rope diameter. Specifically, similar to Example 1, after the rotation angle is output by the axis attitude estimation network, the number of rotations is accumulated by accumulating the angle difference between consecutive frames. That is, the difference between this angle and the polar angle of the previous frame gives the angle the marker has rotated in this frame. The key here is handling the angle value wrapping—the angle range is from -π to π. If the marker crosses this break zone, a direct subtraction will result in an error value close to a full rotation. Therefore, the system standardizes the difference to between -π and π to ensure that it reflects the shortest turning point. This raw angle difference is smoothed by a first-order exponential smoothing filter to remove high-frequency noise caused by detection frame jitter. At the same time, the system maintains a "small angle accumulation pool" and a "full rotation counter". Each time the raw angle difference is obtained, it is accumulated in the accumulation pool; once the absolute value of the accumulation pool exceeds one rotation (2π radians), the number of rotations in this part is moved into the full rotation counter, and the corresponding full rotation radian is subtracted from the accumulation pool. In this way, each trajectory has an accurate "total rotation angle = total number of complete revolutions × 2π + current small angle" at any given time. After processing all matches in each frame, the system extracts the total amount of rotation added this time from each trajectory and immediately clears its cumulative pool and number of revolutions to zero to avoid duplicate calculations.

[0080] After obtaining the rotation angle and the number of rotations, the hook height is calculated in conjunction with the rope winding smuggling calculation. For the specific calculation formula, please refer to Example 1.

[0081] Since deep learning models directly output absolute angles without the need for encoding and decoding, only the image of the rotating shaft needs to be input. To prevent errors in the number of revolutions due to extremely similar appearances, motor speed or inverter frequency signals are introduced as auxiliary revolution count verification. When the difference between the visually estimated number of revolutions and the number of revolutions estimated by the electrical signal exceeds one revolution, the electrical signal is used for correction.

[0082] Step 4: Online Learning and Adaptation During system operation, the known reference height when the hook touches the bottom or reaches a fixed calibration height is used to calculate the absolute angle of the shaft at that time. This is then used as a self-supervised sample to fine-tune the attitude estimation network, so as to adapt to the slow changes in appearance caused by radiation and oil contamination, and achieve long-term maintenance-free drift suppression.

[0083] This embodiment utilizes the inherent characteristics of the shaft directly through deep learning, eliminating the need for a dedicated coding tape and simplifying installation. At the same time, with the help of redundant binoculars and electrical auxiliary verification, the system robustness is significantly improved, and the height measurement accuracy is within ±0.5 mm, making it suitable for high reliability requirements in extreme environments.

[0084] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. For the apparatus disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple; relevant parts can be referred to the method section.

[0085] Those skilled in the art will further recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of both. To clearly illustrate the interchangeability of hardware and software, the components and steps of the various examples have been generally described in terms of functionality in the foregoing description. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0086] The steps of the methods or algorithms described in conjunction with the embodiments disclosed herein can be implemented directly by hardware, a software module executed by a processor, or a combination of both. The software module can be located in random access memory (RAM), main memory, read-only memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art.

[0087] The above embodiments are only for illustrating the technical concept and features of the present invention, and are intended to enable those skilled in the art to understand the content of the present invention and implement it accordingly. They do not limit the scope of protection of the present invention. All equivalent changes and modifications made within the scope of the claims of the present invention should fall within the scope of the claims of the present invention.

Claims

1. A method for measuring the hoisting height of nuclear power plants based on visual recognition, characterized in that, Includes the following steps: step S10: Acquire an image of the shaft end face of the roller shaft being lifted by the image acquisition unit; Step S20: Based on the shaft end face image, calculate using the first calculation method or the second calculation method to obtain the rotation parameters of the roller shaft; The rotation parameters include: number of rotations and rotation angle; in step S20, the calculation using the first calculation method includes: Gaussian filtering is applied to each frame of the axis end face image for noise reduction. The calibration parameters are used for correction, and the annular relative coding band area is expanded into a rectangular strip image through polar coordinate transformation; the annular relative coding band is pasted on the end face of the shaft of the roller shaft of the ring hoist; The relative angle decoding and rotation recognition of the rectangular strip image are performed to obtain the rotation parameters of the roller shaft; the process of performing relative angle decoding and rotation recognition of the rectangular strip image to obtain the rotation parameters of the roller shaft includes: The YOLOv5 target detection algorithm is used to perform target detection on the rectangular strip image to obtain the target image; Multi-target tracking can be performed using the Hungarian algorithm or greedy matching. The roller center is predicted based on the Hough plot detection method and combined with the least squares circle fitting method. The total rotation is calculated using a polar angle difference accumulation and weighted fusion algorithm, and the number of rotations and the rotation angle are output. The calculation using the second calculation method includes: The shaft end face image is input into a deep learning-based shaft pose estimation network for prediction to obtain the rotation angle; The number of rotations is determined based on the rotation angle; Step S30: Based on the number of rotations and the rotation angle, and combined with the rope diameter, calculate to obtain the hook height; Step S40: Send the hook height to the terminal device or control system for real-time display and positioning prompt.

2. The method for measuring the hoisting height of a nuclear power plant based on visual recognition according to claim 1, characterized in that, The first calculation method further includes: If the current frame of the image is lost, the current rotation angle is predicted using a motion model based on constant angular velocity. If consecutive frame drops exceed the set threshold, the output will maintain a valid height and trigger a cleaning and maintenance prompt.

3. The method for measuring the hoisting height of a nuclear power plant based on visual recognition according to claim 1, characterized in that, The height of the hook is calculated using the following formula: H = H0 + (C × π×De) + (θ / 360° × π×De); Where H is the hook height; H0 is the initial height; C is the number of rotations; θ is the rotation angle; and De is the rope diameter.

4. A vision-based nuclear power plant hoisting height measurement system, applied to the vision-based nuclear power plant hoisting height measurement method according to any one of claims 1-3, characterized in that, include: The image acquisition unit is used to acquire images of the shaft end face of the drum shaft used for hoisting the ring crane; Image processing unit, used for: The rotation parameters of the roller shaft are obtained by calculating using either the first or the second calculation method based on the shaft end face image; the rotation parameters include: the number of rotations and the rotation angle. The hook height is calculated based on the number of rotations and the rotation angle, combined with the rope diameter. The hook height is sent to the terminal device or control system for real-time display and positioning prompts; An illumination compensation unit is used to provide illumination compensation for the image acquisition unit.

5. The nuclear power plant hoisting height measurement system based on visual recognition according to claim 4, characterized in that, Also includes: A visual marking unit is attached to the end face of the roller shaft.

6. The nuclear power plant hoisting height measurement system based on vision recognition according to claim 5, characterized in that, The visual marking unit includes: an annular relative coding band; The annular relative coding band is divided into N uniquely coded sectors along the circumference, and each sector corresponds to a unique binary pattern.

7. The nuclear power plant hoisting height measurement system based on vision recognition according to claim 4, characterized in that, The image acquisition unit adopts a single-camera architecture or a dual-machine redundant architecture. In the dual-machine redundancy architecture, both cameras are equipped with a deep learning-based axis pose estimation network.

Images