Nuclear power embedded part intelligent detection and acceptance method based on binocular camera

Abstract

The invention provides an intelligent nuclear power embedded part detection and acceptance method based on a binocular camera, belongs to the technical field of detection and acceptance, is an intelligent detection algorithm based on binocular vision and a deformable attention mechanism, and aims to break through the technical bottleneck of three-dimensional positioning and multi-scale feature fusion of embedded parts in a complex scene. A three-dimensional pyramid network with reserved features is designed, multi-scale features are fused in a layered mode, binocular geometric consistency is kept, and the problem of information loss in cross-resolution feature fusion is effectively solved; parallax perception position coding is innovatively introduced, binocular parallax information and two-dimensional position coding are deeply fused, and the three-dimensional space perception capability is enhanced; a non-parametric anchor-based target query mechanism is provided, query embedding covered by dense space is generated by multiplexing low-resolution three-dimensional features, the model convergence efficiency and the detection precision are improved, the detection cost is reduced, the detection efficiency is improved, and the detection safety is guaranteed.

Description

Technical Field

[0001] This invention belongs to the field of testing and acceptance technology, and also to the field of nuclear power engineering technology. Specifically, it relates to an intelligent testing and acceptance method for nuclear power plant embedded parts based on a binocular camera. Background Technology

[0002] With the continued development of the global economy, the world's demand for energy is increasing daily. Nuclear power, as a clean energy source, has received increasing attention. In the construction process of nuclear power plants, quality inspection is a crucial step in ensuring their safe operation. This process requires the use of high-precision testing equipment and rigorous testing procedures to ensure that all components and systems meet design specifications and international safety standards.

[0003] In nuclear power plant construction, embedded parts refer to metal components pre-installed in the concrete structure. They are primarily used to secure subsequent pipelines, cable trays, and large machinery, forming the foundation for ensuring the normal operation of all systems within the nuclear power plant. Nuclear power projects have extremely high requirements for safety and stability, making the installation accuracy of embedded parts crucial. Even slight deviations in the installation accuracy of embedded parts can lead to difficulties in installing subsequent equipment, preventing precise alignment, affecting the normal operation and lifespan of the equipment, and consequently impacting the safe operation and performance of the nuclear power plant.

[0004] In nuclear power plant construction, as mentioned in the prior art patent publication number "CN119177638A", the installation scale of embedded parts for nuclear power plants is enormous, widely distributed, numerous in type and quantity, and varies greatly in size and weight. Installation locations are spread across various buildings and areas of the nuclear island and conventional island. Taking the "Hualong One" reactor as an example, a single unit involves over 180 different types of embedded plates, totaling more than 100,000 pieces. Traditional acceptance testing of embedded parts relies on manual point-by-point measurement, which is easily affected by human vision or operation, leading to significant errors. Furthermore, manual testing is inefficient. On-site inspection involves five rounds of testing to assess the quality of embedded parts installation: team self-inspection, three-level QC inspection (QC1, QC2, QC3), and supervisor spot checks. Approximately 40% of the embedded parts are located at heights of 2 meters or higher, or in enclosed spaces (such as inside a dome), requiring inspection personnel to climb scaffolding, posing safety hazards such as falls and collisions.

[0005] Traditional instrument inspection of nuclear power plant embedded parts mainly relies on manual inspection using measuring instruments and tools such as total stations, levels, tape measures, and plumb bobs. The technical solution is as follows:

[0006] ① Coordinate conversion: Before the inspection, the inspection personnel need to convert the position coordinates on the design drawings into standard values ​​that can be referenced by manual measurement on site, and generate tabular data for on-site acceptance comparison and recording;

[0007] ② Pulling lines: At the inspection site, a laser level is used to pull lines according to the markings on the reinforcing bars by the surveying team, which serve as control lines for measuring the elevation of embedded parts;

[0008] ③ Measure the elevation of the embedded parts: Use a measuring tape to pull from the upper edge of the embedded part to the control line, record the data as the elevation measurement value;

[0009] ④ Setting up location standards: On the ground along the corner of the wall, use a measuring tape to pull it from the base of the wall to the end of the entire measurement area to read the positioning difference of the center point of the embedded part;

[0010] ⑤ To measure the horizontal position: Use a plumb bob to pass through the center point of the embedded part, with the plumb bob pointing to the scale on the bottom measuring tape, and take the reading;

[0011] ⑥ Embedded part adjustment: For embedded parts whose measured value exceeds the standard value by 5mm (including 5mm), notify the team members to adjust the position of the embedded part;

[0012] ⑦ Retest: After the team has adjusted the embedded parts, a retest is conducted. The operation steps are the same as above, until the adjustment is qualified.

[0013] ⑧ Next-level QC acceptance: After the first-level QC acceptance is passed, the next-level QC is notified to conduct acceptance. The acceptance process and operation are exactly the same as above.

[0014] The main disadvantages of traditional manual inspection of nuclear power plant embedded parts are as follows:

[0015] (1) Insufficient detection accuracy

[0016] Human error has a significant impact: Manual inspection relies heavily on the experience and operational skills of the inspectors. Differences in the professional level, work attitude, and sense of responsibility among different inspectors can lead to deviations in the inspection results. For example, when measuring the positional accuracy of embedded parts, different inspectors may obtain different estimated values ​​from the measuring instrument data, thus affecting the accuracy of the measurement.

[0017] Difficulty in detecting complex situations: Embedded components in nuclear power plant construction may be located in complex environments, such as being partially covered by concrete or surrounded by obstacles. Manual inspection is difficult to fully and accurately determine the actual condition of embedded components in these complex situations, and some internal defects or problems in hidden areas may not be detected in time.

[0018] (2) Low detection efficiency

[0019] Slow inspection speed: Manual inspection requires inspectors to check and measure each embedded part one by one. For a large number of embedded parts, the inspection process is very time-consuming. For example, in a large nuclear power project, there may be tens of thousands of embedded parts that need to be inspected, and manual inspection may take weeks or even months to complete.

[0020] Data processing is cumbersome: Data obtained through manual testing requires manual recording, organization, and comparative analysis. This process is not only inefficient but also prone to errors or loss. Furthermore, statistical analysis of large amounts of data requires significant time and effort.

[0021] (3) Poor security

[0022] Hazardous on-site environment: Nuclear power plant construction sites present complex environments with various hazards, such as working at heights, overlapping operations, and material transportation. Inspection personnel must frequently enter the construction site for manual inspections, facing significant safety risks. For example, inspecting embedded parts at heights requires personnel to climb scaffolding or use aerial work platforms, increasing safety risks.

[0023] (4) High testing costs

[0024] High labor costs: Manual inspection requires a large number of personnel. In addition to the inspectors themselves, two construction team members are needed to operate the equipment. Furthermore, the inspection work requires specialized skills and experience, resulting in high labor costs. In addition, to ensure the quality and progress of the inspection work, training and management of the inspectors are also necessary, which further increases labor costs.

[0025] High time costs: Due to the low efficiency of manual inspection, the inspection cycle is extended, which will affect the progress of the entire nuclear power construction project and thus lead to a series of increased project management costs. Summary of the Invention

[0026] To address the shortcomings of existing technologies, this invention presents an intelligent inspection and acceptance method for nuclear power plant embedded parts based on binocular cameras. Its intelligent detection algorithm, based on binocular vision and deformable attention mechanisms, aims to overcome the technical bottlenecks of 3D localization and multi-scale feature fusion of embedded parts in complex scenarios. A pyramid network with stereo feature preservation is designed to fuse multi-scale features hierarchically while maintaining binocular geometric consistency, effectively solving the information loss problem in cross-resolution feature fusion. By innovatively introducing disparity-aware position coding, binocular disparity information is deeply fused with 2D position coding, enhancing 3D spatial perception capabilities. A non-parametric anchor-based target query mechanism is proposed, reusing low-resolution stereo features to generate query embeddings with dense spatial coverage, improving model convergence efficiency and detection accuracy, reducing detection costs, increasing detection efficiency, and ensuring detection security.

[0027] The present invention employs the following technical solution.

[0028] A smart inspection and acceptance method for nuclear power plant embedded parts based on binocular cameras includes:

[0029] Step 1: Perform multi-scale extraction and fusion of binocular images based on a pyramid network with stereo-preserving features;

[0030] Step 2: Design a DETR decoder based on parallax-aware location encoding and nonparametric anchor-based target query.

[0031] Furthermore, in step 1, two weight-shared ResNet+FPN are used to extract multi-scale monocular features from the left and right views of the stereo images captured by the stereo camera. ResNet obtains basic features, and FPN obtains enhanced basic features and fuses information from different levels to form multi-scale monocular features. Based on these monocular features, a multi-scale correlation cost pyramid is constructed by applying a stereo-preserving feature pyramid, and feature fusion is performed to obtain multi-scale stereo features.

[0032] Furthermore, in step 1, during the feature fusion process, stereo features of the same resolution are directly added together, while features of different resolutions are fused across scales through channel expansion and stitching to preserve the corresponding information in the image.

[0033] Furthermore, in step 1, the stereo-preserving feature pyramid network includes a monocular feature pyramid, the upper half of which is the initial pyramid and the lower half of which is the enhancement pyramid; six pairs of monocular features with three resolution levels are used to construct a six-level correlation-based cost volume pyramid by applying the stereo-preserving feature pyramid.

[0034] Furthermore, in step 1, the upper three layers of the six-level correlation-based cost volume pyramid are obtained from the initial monocular features of the six-level stereo-preserving feature-based pyramid network, and the lower three layers are obtained from the enhanced monocular features of the stereo-preserving feature-based pyramid network; the upper three layers form the initial stereo features, and the lower three layers form the enhanced stereo features, and the initial stereo features and enhanced stereo features are fused according to the resolution.

[0035] Furthermore, in step 1, in the initial monocular features, the left monocular feature and right monocular features The initial 3D cost body is constructed using formula (1). :

[0036] (1)

[0037] in, It is a cost body based on correlation. , , The maximum disparity value. The scale index representing the cost volume pyramid. Represents the left view. Represents the right view. Represents cost space; Left-enhanced monocular features The same method was used to construct the augmented 3D cost volume for right-side enhanced monocular features. .

[0038] Further, in step 1, the first Initial three-dimensional features of scale As shown in formula (2):

[0039] (2)

[0040] in Cross-scale fusion is represented by recursive formula (3):

[0041] (3)

[0042] This indicates that the kernel size is A convolutional layer with a stride of 2. For the first Scale-based convolutional layers, It is the convolution kernel.

[0043] Furthermore, in step 2, the information corresponding to the binocular images is encoded into the stereo features.

[0044] Further, in step 2, the method for encoding the corresponding information of the binocular images into stereo features includes: disparity-aware position encoding explicitly encodes the corresponding information of the binocular images based on disparity log odds. The generated position code contains three-dimensional position information in the scene. This three-dimensional position information is input into the decoder as a supplement to the non-parametric anchor-based target query, thereby introducing the three-dimensional position information into the Transformer decoder. The multi-scale stereo features are flattened and concatenated to form a two-dimensional feature matrix, which is used as a multi-scale key / value pair in MSDeformCA. The downsampling rates of the feature maps at three resolutions relative to the input images are 4, 8, and 16, respectively. Let be the dimension of the feature vector in the decoder. For the first Feature maps at each level.

[0045] Furthermore, in step 2, for other hyperparameters in MSDeformCA, use There are 1 decoder layer, each with 1 decoder layer. Each head, and each MSDeformCA sample One offset point;

[0046] During training, targets are assigned based on the 2D intersection-union ratio (IU) between anchor boxes and ground truth boxes. If an IU exceeds a preset foreground threshold... The corresponding anchor point is then assigned as a matching positive sample; conversely, if the intersection-union ratio is less than a preset background threshold, the corresponding anchor point is assigned as a positive sample. The anchor point is then assigned as a negative sample.

[0047] The beneficial effects of the present invention are as follows, compared with the prior art:

[0048] 1. High-precision feature extraction: Using a pyramid network that preserves three-dimensional features, it can effectively extract the size and location of embedded parts in complex environments. The target detection recall rate is 98%, the positioning detection accuracy reaches ±3mm, and the physical size accuracy can reach ±1mm, reducing the false detection rate and ensuring the safety and reliability of key areas in nuclear power construction.

[0049] 2. Enhanced 3D spatial perception capability: By introducing parallax perception position coding, the deep fusion of binocular parallax information and 2D position coding is achieved, which improves the accurate reflection of the position information of objects in the 3D scene and thus improves the detection accuracy.

[0050] 3. Effective control of information loss: In the process of cross-scale and multi-resolution feature fusion, the problem of information loss in traditional methods is innovatively solved, ensuring the effective fusion of stereo features and geometric information and improving the overall detection performance.

[0051] 4. Balance between real-time performance and accuracy: The lightweight two-stage detection framework is designed to ensure that while meeting real-time requirements, it maintains high detection accuracy and adapts to the actual needs of nuclear power plant construction sites.

[0052] 5. Wide applicability: This invention eliminates the dependence on LiDAR, supports low-cost binocular camera training and deployment, reduces the requirements for training data and detection equipment, and has strong market adaptability. Attached Figure Description

[0053] Figure 1 This is a flowchart of the intelligent inspection and acceptance method for nuclear power plant embedded parts based on a binocular camera in this invention. Detailed Implementation

[0054] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of this invention. The embodiments described in this application are merely some embodiments of this invention, and not all embodiments. Based on the spirit of this invention, other embodiments obtained by those skilled in the art without creative effort are all within the protection scope of this invention.

[0055] like Figure 1 As shown, a smart inspection and acceptance method for nuclear power plant embedded parts based on binocular cameras includes:

[0056] This invention addresses the following key problems:

[0057] (1) Accurate extraction of embedded parts features in complex environments: The installation of embedded parts in nuclear power plant construction is highly complex, especially in key areas such as the nuclear island and containment. The distribution of embedded parts involves not only high altitudes and curved surfaces, but also various specifications and shapes. This places high demands on feature extraction algorithms, which are required to effectively extract the size and location of embedded parts in complex environments and reduce the false detection rate of the algorithm.

[0058] (2) Effective combination of binocular image parallax perception and position coding: Existing 2D detection position coding often only contains planar position information of the image and lacks perception of three-dimensional spatial depth; while 3D point cloud detection methods usually ignore the geometric correspondence between binocular images. This invention targets 2D images acquired by binocular cameras and, through effective parallax perception position coding, enables them to accurately reflect the position information of objects in 3D scenes, while taking into account both detection accuracy and computational efficiency.

[0059] (3) Reduce information loss in cross-scale and multi-resolution feature fusion: Traditional methods such as Feature Pyramid Network (FPN, BiFPN) can effectively handle multi-scale problems, but when processing binocular images, information loss often occurs due to resolution differences and the destruction of geometric information. This invention can effectively fuse features of different resolutions in the Feature Pyramid Network, especially avoiding information loss in the process of fusion of stereo features and geometric information.

[0060] (4) Balancing the timeliness and accuracy of the model: At nuclear power plant construction sites, the inspection of embedded parts requires high real-time performance, while the inspection accuracy cannot be compromised. This invention designs a lightweight and efficient two-stage inspection framework to ensure that the real-time requirements are met without sacrificing the inspection accuracy.

[0061] In this invention, the accuracy of dimension measurement based on binocular vision mainly depends on two core elements: high-precision imaging system calibration and a robust stereo matching algorithm. This characteristic is particularly prominent in complex industrial scenarios such as embedded part inspection. Embedded parts, due to their installation on densely reinforced concrete walls and the presence of impurities such as handwritten marks on their surfaces, pose a significant challenge to binocular image matching. This invention applies a binocular 3D detection model in complex backgrounds based on a deformable attention mechanism (DETR) to enhance the robustness of binocular image matching and improve detection accuracy. This invention includes the following steps:

[0062] Step 1: Perform multi-scale extraction and fusion of binocular images based on a pyramid network with stereo-preserving features;

[0063] In a preferred but non-limiting embodiment of the present invention, in step 1, in order to ensure that Deformable DETR feature recognition can extract multi-scale features of embedded parts while preserving stereo features of the stereo image of embedded parts when processing stereo images of steel bars in complex backgrounds, a multi-scale extraction and fusion of stereo images of embedded parts based on a pyramid network with stereo features is proposed.

[0064] The relevant technical features are described below:

[0065] The stereo-preserving feature pyramid network is the "producer" or "upstream".

[0066] The multi-scale correlation cost volume pyramid is the "consumer" or "downstream".

[0067] Below, we will break down their individual functions and the relationships between them in detail.

[0068] 1. Stereo-Preserving FPN: A Stereo-Preserving FPN is a 2D convolutional neural network responsible for extracting features from an input 2D image. Its core objective is:

[0069] Multi-scale features (Pyramid Network / FPN): Objects in the real world have different sizes, and scenes have different structures. The FPN (Feature Pyramid Network) structure can simultaneously extract high-level semantic information (low resolution, large receptive field, for understanding large structures) and low-level detail information (high resolution, small receptive field, for accurately locating edges and textures) from an image. This is crucial for matching at different scales.

[0070] Stereo-preserving: This is crucial. The features it extracts must be suitable for cross-view matching. This means that the features need to be robust to changes in viewpoint, lighting, and slight occlusion, while also being sufficiently sensitive to geometric position and details. Good features ensure that the projections of the same spatial point are very similar in feature space from different viewpoints, while the projections of different spatial points differ significantly.

[0071] For each input view (reference image and source image), the network outputs a set of feature maps at multiple scales. For example, for an HxW image, it may output feature maps at different resolutions such as H / 4xW / 4, H / 8xW / 8, and H / 16xW / 16, where H is the height of the image and W is the width of the image.

[0072] 2. Multi-scale Correlation Cost Volume Pyramid: The cost volume is the core data structure of the MVS algorithm. It is a 4D tensor (depth x height x width x feature or cost) used to store the matching cost of each pixel under different depth assumptions.

[0073] Correlation / Cost Aggregation: Its construction process involves selecting a reference view.

[0074] For each pixel in the reference view, assume a series of possible depth values.

[0075] For each depth hypothesis, the pixel and its features are "projected" onto other source views using homography warping, utilizing camera parameters (intrinsic and extrinsic).

[0076] Calculate the correlation or similarity (e.g., variance, dot product, L1 distance, etc.) between the features of the reference view and the features "sampled" back from the source view. This similarity (or dissimilarity) is the "cost".

[0077] Multi-scale pyramids: Directly constructing a massive cost volume for all depths at full resolution incurs extremely high computational and memory costs. Therefore, modern methods (such as CasMVSNet) employ a pyramid or cascade strategy: coarse-to-fine. First, a coarse cost volume is constructed on the lowest-resolution feature maps (from FPN). This cost volume assumes a wide range of depths but has a small number of features.

[0078] A low-resolution depth map is predicted from this coarse cost volume.

[0079] Then, we move on to the next higher resolution level. Using the coarse depth predicted by the previous layer, we narrow down the depth search range and construct a more refined cost volume with more focused depth assumptions.

[0080] Repeat this process until the final fine depth map is obtained on the highest resolution features.

[0081] This sequence of cost bodies, from coarse to fine and optimized step by step, constitutes the **"multi-scale correlation cost body pyramid"**.

[0082] Summary of the relationship between the two

[0083] A flowchart can be used to clearly illustrate their relationships:

[0084] Input image (multiple views)

[0085] |

[0086] V

[0087] 1. Stereo-Preserving FPN

[0088] Perform 2D convolution independently on each view to extract features.

[0089] Output: [view] Figure 1 Feature Pyramid, View 2 Feature Pyramid, ...]

[0090] |

[0091] V

[0092] 2. Construct a multi-scale correlation cost volume pyramid (MVS).

[0093] Level 1 (coarsest):

[0094] * Use the lowest resolution feature output by FPN.

[0095] * A coarse cost volume is constructed through homography transformation and correlation calculation.

[0096] *After 3DCNN regularization, a coarse depth map is predicted.

[0097] Level 2 (More Refined):

[0098] *Use medium-resolution features from FPN output.

[0099] *Use the coarse depth map from the previous level to narrow down the depth search range.

[0100] *Construct a more refined cost body.

[0101] *Predicts more accurate depth maps.

[0102] ... (Repeat this process)

[0103] Ultimate (Most Refined):

[0104] *Use the highest resolution feature output by FPN.

[0105] *Construct the final, refined cost body.

[0106] |

[0107] V

[0108] 3. Depth Map Regularization and Output (3D U-Net & Argmin)

[0109] 3D convolutions (such as 3D U-Net) are applied to the cost volume of the last stage to smooth and optimize the cost distribution.

[0110] The argmin operation is used to find the depth value that minimizes the cost for each pixel.

[0111] |

[0112] V

[0113] Final high-precision depth map

[0114] Core Relationships:

[0115] Dependency: The construction of the cost volume depends entirely on the features provided by the feature extraction network. The quality of the features directly determines the signal-to-noise ratio of the cost volume. High-quality features are crucial.

[0116] This invention improves the encoder calculation method by using two weight-shared ResNet+FPN to extract multi-scale monocular features from the left and right views of the stereo images captured by the stereo camera. ResNet obtains basic features, while FPN obtains enhanced basic features and fuses information from different levels to form multi-scale monocular features. Based on these monocular features, a multi-scale correlation cost pyramid is constructed by applying a stereo-preserving feature pyramid, and feature fusion is performed to obtain multi-scale stereo features.

[0117] In a preferred but non-limiting embodiment of the present invention, in step 1, during the feature fusion process, stereo features of the same resolution are directly added together, and features of different resolutions are fused across scales through channel expansion and stitching to preserve the corresponding information of the image.

[0118] In a preferred but non-limiting embodiment of the present invention, in step 1, the stereo-preserving feature pyramid network includes a monocular feature pyramid. The upper part of the monocular feature pyramid is the initial pyramid (the initial pyramid is divided into three layers: C2-C4), and the lower part of the monocular feature pyramid is the enhancement pyramid (the enhancement pyramid is divided into three layers: P2-P4). Six pairs of monocular features with three resolution levels are used to construct a six-level correlation-based cost volume pyramid (i.e., a multi-scale correlation cost volume pyramid) by applying the stereo-preserving feature pyramid.

[0119] In a preferred but non-limiting embodiment of the present invention, in step 1, the upper three layers of the six-level correlation-based cost volume pyramid are obtained from the initial monocular features (basic features) of the six-level stereo-preserving feature-based pyramid network, which contain rich detail information; the lower three layers are from the enhanced monocular features (enhanced basic features) of the stereo-preserving feature-based pyramid network, which contain more semantic information; the upper three layers form the initial stereo features (which are obtained through the initial 3D cost volume), and the lower three layers form the enhanced stereo features (which are obtained through the enhanced 3D cost volume). The initial stereo features and the enhanced stereo features are fused according to the resolution. Since the channel dimensions of stereo features at the same resolution have the same physical meaning, the two feature tensors are directly added. The resulting multi-scale stereo features contain rich detail.

[0120] In a preferred but non-limiting embodiment of the present invention, in step 1, among the initial monocular features, the left monocular feature... and right monocular features The initial 3D cost body is constructed using formula (1). :

[0121] (1)

[0122] in, It is a cost body based on correlation. , , The maximum disparity value. The scale index representing the cost volume pyramid. Represents the left view. Represents the right view. Represents cost space; Left-enhanced monocular features The same method was used to construct the augmented 3D cost volume for right-side enhanced monocular features. .

[0123] It can be seen that, and The definitions in the parallax dimension are the same, so addition does not change the physical meaning of that dimension. Therefore, in a preferred but non-limiting embodiment of the present invention, in step 1, the first... Initial three-dimensional features of scale As shown in formula (2):

[0124] (2)

[0125] in Since the definition of disparity dimension is not the same across different scales, cross-scale fusion can be expressed by recursive formula (3):

[0126] (3)

[0127] This indicates that the kernel size is A convolutional layer with a stride of 2. For the first Scale-based convolutional layers, It is the convolution kernel.

[0128] The feature fusion direction of FPN is top-down, that is, using high-level features to supplement the semantic information of low-level features. However, the lowest resolution stereo features are crucial in nonparametric anchor-based target queries, so high-resolution features are used to supplement the detailed information of low-resolution features. This method often results in high-resolution features lacking semantic information. Therefore, an hourglass-shaped stereo-preserving feature pyramid network (STMP) was designed. In this STMP pyramid network, the high-resolution features at the bottom can be enhanced by rich semantic information.

[0129] Step 2: Design a DETR decoder based on parallax-aware location encoding and nonparametric anchor-based target query.

[0130] The Deformable DETR introduces a Multi-Scale Deformable Cross-Attention (MSDeformCA) layer into the DETR decoder. While MSDeformCA has been successful in 2D object detection, directly applying it to binocular 3D object detection of embedded parts in complex backgrounds still leads to model convergence difficulties during training. To accelerate the convergence of such models, in a preferred but non-limiting embodiment of this invention, in step 2, the binocular image correspondence information is encoded into stereo features. This image correspondence information ensures that the model fully utilizes stereo features, rather than getting trapped in local minima of monocular 3D object detection. Therefore, a step is proposed to design a DETR decoder based on disparity-aware position encoding and nonparametric anchor-based target query.

[0131] In a preferred but non-limiting embodiment of the present invention, in step 2, the method of encoding the corresponding information of the binocular images into stereo features includes: disparity-aware position encoding explicitly encodes the corresponding information of the binocular images using disparity logits, the generated position code containing three-dimensional position information in the scene, which is input into the decoder as a supplement to the non-parametric anchor-based target query, thereby introducing the three-dimensional position information into the Transformer decoder; multi-scale stereo features are flattened and concatenated to form a two-dimensional feature matrix, which is used as a multi-scale key / value pair in MSDeformCA; the downsampling rates of the feature maps at three resolutions relative to the input image (binocular images) are 4, 8, and 16, respectively. Let be the dimension of the feature vector in the decoder. For the first Feature maps at multiple levels. In nonparametric anchor-based target queries, only the lowest resolution stereo features are reused, which accelerates training convergence while maintaining inference speed.

[0132] In a preferred but non-limiting embodiment of the present invention, in step 2, for other hyperparameters in MSDeformCA, the following is used: There are 1 decoder layer, each with 1 decoder layer. Each head, and each MSDeformCA sample One offset point;

[0133] To ensure training efficiency while covering the entire 3D space, the nonparametric anchor-based target query reuses the lowest-resolution features from the feature pyramid. This feature map covers the entire 2D image space and, with the help of the proposed disparity-aware location encoding, also covers the 3D space. Because the coordinate correspondence between the nonparametric anchor-based target query and the 2D image space is clear, reference point coordinates are directly generated based on this correspondence. During training, targets are assigned based on the 2D intersection-union ratio (IU) between the anchor boxes and the ground truth boxes. If an IU exceeds a preset foreground threshold... The corresponding anchor point is then assigned as a matching positive sample; conversely, if the intersection-union ratio is less than a preset background threshold, the corresponding anchor point is assigned as a positive sample. The anchor point is then assigned as a negative sample.

[0134] The innovative point of this invention is:

[0135] A stereo 3D detection model based on the Deformable Attention (DETR) mechanism in complex backgrounds. The main innovations of this model are as follows:

[0136] (1) Innovation Point 1: Parallax Perception Position Encoding

[0137] To address the limitations of existing positional encoding in 3D perception, this technique encodes the correspondence between binocular images into normalized sub-pixel disparity and deeply fuses it with sinusoidal 2D positional encoding. By reusing the disparity logits output by a stereo matching network to generate a probabilistic disparity distribution, a positional encoding with 3D spatial perception capabilities is constructed. This design overcomes two major limitations of traditional methods: 2D detection positional encoding lacks depth information; monocular / point cloud detection positional encoding ignores binocular correspondence. Disparity-aware positional encoding, through implicitly learning the 3D geometric features of the scene, enables the decoder to accurately locate the center projection of 3D objects, thereby significantly improving the accuracy of depth and size regression.

[0138] (2) Innovation Point 2: Pyramid Network with Three-Dimensional Preservation Features

[0139] To address the drawback of traditional Feature Pyramid Networks (FPN / BiFPN) in disrupting binocular correspondence, this technique employs a hierarchical fusion strategy: constructing a six-level correlation cost volume pyramid to fuse fine-grained texture and semantic information from both original and enhanced features; and using a specific strategy in cross-scale aggregation to avoid geometric information distortion caused by resolution differences. Compared to traditional FPN / BiFPN, the stereo-preserving feature pyramid network maintains the physical consistency of the cost volume dimensions, enabling multi-scale stereo features to simultaneously preserve spatial correspondence and rich details, providing the decoder with a more discriminative 3D scene representation.

[0140] (3) Innovation Point 3: Non-parametric anchor base target query

[0141] To address the convergence challenge of Deformable DETR decoders in sparse sample scenarios, this technique uses the lowest resolution stereo features (1 / 16 scale) as uniformly distributed query embeddings. This method offers two advantages: by reusing geometrically perceptual features (rather than learnable parameters) to generate dense queries, it achieves full 3D spatial coverage while reducing training parameters; and by combining the 3D position encoding of a pyramid network with stereo-preserving features, it constructs an explicit 2D anchor point prior, improving training efficiency through an IoU-driven positive and negative sample allocation strategy.

[0142] (4) Weak supervision data augmentation strategy

[0143] By utilizing semi-supervised domain adaptive techniques, the reliance on labeled data is reduced (training data requirements are reduced by 40%), and the algorithm's generalization ability is improved by 35%.

[0144] (5) Lightweight two-stage detection framework

[0145] The first stage generates coarse-grained region proposals (takes <30ms), and the second stage performs fine-grained classification and localization, achieving real-time detection of 100ms / frame on portable hardware (host weight / volume <10kg / 15L).

[0146] This invention targets the inspection of embedded parts in nuclear power plants, and based on cutting-edge technology, achieves an advanced level in the following dimensions:

[0147] Binocular high-precision detection: accurate identification of embedded parts (target detection recall rate of 98%), positioning and detection accuracy of embedded parts reaches ±3mm, and their physical size accuracy can reach ±1mm, meeting the actual needs of nuclear power engineering.

[0148] High-efficiency operation of lightweight host: High-precision detection is achieved using a lightweight host with limited computing power (using a single GPU host or a host equipped with a mobile GPU, host weight / volume <10kg / 15L), with an inference speed of 100ms / frame;

[0149] Application versatility: It eliminates the dependence on LiDAR, supports low-cost training and deployment of binocular cameras, and has low requirements for training data and detection equipment.

[0150] Compared with similar studies at home and abroad, this invention provides a brand-new solution for object detection in complex engineering scenarios through innovative construction of Deformable DETR, and has significant technological leadership and engineering application potential.

[0151] This invention, based on the strategic needs of intelligent construction, addresses the core challenges of embedded component inspection by achieving breakthroughs in algorithm architecture, feature modeling, and resource optimization. It not only improves engineering quality and construction efficiency but also promotes the theoretical deepening and application expansion of deep learning technology in structural inspection, providing crucial technical support for the digital transformation of my country's construction industry.

[0152] The technical approach of this invention is as follows:

[0153] (1) Phase 1: Algorithm Design and Model Development

[0154] ① Binocular vision data acquisition and annotation

[0155] Images of embedded parts at the construction site were acquired using a binocular camera, covering different lighting conditions (strong light, shadow), different angles (direct, oblique) and densely arranged scenes; the location, size and spacing of the embedded parts were finely annotated using custom scripts or other methods, and a multi-scene embedded part detection dataset containing 10,000+ samples was constructed; the labeled dataset (including RGB images, disparity maps and 3D coordinate ground truth) was also constructed.

[0156] ②Deformable DETR Development

[0157] Based on the PyTorch framework, a stereo-preserving pyramid network and a disparity-aware position coding module were implemented, and cross-scale feature fusion and disparity coding strategies were designed. The module parameters (such as the number of feature pyramid layers and the number of attention heads) were optimized through ablation experiments on an existing stereo image dataset of embedded parts (4500+ images). The Deformable DETR prototype model was obtained, which supports stereo input and 3D coordinate output.

[0158] (2) Phase Two: Algorithm Optimization and Efficiency Improvement

[0159] ① Construction of a lightweight two-stage detection framework

[0160] A coarse-grained region proposal is generated using a Deformable DETR encoder (time ≤ 30ms); fine-grained classification and localization are performed based on an improved Deformable DETR decoder; and knowledge distillation technology is used to compress the model size, keeping the number of model parameters within 50MB.

[0161] ② Dynamic resource allocation and weakly supervised learning

[0162] The design includes a scenario complexity evaluation module to dynamically adjust the feature map resolution and the number of attention heads; it utilizes contrastive learning and domain adaptation techniques to pre-train on unlabeled data, reducing label dependency; and it tests generalization ability on cross-scenario datasets to compare the performance of fully supervised models.

[0163] ③ Edge deployment and real-time testing

[0164] The system will be deployed based on a single GPU computing platform; FP16 quantization and layer fusion technology will be adopted to achieve an inference speed of ≤100ms / frame; and an embedded detection system prototype will be built to support real-time visualization of detection results.

[0165] (3) Phase 3: Engineering verification and system integration

[0166] Conduct on-site testing in conjunction with actual engineering projects; compare the results of manual measurement to verify the errors in size (±1mm) and position (±3mm); record the testing time of a single module (4m×6m) and compare it with the efficiency of traditional manual methods; and compile test reports and algorithm iteration plans.

[0167] To address the shortcomings of existing technologies, such as insufficient detection accuracy, low efficiency, poor safety, and high cost, this invention proposes an intelligent inspection and acceptance method for nuclear power plant embedded components based on a binocular camera. Its key technical points are as follows:

[0168] 1. Binocular vision-based 3D information acquisition and high-precision measurement technology for embedded parts

[0169] Technical means: The left and right view images of the embedded part are acquired simultaneously using a binocular camera, the disparity map is calculated using a stereo matching algorithm, and the three-dimensional point cloud model of the embedded part and its surrounding environment is reconstructed by combining the camera calibration parameters or the three-dimensional coordinates are directly calculated.

[0170] Disadvantages of the solution:

[0171] Insufficient detection accuracy: Instead of manual reading and estimation, computer vision algorithms can accurately calculate the spatial position (X, Y, Z coordinates) and orientation of the embedded parts, as well as their physical dimensions, significantly reducing human error and improving measurement accuracy (target positioning ±3mm, physical dimensions ±1mm).

[0172] Difficult to detect complex situations: Binocular vision can acquire information from different perspectives, and combined with subsequent algorithms, it can better handle situations where embedded parts partially obstruct the view and complex background interference (such as dense steel bars), and extract effective features.

[0173] 2. Intelligent identification and positioning technology for embedded parts in complex backgrounds based on Deformable Attention Mechanism (DETR)

[0174] Technical approach: An improved Deformable DETR model is applied, which can adaptively focus on key areas (embedded part features) in the image and effectively handle complex background interference such as dense steel bars, lighting changes, and surface markings of embedded parts at nuclear power plant sites.

[0175] Disadvantages of the solution:

[0176] Insufficient detection accuracy (human error, complex situations): Improve the robustness of the model in complex backgrounds and the recall (98%) and accuracy of embedded part identification, and reduce false detections and missed detections, especially in scenarios where it is difficult for traditional manual methods to make accurate judgments.

[0177] Low detection efficiency: Automated identification and positioning eliminates the need for manual point-by-point searching and judgment of embedded parts.

[0178] 3. Stereo-Retaining Feature Pyramid Network (SR-FPN) technology

[0179] Technical approach: An hourglass-shaped feature pyramid network is designed. Multi-scale monocular features of the left and right views are extracted by ResNet+FPN with weight sharing. A multi-scale correlation cost pyramid is constructed. During feature fusion, a specific strategy is adopted (direct addition at the same resolution, and channel expansion and stitching at different resolutions) to perform cross-scale fusion. This is to preserve the stereo geometric correspondence and detail information of the binocular images to the maximum extent while extracting multi-scale features.

[0180] Disadvantages of the solution:

[0181] Insufficient detection accuracy (information loss): This addresses the problem of stereo correspondence information loss or geometric information destruction that may occur when traditional FPN processes binocular images due to cross-resolution feature fusion, providing richer details and more accurate stereo features for subsequent high-precision 3D reconstruction and localization.

[0182] Balancing the timeliness and accuracy of the model: Through effective multi-scale feature fusion, the model can simultaneously focus on global context and local details, which helps to optimize the model's ability to detect embedded parts of different sizes while ensuring accuracy.

[0183] 4. Design of a DETR decoder based on disparity-aware positional encoding (DAPE)

[0184] Technical approach: The correspondence between binocular images (probabilistic disparity distribution output by stereo matching network) is innovatively encoded into normalized subpixel level disparity, and then deeply fused with traditional two-dimensional sinusoidal position coding to generate a position code containing three-dimensional spatial depth information, which is then input into the decoder of Deformable DETR.

[0185] Disadvantages of the solution:

[0186] Insufficient detection accuracy (lack of 3D perception): Overcomes the limitations of existing 2D detection location encoding lacking depth perception and 3D point cloud detection methods ignoring the geometric correspondence of binocular images. By introducing parallax information, the decoder can more accurately perceive and locate objects in 3D space, improving the accuracy of depth and size regression.

[0187] Model convergence and detection accuracy: Enhancing the model's understanding of 3D space helps to accelerate the convergence speed of the model in binocular 3D target detection tasks and improve the final detection accuracy.

[0188] 5. Non-parametric Anchor-based Object Query Mechanism

[0189] Technical approach: The lowest resolution stereo feature map in SR-FPN is reused as the query embedding to generate dense, non-parametric target queries covering the entire 3D space. Combined with the 3D location information provided by DAPE, an explicit 2D anchor point prior is constructed, and an IoU-driven positive and negative sample allocation strategy is adopted.

[0190] Disadvantages of the solution:

[0191] Low detection efficiency (slow model convergence): This addresses the issue of slow training convergence and low efficiency that Deformable DETR may encounter in sparse samples or complex 3D scenes. By reusing features to replace learnable parameters in query generation, the number of training parameters is reduced while ensuring dense coverage of the 3D space, thus improving the model's convergence efficiency and training stability.

[0192] Balancing the timeliness and accuracy of the model: While ensuring detection accuracy, the inference speed of the model is optimized through an efficient query mechanism and sample allocation.

[0193] 6. Lightweight two-stage detection framework and efficient edge deployment technology

[0194] Technical approach: The design involves a first stage for rapid generation of coarse-grained region proposals (time <30ms), followed by a second stage utilizing an improved Deformable DETR for refined classification and localization. Combining model compression and acceleration techniques such as knowledge distillation, FP16 quantization, and layer fusion, the system achieves efficient operation (inference speed 100ms / frame) on lightweight hosts (weight / volume <10kg / 15L) with a single GPU or mobile GPU.

[0195] Disadvantages of the solution:

[0196] Low detection efficiency: Significantly improves detection speed, achieving near real-time detection, far exceeding manual efficiency.

[0197] High testing costs (manpower, time): By automating and rapidly testing, the manpower (no need for multiple people to cooperate) and time required for testing are significantly reduced, thus lowering the overall cost.

[0198] Poor safety: Portable equipment and remote operation capabilities allow inspection personnel to operate in safe areas, reducing the need to enter dangerous areas such as high altitudes and enclosed spaces, thus improving operational safety.

[0199] 7. Weakly supervised data augmentation strategy (semi-supervised domain adaptive technique)

[0200] Technical means: By using semi-supervised learning and domain adaptation techniques, some unlabeled or weakly labeled data can be effectively utilized during the training process, reducing the dependence on a large amount of finely labeled data.

[0201] Disadvantages of the solution:

[0202] The detection cost is relatively high (data annotation cost): it reduces the human and time costs of building high-quality training datasets (reducing training data requirements by 40%), while improving the generalization ability of the algorithm (by 35%).

[0203] In summary, the key technical points of this patent are as follows: This invention deeply optimizes and adapts the Deformable DETR model by using an innovative pyramid network that preserves stereo features, parallax-aware position encoding, and a non-parametric anchor-based target query mechanism. Combined with the principle of binocular vision three-dimensional measurement and a lightweight deployment strategy, it achieves high-precision, high-efficiency, and high-safety intelligent detection of nuclear power plant embedded parts in complex backgrounds, effectively overcoming many drawbacks of traditional manual detection methods.

[0204] The beneficial effects of the present invention are as follows, compared with the prior art:

[0205] 1. High-precision feature extraction: Using a pyramid network that preserves three-dimensional features, it can effectively extract the size and location of embedded parts in complex environments. The target detection recall rate is 98%, the positioning detection accuracy reaches ±3mm, and the physical size accuracy can reach ±1mm, reducing the false detection rate and ensuring the safety and reliability of key areas in nuclear power construction.

[0206] 2. Enhanced 3D spatial perception capability: By introducing parallax perception position coding, the deep fusion of binocular parallax information and 2D position coding is achieved, which improves the accurate reflection of the position information of objects in the 3D scene and thus improves the detection accuracy.

[0207] 3. Effective control of information loss: In the process of cross-scale and multi-resolution feature fusion, the problem of information loss in traditional methods is innovatively solved, ensuring the effective fusion of stereo features and geometric information and improving the overall detection performance.

[0208] 4. Balance between real-time performance and accuracy: The lightweight two-stage detection framework is designed to ensure that while meeting real-time requirements, it maintains high detection accuracy and adapts to the actual needs of nuclear power plant construction sites.

[0209] 5. Wide applicability: This invention eliminates the dependence on LiDAR, supports low-cost binocular camera training and deployment, reduces the requirements for training data and detection equipment, and has strong market adaptability.

[0210] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention without departing from the spirit and scope of the present invention. Any modifications or equivalent substitutions should be covered within the scope of protection of the claims of the present invention.

Claims

1. A method for intelligent inspection and acceptance of nuclear power plant embedded parts based on binocular cameras, characterized in that, include: Step 1: Perform multi-scale extraction and fusion of binocular images based on a pyramid network with stereo-preserving features; Step 2: Design a DETR decoder based on parallax-aware location encoding and nonparametric anchor-based target query.

2. The intelligent inspection and acceptance method for nuclear power plant embedded parts based on a binocular camera according to claim 1, characterized in that, In step 1, two weight-shared ResNet+FPN are used to extract multi-scale monocular features from the left and right views of the stereo images captured by the stereo camera. ResNet obtains basic features, while FPN obtains enhanced basic features and fuses information from different levels to form multi-scale monocular features. Based on these monocular features, a pyramid of stereo-preserving features is used to construct a multi-scale correlation cost volume pyramid, and feature fusion is performed to obtain multi-scale stereo features.

3. The intelligent inspection and acceptance method for nuclear power plant embedded parts based on a binocular camera according to claim 2, characterized in that, In step 1, during the feature fusion process, stereo features of the same resolution are directly added together, while features of different resolutions are fused across scales through channel expansion and stitching to preserve the corresponding information in the image.

4. The intelligent inspection and acceptance method for nuclear power plant embedded parts based on a binocular camera according to claim 3, characterized in that, In step 1, the stereo-preserving feature pyramid network includes a monocular feature pyramid, the upper half of which is the initial pyramid and the lower half of which is the enhancement pyramid; six pairs of monocular features with three resolution levels are used to construct a six-level correlation-based cost volume pyramid by applying the stereo-preserving feature pyramid.

5. The intelligent inspection and acceptance method for nuclear power plant embedded parts based on a binocular camera according to claim 4, characterized in that, In step 1, the top three layers of the six-level correlation-based cost volume pyramid are obtained from the initial monocular features of the six-level stereo-preserving feature-based pyramid network, and the bottom three layers are obtained from the enhanced monocular features of the stereo-preserving feature-based pyramid network. The upper three layers form the initial stereo features, and the lower three layers form the enhanced stereo features. The initial stereo features and the enhanced stereo features are then fused according to the resolution.

6. The intelligent inspection and acceptance method for nuclear power plant embedded parts based on a binocular camera according to claim 5, characterized in that, In step 1, in the initial monocular features, the left monocular feature and right monocular features The initial 3D cost body is constructed using formula (1). : (1) in, It is a cost body based on correlation. , , The maximum disparity value. The scale index representing the cost volume pyramid. Represents the left view. Represents the right view. Represents cost space; Left-enhanced monocular features The same method was used to construct the augmented 3D cost volume for right-side enhanced monocular features. .

7. The intelligent inspection and acceptance method for nuclear power plant embedded parts based on a binocular camera according to claim 6, characterized in that, In step 1, the first Initial three-dimensional features of scale As shown in formula (2): (2) in Cross-scale fusion is represented by recursive formula (3): (3) This indicates that the kernel size is A convolutional layer with a stride of 2. For the first Scale-based convolutional layers, It is the convolution kernel.

8. The intelligent inspection and acceptance method for nuclear power plant embedded parts based on a binocular camera according to claim 7, characterized in that, In step 2, the information corresponding to the binocular images is encoded into the stereo features.

9. The intelligent inspection and acceptance method for nuclear power plant embedded parts based on a binocular camera according to claim 8, characterized in that, In step 2, the method for encoding the corresponding information of the binocular images into stereo features includes: disparity-aware position encoding explicitly encodes the corresponding information of the binocular images based on disparity log odds. The generated position code contains three-dimensional position information in the scene. This three-dimensional position information is input into the decoder as a supplement to the non-parametric anchor-based target query, thereby introducing the three-dimensional position information into the Transformer decoder. Multi-scale stereo features are flattened and concatenated to form a two-dimensional feature matrix, which is used as a multi-scale key / value pair in MSDeformCA. The downsampling rates of the feature maps at three resolutions relative to the input images are 4, 8, and 16, respectively. Let be the dimension of the feature vector in the decoder. For the first Feature maps at each level.

10. The intelligent inspection and acceptance method for nuclear power plant embedded parts based on a binocular camera according to claim 9, characterized in that, In step 2, for other hyperparameters in MSDeformCA, use There are 1 decoder layer, each with 1 decoder layer. Each head, and each MSDeformCA sample One offset point; During training, targets are assigned based on the 2D intersection-union ratio (IU) between anchor boxes and ground truth boxes. If an IU exceeds a preset foreground threshold... The corresponding anchor point is then assigned as a matching positive sample; conversely, if the intersection-union ratio is less than a preset background threshold, the corresponding anchor point is assigned as a positive sample. The anchor point is then assigned as a negative sample.

Images