Intelligent detection platform and method for nuclear power embedded part installation based on machine vision

Abstract

The invention provides a nuclear power embedded part installation intelligent detection platform and method based on machine vision, and belongs to the technical field of data processing, and the method comprises the steps: 1, carrying out the high-precision joint calibration of a high-definition camera and a laser radar; 2, identifying the nuclear power embedded parts in real time by using a high-definition camera, and detecting the sizes and positions of all the nuclear power embedded parts by using the calibrated high-definition camera and laser radar; 3, a high-definition camera is used for shooting and detecting the inner wall body of the building where the nuclear power embedded part is located for multiple times, detection data are spliced, and nuclear power embedded part installation data of the complete wall body are formed; and 4, comparing with the BIM data of the building where the nuclear power embedded parts are located, and identifying the abnormally installed nuclear power embedded parts. The defects of limited detection precision, low detection efficiency, poor safety, low visualization degree and low automation degree of nuclear power embedded part detection in the prior art are effectively avoided.

Description

Technical Field

[0001] The present invention belongs to the technical field of data processing, and in particular relates to a machine vision-based intelligent detection platform and method for the installation of nuclear power embedded parts. Background Art

[0002] Against the backdrop of a global energy transition, nuclear power, as a clean, efficient, and stable form of energy, plays a vital role in meeting growing energy demand and reducing carbon emissions. The nuclear power industry is currently experiencing rapid development, with the scale and number of nuclear power plants expanding. The construction of nuclear power projects involves numerous complex technologies and stringent quality standards, with long construction cycles and substantial investments, placing extremely high demands on safety and reliability. Throughout the entire nuclear power project construction process, quality control at every stage is crucial. One facility used in nuclear power projects is the nuclear power embedded components described in the prior art solution with patent publication number "CN208857970U."

[0003] Specifically, nuclear power embedded parts are critical components pre-installed during the civil construction of nuclear power projects. They primarily include penetrations, embedded parts, opening boxes, and anchors. These embedded parts play a crucial role in connection and support during subsequent installation, ensuring the proper functioning of all nuclear power plant systems. For example, in areas such as the nuclear island, conventional island, and auxiliary buildings, the accurate installation of these embedded parts is crucial for the welding and installation of pipelines and equipment supports. The accuracy of their type, position, and dimensions directly impacts the precision and stability of subsequent equipment installation, ultimately impacting the safe operation of the entire nuclear power plant. Misalignment in the installation of these embedded parts can lead to problems such as loose pipeline connections and unstable equipment supports.

[0004] Traditional instruments for detecting embedded parts of nuclear power plants mainly rely on tools such as total stations, levels, tape measures, plumb bobs, etc. The technical solutions are as follows:

[0005] At the construction site, inspectors first use a total station to measure the embedded nuclear power components. This instrument transmits and receives laser signals to measure the three-dimensional coordinates of specific points on the components. For example, for a large reactor pressure vessel foundation embedded component, inspectors need to set up total stations at different locations, aiming at multiple measurement points on the component, such as edge corners and center positions, to obtain coordinate data (e.g., X, Y, and Z coordinates). Next, they use a level to measure the levelness of the embedded component to ensure that it meets the design requirements horizontally. A level is also used to assist in measuring height differences, such as the relative heights of embedded components on different floors.

[0006] Tape measures are widely used for dimensional measurement. Inspectors use them to directly measure the length, width, and height of embedded nuclear power components. For areas that are difficult to measure directly, such as holes within embedded nuclear power components, they may use tools like plumb bobs to indirectly obtain dimensional information by measuring the distance between the plumb bob and the edge of the hole.

[0007] After the measurement is complete, the inspectors manually record the coordinates, dimensions, and other data on paper or electronic documents. These records are then compared with the design parameters for the nuclear power embedded components on the design drawings. The design drawings detail the ideal position, size, and shape of each nuclear power embedded component. Through visual observation and simple calculations, the inspectors determine whether the deviation between the actual measured values ​​and the designed values ​​is within the allowable range, thereby confirming the quality of the nuclear power embedded components.

[0008] The main shortcomings of traditional instruments for detecting nuclear power embedded parts are as follows:

[0009] (1) Limited detection accuracy:

[0010] While total stations can measure three-dimensional coordinates, their accuracy is easily affected by various factors in complex construction site environments. For example, the presence of large amounts of construction materials, equipment, and personnel can obstruct the total station's line of sight, leading to incomplete or inaccurate signal reflections and, consequently, deviations in the measured coordinates.

[0011] Tape measure measurement relies on manual operation, and measurement accuracy is significantly affected by the inspector's skill and experience. Factors such as the tape's tension and the determination of the starting and ending points can introduce errors during the measurement process. Especially with long or irregularly shaped embedded parts, the tape measure struggles to precisely follow their edges, resulting in inaccurate dimensional measurements. Furthermore, the tape measure cannot accurately measure subtle dimensional variations, such as surface flatness deviations, making it easy to overlook potential quality issues.

[0012] Manual data recording and comparison are also prone to errors. Due to the large volume of data, inspectors may make typos or data entry errors during the recording process. When comparing against design drawings, manual calculations and judgments can be prone to oversights, failing to accurately identify minor deviations, which can affect the accurate assessment of the embedded component installation quality.

[0013] (2) Low detection efficiency:

[0014] Traditional instrument testing requires inspectors to measure each embedded component individually, a cumbersome process. The large number of embedded nuclear power components, such as the numerous steel plates, casing, and anchors found in the Hualong One nuclear island, creates a significant testing workload. Each embedded component requires multiple measurements and data recording, which is time-consuming. For example, using a total station to measure the coordinates of multiple points on a single embedded component can take several minutes. Adding the need to adjust the instrument's position and record the data, testing a single embedded component can take more than ten minutes or even longer, resulting in a lengthy testing cycle and a significant impact on project progress.

[0015] Switching and coordinating between different instruments can also reduce inspection efficiency. During the inspection process, tools such as total stations, levels, and tape measures need to be frequently switched according to different inspection requirements. This not only increases operational complexity but also easily leads to instrument setup errors, further delaying inspection time. Furthermore, the data formats and recording methods of different instruments vary, requiring additional processing time for data integration and comparison.

[0016] (3) Poor security:

[0017] Some embedded components in nuclear power plants are installed at high altitudes, such as high up in the nuclear island or above large equipment. When using traditional instruments for inspection, inspectors must use auxiliary equipment such as scaffolding and ladders to perform these tasks. This increases the risk of accidents such as falls and collisions. For example, during the construction and climbing of scaffolding and ladders, improper operation or equipment failure can lead to falls and injuries. When working at height, inspectors may also be in danger due to an unstable center of gravity or external forces.

[0018] Construction sites are complex environments with various potential hazards, such as unsecured building materials and operating machinery. When inspectors are focused on measuring operations, they tend to neglect the safety of their surroundings, increasing the likelihood of accidental injury.

[0019] (4) Low level of visualization:

[0020] Traditional inspection methods rely primarily on manual data recording and comparison with paper design drawings. This method fails to clearly demonstrate discrepancies between the actual installation of nuclear power embedded components and design requirements. Inspectors must visualize and compare measured data with design drawings, making it difficult to quickly and accurately identify installation quality issues. For example, for complex nuclear power embedded components, such as those with multiple holes and protrusions, data comparison makes it difficult to intuitively determine the accuracy of their installation position and shape.

[0021] There's no real-time feedback on test results. During the inspection process, inspectors can't immediately see the overall installation status and deviations of nuclear power embedded components. Conclusions can only be drawn after all measurements and comparisons are complete. If installation issues aren't promptly adjusted and corrected, subsequent construction could proceed based on the incorrect nuclear power embedded component installation, increasing rectification costs and the risk of project delays.

[0022] (5) Low degree of automation:

[0023] Traditional testing relies entirely on manual labor, requiring every step, from instrument setup and measurement to data recording and comparison, to be completed manually by testers. This is not only labor-intensive but also susceptible to factors such as tester fatigue and mood, leading to reduced stability and reliability of test results. For example, prolonged testing can cause testers to lose focus, affecting measurement accuracy and data recording.

[0024] Lack of automated data processing and analysis capabilities. Traditional inspection methods are unable to automatically compile, analyze, and mine large amounts of inspection data, and are unable to provide comprehensive assessments and trend analysis of embedded component installation quality. For example, the inability to automatically calculate information such as the average deviation and deviation distribution of embedded components in different areas hinders overall control and optimization of project quality. Summary of the Invention

[0025] In order to solve the defects in the existing technology, the present invention provides a machine vision-based intelligent detection platform and method for the installation of nuclear power embedded parts, which effectively avoids the defects of the existing technology in detecting nuclear power embedded parts, such as limited detection accuracy, low detection efficiency, poor safety, low visualization and low automation.

[0026] The present invention utilizes the following technical solutions.

[0027] A machine vision-based intelligent detection method for nuclear power embedded parts installation, comprising:

[0028] Step 1: Perform high-precision joint calibration of the HD camera and LiDAR;

[0029] Step 2: Use high-definition cameras to identify nuclear power embedded parts in real time, and use calibrated high-definition cameras and lidar to detect the size and position of all nuclear power embedded parts;

[0030] Step 3: Use a high-definition camera to take multiple photos and inspect the wall surface of the building where the nuclear power embedded parts are located, and splice the inspection data to form the nuclear power embedded parts installation data of the complete wall surface;

[0031] Step 4: Compare the data with the BIM data of the building where the nuclear power embedded parts are located to identify abnormally installed nuclear power embedded parts.

[0032] Furthermore, step 1 specifically includes:

[0033] Step 1-1: HD camera calibration;

[0034] Step 1-2: Joint calibration of HD camera and lidar.

[0035] Furthermore, step 1-1 specifically includes:

[0036] Step 1-1-1: Use a high-definition camera to obtain a grayscale image of the checkerboard image and transmit it to the mainboard's CPU;

[0037] Step 1-1-2: The CPU of the motherboard searches for the corner points of the grayscale image of the checkerboard image;

[0038] Step 1-1-3: Perform HD camera calibration based on the corner points of the grayscale image of the checkerboard image. HD camera calibration is to estimate the HD camera internal parameters, which include the focal length, principal point coordinates, and distortion coefficient of the HD camera.

[0039] Furthermore, in step 1-1-1, multiple sets of grayscale images of checkerboard images are captured by panning left and right and tilting the lens of the high-definition camera up and down, so that the checkerboard in the grayscale image of each checkerboard image occupies no less than two-thirds of the pixels in the picture.

[0040] Furthermore, in step 1-1-2, the algorithm used by the central processing unit of the mainboard to search for corner points of the grayscale image of the grid image is the Harris corner detection algorithm or the Shi-Tomasi corner detection algorithm.

[0041] Furthermore, in step 1-1-3, the direct linear transformation algorithm is applied to convert the image coordinates of the corner points of the grayscale image into real world coordinates, and the imaging model of the high-definition camera is defined as:

[0042]

[0043] in, is the image coordinate of the corner point of the grayscale image, is the actual world coordinate of the corner point of the grayscale image, is the focal length of the HD camera, is the principal point coordinate of the HD camera, and They are the distortion component on the X-axis of the image coordinate and the distortion component on the Y-axis of the image coordinate, respectively.

[0044] Furthermore, steps 1-2 specifically include:

[0045] Step 1-2-1: Fix the HD camera and laser radar to make their positions relatively stable;

[0046] Step 1-2-2: Fix a specific stereo calibration plate. The stereo calibration plate is divided into 4 groups of images. Each group of images contains an aurco mark. The length, width and position of the stereo calibration plate are all fixed parameters.

[0047] Step 1-2-3: Use a high-definition camera and a lidar to collect 2D high-definition images and 3D lidar data respectively and transmit them to the mainboard's central processor. The 2D high-definition image includes 2D information of four sets of image corner points, and the 3D lidar data includes 3D corner point information of four sets of images.

[0048] Step 1-2-4: The mainboard's CPU converts the laser radar's coordinate system to the HD camera's visual coordinate system.

[0049] Furthermore, steps 1-2-4 specifically include:

[0050] Step 1-2-4-1: Label 2D high-definition images and 3D lidar data;

[0051] Step 1-2-4-2: Select a suitable deep learning architecture, which can be a convolutional neural network or a recurrent neural network.

[0052] Step 1-2-4-3: Apply a deep learning architecture to learn the nonlinear transformation relationship between 2D HD images and 3D LiDAR data to obtain a trained model;

[0053] Step 1-2-4-4: Input the new HD camera image and lidar data into the trained model to obtain the conversion result between the lidar coordinate system and the HD camera visual coordinate system;

[0054] Step 1-2-4-5: Filter out the 3D lidar data except the aurco mark in the 3D lidar data to obtain the aurco point cloud data;

[0055] Steps 1-2-4-6: Project the ArUco point cloud data, that is, convert the ArUco point cloud data into the visual coordinate system of the HD camera, and use a polygon to select the ArUco point cloud data converted to the visual coordinate system of the HD camera;

[0056] Steps 1-2-4-7: Obtain the corner points of the polygon and record the coordinates of the corner points on the 2D HD image and the coordinates of the 3D LiDAR data;

[0057] Steps 1-2-4-8: Obtain the R|T matrix from the HD camera to the LiDAR based on the coordinates of the corner points on the 2D HD image and the coordinates of the 3D LiDAR data. The R|T matrix is ​​the rotation matrix R and translation vector T in the external parameters.

[0058] Furthermore, in steps 1-2-4-8, the coordinates of the corner point 3D lidar data are defined as , the coordinates of the corner points on the 2D high-definition image are , the extrinsic rotation matrix is , the translation vector is , and Obtained by the following formula:

[0059] .

[0060] Furthermore, step 2 specifically includes:

[0061] Step 2-1: Use high-definition cameras and lidar to collect high-definition images and lidar data of nuclear power embedded parts respectively;

[0062] Step 2-2: Pre-processing high-definition images of nuclear power embedded parts;

[0063] Step 2-2: Identify nuclear power embedded parts in high-definition images.

[0064] Furthermore, step 2 specifically includes:

[0065] Edge defect detection uses the Hough transform to detect rectangular boxes in an image. Edge detection algorithms extract edges within the image, which serve as the basis for rectangle detection. The Hough transform converts each edge line in the image into a point in parameter space, and a voting mechanism is used to find the most likely line. By combining several lines, candidate rectangles are generated. The algorithm maps each detected rectangle into a parameter space, where each corner of the rectangle is represented by a set of parameters. Voting across these parameter spaces finds the optimal rectangle. Based on the rectangles, sub-pixel detection is used to accurately determine the coordinates of the corner points.

[0066] Furthermore, step 3 specifically includes:

[0067] Step 3-1: Extract the contour depth data within the corner point range from the laser radar point cloud data;

[0068] Step 3-2: Calculate its physical size through 3D affine.

[0069] A machine vision-based intelligent detection platform for nuclear power embedded parts installation, including:

[0070] Display screen, HD camera and LiDAR connected to the motherboard’s CPU;

[0071] The modules running on the motherboard's CPU include:

[0072] Calibration module, which is used to perform high-precision joint calibration of high-definition cameras and lidar;

[0073] The detection module is used to identify nuclear power embedded parts in real time using a high-definition camera and detect the size and position of all nuclear power embedded parts using a calibrated high-definition camera and lidar;

[0074] The splicing module is used to use a high-definition camera to take multiple photos and detect the wall surface of the building where the nuclear power embedded parts are located, and then splice the detection data to form the nuclear power embedded parts installation data of the complete wall surface;

[0075] The comparison module is used to compare with the BIM data of the building where the nuclear power embedded parts are located, and identify abnormally installed nuclear power embedded parts.

[0076] The beneficial effects of the present invention are as follows:

[0077] By employing advanced machine vision technologies, such as high-precision visual sensors (panoramic cameras, high-pixel industrial cameras, and LiDAR), precise object detection algorithms (based on the YOLO framework and proprietary algorithms), and automated data processing and visualization systems, we can effectively address the challenges inherent in traditional instrument-based inspections. Visual sensors can quickly and accurately capture image information of embedded components, unaffected by environmental factors and human interaction. Object detection algorithms automatically analyze images for highly accurate object identification, location, and dimensional measurement. Automated data processing and visualization systems provide real-time feedback on inspection results, improving efficiency and accuracy while enhancing visualization. This provides a more reliable and efficient solution for quality inspection of embedded component installations in nuclear power plants. BRIEF DESCRIPTION OF THE DRAWINGS

[0078] Figure 1 This is a flow chart of the intelligent detection method for nuclear power embedded parts installation based on machine vision in the present invention.

[0079] Figure 2 This is a partial structural diagram of the machine vision-based intelligent detection platform for nuclear power embedded parts installation in the present invention;

[0080] Figure 3 Schematic diagram of four groups of images of the stereo calibration plate in the present invention. DETAILED DESCRIPTION

[0081] To make the objectives, technical solutions, and advantages of the present invention more clear, the following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely express the technical solutions of the present invention. The embodiments expressed in this application are only part of the embodiments of the present invention, not all of the embodiments. Based on the spirit of the present invention, other embodiments obtained by ordinary technicians in this field without making creative work are all within the scope of protection of the present invention.

[0082] like Figure 1 As shown, a method for intelligent detection of nuclear power embedded parts installation based on machine vision includes:

[0083] Step 1: Fix the high-definition camera and lidar in the machine vision-based intelligent detection platform for nuclear power embedded parts installation and perform high-precision joint calibration;

[0084] Step 2: Use high-definition cameras to identify nuclear power embedded parts in real time, and use calibrated high-definition cameras and lidar to detect the size and position of all nuclear power embedded parts;

[0085] Step 3: Use a high-definition camera to take multiple photos and inspect the wall surface of the building where the nuclear power embedded parts are located, and splice the inspection data to form the nuclear power embedded parts installation data of the complete wall surface;

[0086] Step 4: Compare the data with the BIM data of the building where the nuclear power embedded components are located to quickly identify abnormally installed nuclear power embedded components. This method can efficiently and accurately detect nuclear power embedded components in buildings and quickly integrate them into the digital system.

[0087] The core inspection criteria of this invention are to measure the size and position accuracy of nuclear power embedded parts, as well as the integrity of their edges. Furthermore, this equipment involves precision optical testing, requiring the calibration of white light 2D images captured by a high-definition camera and laser depth-of-field images captured by a lidar, and regular recalibration.

[0088] Therefore, the main algorithms include: 2D3D (HD camera and lidar) calibration, size detection, edge defect detection, and position detection.

[0089] In a preferred but non-limiting embodiment of the present invention, step 1 specifically comprises:

[0090] The main errors of 2D high-definition cameras come from fixed errors such as focal length tolerance caused by production, principal point offset caused by assembly, and distortion caused by the lens. These errors can be corrected by calibrating the 2D high-definition camera and calculating the camera's intrinsic parameters, extrinsic parameters, and distortion coefficients. The main error of 3D lidar comes from spot offset caused by hardware and timing. This error generally cannot be corrected directly and requires the subsequent acquisition of depth of field data using a specific stereo calibration plate, followed by calculation of the various parameters of the 2D projection image (intrinsic parameters, extrinsic parameters, and distortion coefficients) after projection.

[0091] Step 1-1: 2D HD camera calibration;

[0092] In a preferred but non-limiting embodiment of the present invention, step 1-1 specifically comprises:

[0093] Step 1-1-1: Use a high-definition camera to obtain a grayscale image of the checkerboard image and transmit it to the mainboard's CPU;

[0094] In a preferred but non-limiting embodiment of the present invention, in step 1-1-1, the high-definition camera lens is panned left and right and tilted up and down to capture multiple sets of grayscale images of the checkerboard pattern, such that the checkerboard grid in each grayscale image occupies no less than two-thirds of the pixels in the image. The high-definition camera captures a total of approximately sixteen images. The distance between the high-definition camera and the checkerboard pattern image is approximately 3 meters.

[0095] Step 1-1-2: The CPU of the motherboard searches for the corner points of the grayscale image of the checkerboard image;

[0096] In a preferred but non-limiting embodiment of the present invention, in step 1-1-2, the algorithm used by the CPU of the mainboard to search for corner points of the grayscale image of the grid image is the Harris corner detection algorithm or the Shi-Tomasi corner detection algorithm.

[0097] Step 1-1-3: Perform HD camera calibration based on the corner points of the grayscale image of the checkerboard image. HD camera calibration is to estimate the HD camera internal parameters, which include the focal length, principal point coordinates, and distortion coefficient of the HD camera.

[0098] In a preferred but non-limiting embodiment of the present invention, in step 1-1-3, a direct linear transformation algorithm is applied to convert the image coordinates of the corner points of the grayscale image into real world coordinates, and the imaging model of the high-definition camera is defined as:

[0099]

[0100] in, is the image coordinate of the corner point of the grayscale image, is the actual world coordinate of the corner point of the grayscale image, is the focal length of the HD camera, is the principal point coordinate of the HD camera, and They are the distortion component on the X-axis of the image coordinate and the distortion component on the Y-axis of the image coordinate, respectively.

[0101] Step 1-2: Joint calibration of 2D HD camera and 3D lidar.

[0102] In a preferred but non-limiting embodiment of the present invention, steps 1-2 specifically include:

[0103] To ensure high-precision measurement, the present invention adopts a joint calibration method of 2D high-definition camera and 3D laser radar to solve the following detection difficulties: 1. Eliminate the errors of 2D high-definition camera and 3D laser radar themselves; 2. Determine the relative distance between the camera and radar, that is, the fixed RT matrix of relative posture; 3. Determine the corresponding distance between camera pixels and radar laser spot center, 2D\3D full image calibration; 4. Improve robustness, that is, in complex lighting conditions, the stability of the system can still be guaranteed; its core function is to determine the relative posture between the camera and radar, that is, determine the fixed RT matrix of the relative posture of the camera and radar. The calibration steps are:

[0104] Step 1-2-1: Fix the HD camera and laser radar to make their positions relatively stable;

[0105] Step 1-2-2: Fix a specific stereo calibration plate. The stereo calibration plate is divided into Figure 3 The four sets of images shown in the figure each contain an Aurco mark, whose length, width and position of the stereo calibration plate are all fixed parameters; .aurcomark is the Aurco mark.

[0106] Step 1-2-3: Use a 2D HD camera and a 3D LiDAR to collect 2D HD images and 3D LiDAR data respectively and transmit them to the mainboard's CPU. The 2D HD image includes 2D information of four sets of image corner points, and the 3D LiDAR data includes 3D corner point information of four sets of images.

[0107] Step 1-2-4: Since the coordinate system of the laser radar is different from the visual coordinate system of the HD camera (the laser radar is x-axis forward, and the HD camera is z-axis forward), the CPU of the motherboard first converts the coordinate system of the laser radar to the visual coordinate system of the HD camera;

[0108] In a preferred but non-limiting embodiment of the present invention, steps 1-2-4 specifically include:

[0109] Step 1-2-4-1: Label 2D high-definition images and 3D lidar data;

[0110] Step 1-2-4-2: Select a suitable deep learning architecture, which can be a convolutional neural network (CNN) or a recurrent neural network (RNN).

[0111] Step 1-2-4-3: Apply a deep learning architecture to learn the nonlinear transformation relationship between 2D HD images and 3D LiDAR data to obtain a trained model;

[0112] Steps 1-2-4-1 to 1-2-4-3 can automatically learn complex conversion patterns without relying on traditional parameters.

[0113] Step 1-2-4-4: Input the new HD camera image and lidar data into the trained model to obtain the conversion result between the lidar coordinate system and the HD camera visual coordinate system;

[0114] Step 1-2-4-5: Filter out the 3D lidar data except the aurco mark in the 3D lidar data to obtain the aurco point cloud data;

[0115] Steps 1-2-4-6: Project the ArUco point cloud data, that is, convert the ArUco point cloud data into the visual coordinate system of the HD camera, and use a polygon to select the ArUco point cloud data converted to the visual coordinate system of the HD camera;

[0116] Steps 1-2-4-7: Obtain the corner points of the polygon and record the coordinates of the corner points on the 2D HD image and the coordinates of the 3D LiDAR data;

[0117] Steps 1-2-4-8: Obtain the R|T matrix from the HD camera to the LiDAR based on the coordinates of the corner points on the 2D HD image and the coordinates of the 3D LiDAR data. The R|T matrix is ​​the rotation matrix R and translation vector T in the external parameters.

[0118] In a preferred but non-limiting embodiment of the present invention, in steps 1-2-4-8, the coordinates of the corner point 3D lidar data are defined as , the coordinates of the corner points on the 2D high-definition image are , the extrinsic rotation matrix is , the translation vector is , and Obtained by the following formula:

[0119] .

[0120] In summary, combined with the selection of hardware in the present invention, that is, a high-definition camera with 50 million pixels, FOV: 80 degrees, shooting at a distance of about 3 meters from the target, the actual distance of 1.5 pixels is about 1 mm, and the point cloud is about 200,000-700,000 pixels. When the calibration pixel error between the camera image and the point cloud projection image is less than 5 pixels, the actual detection accuracy can be guaranteed to be about 3 mm. If the accuracy needs to be further improved, it is necessary to reduce the FOV index, that is, reduce the shooting field of view or increase the camera resolution, but this operation will increase the number of image scans or the analysis time, so it is necessary to select appropriate parameters according to actual needs. The above parameters are all theoretical derivations. In actual conditions, the data can be further corrected by fitting the error function or by taking multiple groups of shots to find the optimal solution to improve the analysis accuracy.

[0121] In a preferred but non-limiting embodiment of the present invention, step 2 specifically comprises:

[0122] This method uses AI recognition of embedded components to roughly locate their four corners and obtains rough distance parameters (with a distance error of approximately 3mm) using LiDAR. The high-definition camera and LiDAR are calibrated in advance, and the registration information includes both RGB and depth information. A 40*40 pixel area in the image is analyzed with the roughly detected corner location as the center. The error cannot exceed 2*2 pixels, meaning that the measured value of any 4*4 pixel area cannot deviate from the specified range, and the error within the 40*40 pixel area cannot exceed 5%.

[0123] Analysis step 2 is as follows. Based on the calibration of the HD camera and LiDAR, it includes:

[0124] Step 2-1: Use high-definition cameras and lidar to collect high-definition images and lidar data of nuclear power embedded parts respectively;

[0125] Step 2-2: Pre-processing high-definition images of nuclear power embedded parts;

[0126] The collected raw image data is first preprocessed. Including: 1. Image color space conversion. Since the image collected from the device is in BGR format, we set it to RGB888 format during AI training and inference, so this conversion needs to be performed first. 2. Image downsampling. Since the camera pixel is as high as 65 million pixels, the image size needs to be reduced before inference; 3. Normalization based on mean subtraction, adjust the mean and variance of the input image to adapt it to the input requirements of the deep learning model, thereby improving the training and prediction effects of the model; 4. Image expansion. Since the image resolution used in AI training is 640*640, and the image ratio collected by the device does not match it, the image needs to be completed and filled. By copying and expanding the image boundary, the image is prevented from losing information due to its small size during convolution operations. It can make the image larger by copying the image boundary, thereby increasing the accuracy of feature extraction;

[0127] Step 2-2: Identify nuclear power embedded parts in high-definition images.

[0128] Data Preparation: The dataset is divided into training and validation sets. Each image has a corresponding label file in the YOLO format: each line represents an object and contains [class_id, x_center, y_center, width, height]. The coordinate values ​​are normalized relative to the image width and height.

[0129] Select the pre-trained model yolov5s.pt, specify the dataset configuration file, model architecture, training cycle and other parameters, and perform pre-embedded parts training.

[0130] Convert the PT model to an ONNX model, and then convert the ONNX model to a PT target model.

[0131] In a preferred but non-limiting embodiment of the present invention, step 2 specifically further comprises:

[0132] Edge defect detection uses the Hough transform to detect rectangular boxes in an image. Compared to traditional line detection methods, rectangle detection is more complex because a rectangle is not just composed of straight lines but also requires consideration of multiple factors such as angle, width, and height. Edge detection algorithms are used to extract edges from the image, which serve as the basis for rectangle detection. Since a rectangle is composed of four straight lines, the traditional Hough transform can be used to detect all the straight lines in the image. This method converts each edge line in the image into a point in parameter space and uses a voting mechanism to find the most likely line. By combining several lines, candidate rectangles may be obtained. To form a rectangle, rectangles are screened based on line constraints (such as perpendicularity between lines and the formation of a closed rectangle). The algorithm maps each detected rectangle into a parameter space, where each corner of the rectangle (or, in other words, the width, height, and orientation of the rectangle) is represented by a set of parameters. Voting across these parameter spaces finds the optimal rectangle. Candidates that do not meet the requirements of a rectangle are filtered out using geometric criteria (such as width, height, angle, and scale). Typically, some thresholds or rules are used to ensure that the selected rectangle conforms to the actual rectangular structure in the image. Based on the rectangle, the precise corner coordinates are detected through sub-pixel detection.

[0133] In a preferred but non-limiting embodiment of the present invention, step 3 specifically comprises:

[0134] Step 3-1: Extract the contour depth data within the above-mentioned corner point range from the laser radar point cloud data;

[0135] Step 3-2: Calculate its physical size through 3D affine.

[0136] Compared with the prior art, the present invention has the following advantages and technical effects:

[0137] High-precision detection capabilities:

[0138] By combining the powerful hardware performance of the HiSilicon SS928 chip with the combined application of a high-definition white light camera (50 megapixels) and a lidar, the present invention can achieve high-precision detection of embedded parts within a range of 3 meters, achieving a measurement error range of ±1mm.

[0139] Based on precise 2D3D calibration and calibration algorithms, a mapping relationship matrix between 3D point clouds and visual image pixels is constructed, effectively reducing the impact of hardware errors and ensuring that the size and position detection accuracy meets engineering requirements.

[0140] Perfect multimodal fusion:

[0141] By performing multimodal fusion of the 2D visual data of the white light camera and the 3D point cloud data of the lidar, the shortcomings of a single sensor in accuracy and scene adaptability are compensated by using 2D3D stitching technology.

[0142] The fused data is processed by AI algorithms to achieve higher-resolution edge detection and corner positioning, providing reliable data support for subsequent size, position and edge defect detection.

[0143] Intelligent analysis and error compensation:

[0144] Leveraging AI recognition technology, key feature points of embedded components are automatically located. Through detailed analysis of a 40×40 pixel area and AI learning, detection accuracy is further improved, achieving sub-pixel precision. A feature enhancement algorithm, insensitive to lighting, angle, and distance, is designed to adapt to detection needs in a variety of complex scenarios and significantly reduce the impact of errors. Dynamic correction of the registration parameters calibrated for the LiDAR and camera further optimizes detection accuracy, keeping errors within a 2×2 pixel range.

[0145] Flexible expansion and efficient application:

[0146] The platform is highly modular in design. The hardware layer supports expansion of various input and output devices (such as USB and HDMI), and the software layer features a rich set of algorithms (such as image enhancement, point cloud stitching, and data comparison), allowing for flexible adaptation to various inspection scenarios and requirements. The addition of calibration plates with specific features further improves inspection accuracy, enhancing the device's applicability in complex engineering environments.

[0147] Efficient detection driven by algorithms:

[0148] The present invention realizes extrapolation and error compensation based on hardware with insufficient precision through AI algorithm.

[0149] By leveraging proprietary super-resolution and pixel motion algorithms, we integrate multi-frame image data for high-precision analysis, addressing the shortcomings of traditional inspection equipment in high-error scenarios. Intelligent algorithms drive the precision inspection process, from feature extraction to data calibration, significantly improving inspection efficiency and accuracy.

[0150] Strong adaptability and high robustness:

[0151] By learning and enhancing data from varying lighting, angles, and distances, the device can operate stably in complex construction environments, with far greater robustness than traditional methods. The design of a specialized calibration plate further enhances the system's adaptability to detection targets, ensuring accurate detection even on complex or damaged surfaces.

[0152] In addition, the present invention proposes a 2D-3D calibration and calibration technology based on a special checkerboard calibration plate. Through feature point extraction, homography matrix calculation and maximum likelihood estimation, a mapping relationship matrix between the laser point cloud and the visual image pixels is constructed to achieve high-precision alignment of the white light camera and the lidar. This is one of the core technologies of the present invention, which directly affects the accuracy of size detection, position detection and edge defect detection. It combines the depth information of the laser point cloud with the high resolution of the visual image to form the basis for multimodal data fusion. The present invention also proposes a size detection and position detection method based on image and point cloud fusion, which specifically includes:

[0153] Image edge and contour extraction;

[0154] Matching of point cloud depth data and image contours;

[0155] Calculate physical size and position through 3D affine;

[0156] Compare precisely the dimensions and position data with the master part.

[0157] By combining the advantages of image and point cloud data, the bottleneck of insufficient accuracy of a single sensor is broken through, and millimeter-level measurement accuracy is achieved to meet the needs of industrial inspection in complex scenarios.

[0158] Based on artificial intelligence technology, this invention uses AI to identify the edge features of embedded parts and locate corner points. Based on rough detection results, it improves the robustness and accuracy of edge detection through detailed analysis and manual labeling of a 40×40 pixel area.

[0159] A feature enhancement algorithm is designed to improve detection capabilities under complex lighting, angle and distance conditions.

[0160] Provides super-resolution and pixel motion algorithms to further reduce detection errors to sub-pixel level.

[0161] AI technology is used to overcome the limitations of hardware accuracy, enabling detection equipment to have self-learning capabilities, thereby improving detection accuracy and adaptability.

[0162] The present invention achieves multimodal data splicing and comprehensive analysis by fusing 2D images and 3D point cloud data. Specifically, it includes:

[0163] Registration and mapping of 2D and 3D data;

[0164] Compare the three-dimensional structures of the detection targets based on BIM data.

[0165] The present invention solves the problem of integrating point cloud data and image data in space and dimension, and provides complete technical support for detection in multi-view and complex scenes.

[0166] like Figure 2 As shown, the present invention provides a machine vision-based intelligent detection platform for nuclear power embedded parts installation, comprising:

[0167] The display screen, HD camera, and LiDAR are connected to the motherboard's central processing unit. Specifically, the 2D HD camera and 3D LiDAR are integrated into a machine vision-based intelligent inspection platform for nuclear power embedded components and calibrated with high precision. The 2D camera is then used to identify nuclear power embedded components in real time. The calibrated machine vision-based intelligent inspection platform, which includes both the 2D HD camera and 3D LiDAR, verifies the size and position of all nuclear power embedded components. Multiple images and inspections are performed on the building's wall surface where the embedded components are located. The inspection data is then stitched together to create complete nuclear power embedded component installation data for the wall surface. This data is then compared with the BIM data of the building where the embedded components are located to quickly identify abnormally installed nuclear power embedded components. This method enables efficient and accurate inspection of nuclear power embedded components within a building and allows for rapid integration into the digital system.

[0168] The modules running on the motherboard's CPU include:

[0169] Calibration module, which is used to perform high-precision joint calibration of high-definition cameras and lidar;

[0170] The detection module is used to identify nuclear power embedded parts in real time using a high-definition camera and detect the size and position of all nuclear power embedded parts using a calibrated high-definition camera and lidar;

[0171] The splicing module is used to use a high-definition camera to take multiple photos and detect the wall surface of the building where the nuclear power embedded parts are located, and then splice the detection data to form the nuclear power embedded parts installation data of the complete wall surface;

[0172] The comparison module is used to compare with the BIM data of the building where the nuclear power embedded parts are located, and quickly identify abnormally installed nuclear power embedded parts.

[0173] The motherboard is an industrial computer or an embedded motherboard based on the HiSilicon SS928 smart chip. The central processing unit (CPU) of the industrial computer and the CPU of the embedded motherboard serve as the motherboard's central processing unit. The display is a 4-7-inch screen, with the specific size determined by the final device size and viewability. It features an LCD display and supports an HDMI interface. The motherboard's CPU is also connected to function buttons, with 4-6 reserved for functions such as power on / off, menu, selection, and analysis start. The HD camera uses a white light camera: the prototype uses the Hikvision MV-CH500-90TM, and the production model can directly integrate the module into the platform. The embedded motherboard ensures integration and expansion. The prototype uses the Livox Avia LiDAR sensor; the production model can directly integrate the module into the system. The embedded motherboard ensures integration and expansion. The motherboard's CPU is also connected to a solid-state drive: a Kingston 1TB USB 2.0 Type A TMAXA hard drive is acceptable. The voltage regulator is a Uni-T UTP3305-II power supply. The external interface of the motherboard supports 2 USB ports, 1 network port, and 1 HDMI port.

[0174] The software layer running on the motherboard's central processing unit can be:

[0175] Framework layer: includes white light camera driver and basic image acquisition function modules; laser scanner driver and basic point cloud acquisition function modules; image algorithms include image / point cloud calibration, image enhancement, image recognition, edge detection, depth of field data calculation and analysis, image stitching, point cloud stitching, comparison of point cloud target data and BIM data, etc.

[0176] Task layer: includes various application and system management functions, such as configuration management, task management, communication management, image analysis tasks, data storage / reading and writing, data export, etc.

[0177] The beneficial effects of the present invention are as follows:

[0178] By employing advanced machine vision technologies, such as high-precision visual sensors (panoramic cameras, high-pixel industrial cameras, and LiDAR), precise object detection algorithms (based on the YOLO framework and proprietary algorithms), and automated data processing and visualization systems, we can effectively address the challenges inherent in traditional instrument-based inspections. Visual sensors can quickly and accurately capture image information of embedded components, unaffected by environmental factors and human interaction. Object detection algorithms automatically analyze images for highly accurate object identification, location, and dimensional measurement. Automated data processing and visualization systems provide real-time feedback on inspection results, improving efficiency and accuracy while enhancing visualization. This provides a more reliable and efficient solution for quality inspection of embedded component installations in nuclear power plants.

[0179] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the above embodiments, ordinary technicians in the relevant field should understand that the specific implementation methods of the present invention can still be modified or replaced by equivalents. Any modification or equivalent replacement that does not deviate from the spirit and scope of the present invention should be covered within the protection space of the claims of the present invention.

Claims

1. A machine vision-based intelligent detection method for nuclear power embedded parts installation, characterized in that: include: Step 1: Perform high-precision joint calibration of the HD camera and LiDAR; Step 2: Use high-definition cameras to identify nuclear power embedded parts in real time, and use calibrated high-definition cameras and lidar to detect the size and position of all nuclear power embedded parts; Step 3: Use a high-definition camera to take multiple photos and inspect the wall surface of the building where the nuclear power embedded parts are located, and splice the inspection data to form the nuclear power embedded parts installation data of the complete wall surface; Step 4: Compare the data with the BIM data of the building where the nuclear power embedded parts are located to identify abnormally installed nuclear power embedded parts.

2. The intelligent detection method for nuclear power embedded parts installation based on machine vision according to claim 1 is characterized in that: Step 1 specifically includes: Step 1-1: HD camera calibration; Step 1-2: Joint calibration of HD camera and lidar.

3. The intelligent detection method for nuclear power embedded parts installation based on machine vision according to claim 2 is characterized in that: Step 1-1 specifically includes: Step 1-1-1: Use a high-definition camera to obtain a grayscale image of the checkerboard image and transmit it to the mainboard's CPU; Step 1-1-2: The CPU of the motherboard searches for the corner points of the grayscale image of the checkerboard image; Step 1-1-3: Perform HD camera calibration based on the corner points of the grayscale image of the checkerboard image. HD camera calibration is to estimate the HD camera internal parameters, which include the focal length, principal point coordinates, and distortion coefficient of the HD camera.

4. The intelligent detection method for nuclear power embedded parts installation based on machine vision according to claim 3 is characterized in that: In step 1-1-1, the lens of the high-definition camera is panned left and right and tilted up and down to capture multiple sets of grayscale images of the checkerboard image, so that the checkerboard in the grayscale image of each checkerboard image occupies no less than two-thirds of the pixels in the picture; In step 1-1-2, the CPU of the motherboard searches for corner points of the grayscale image of the grid image using the Harris corner detection algorithm or the Shi-Tomasi corner detection algorithm. In step 1-1-3, the direct linear transformation algorithm is applied to convert the image coordinates of the corner points of the grayscale image into real world coordinates, and the imaging model of the high-definition camera is defined as: ； in, is the image coordinate of the corner point of the grayscale image, is the actual world coordinate of the corner point of the grayscale image, is the focal length of the HD camera, is the principal point coordinate of the HD camera, and They are the distortion component on the X-axis of the image coordinate and the distortion component on the Y-axis of the image coordinate, respectively.

5. The intelligent detection method for nuclear power embedded parts installation based on machine vision according to claim 4 is characterized in that: Steps 1-2 specifically include: Step 1-2-1: Fix the HD camera and LiDAR to make their positions relatively stable; Step 1-2-2: Fix a specific stereo calibration plate. The stereo calibration plate is divided into 4 groups of images. Each group of images contains an aurco mark. The length, width and position of the stereo calibration plate are all fixed parameters. Step 1-2-3: Use a high-definition camera and a lidar to collect 2D high-definition images and 3D lidar data respectively and transmit them to the mainboard's central processor. The 2D high-definition image includes 2D information of four sets of image corner points, and the 3D lidar data includes 3D corner point information of four sets of images. Step 1-2-4: The mainboard's CPU converts the laser radar's coordinate system to the HD camera's visual coordinate system.

6. The intelligent detection method for nuclear power embedded parts installation based on machine vision according to claim 5 is characterized in that: Steps 1-2-4 specifically include: Step 1-2-4-1: Label 2D high-definition images and 3D lidar data; Step 1-2-4-2: Select a suitable deep learning architecture, which can be a convolutional neural network or a recurrent neural network. Step 1-2-4-3: Apply a deep learning architecture to learn the nonlinear transformation relationship between 2D HD images and 3D LiDAR data to obtain a trained model; Step 1-2-4-4: Input the new HD camera image and lidar data into the trained model to obtain the conversion result between the lidar coordinate system and the HD camera visual coordinate system; Step 1-2-4-5: Filter out the 3D lidar data except the aurco mark in the 3D lidar data to obtain the aurco point cloud data; Steps 1-2-4-6: Project the ArUco point cloud data, that is, convert the ArUco point cloud data into the visual coordinate system of the HD camera, and use a polygon to select the ArUco point cloud data converted to the visual coordinate system of the HD camera; Steps 1-2-4-7: Obtain the corner points of the polygon and record the coordinates of the corner points on the 2D HD image and the coordinates of the 3D LiDAR data; Step 1-2-4-8: Obtain the R|T matrix from the HD camera to the LiDAR based on the coordinates of the corner points on the 2D HD image and the coordinates of the 3D LiDAR data. The R|T matrix is ​​the rotation matrix R and translation vector T in the external parameters. In steps 1-2-4-8, define the coordinates of the corner point 3D lidar data as , the coordinates of the corner points on the 2D high-definition image are , the extrinsic rotation matrix is , the translation vector is , and Obtained by the following formula: 。 7. The intelligent detection method for nuclear power embedded parts installation based on machine vision according to claim 6 is characterized in that: Step 2 specifically includes: Step 2-1: Use high-definition cameras and lidar to collect high-definition images and lidar data of nuclear power embedded parts respectively; Step 2-2: Pre-processing high-definition images of nuclear power embedded parts; Step 2-2: Identify nuclear power embedded parts in high-definition images.

8. The method for intelligent detection of nuclear power embedded parts installation based on machine vision according to claim 7 is characterized in that: Step 2 specifically also includes: Edge defect detection uses the Hough transform to detect rectangular boxes in an image. Edge detection algorithms extract edges within the image, which serve as the basis for rectangle detection. The Hough transform converts each edge line in the image into a point in parameter space, and a voting mechanism is used to find the most likely line. By combining several lines, candidate rectangles are generated. The algorithm maps each detected rectangle into a parameter space, where each corner of the rectangle is represented by a set of parameters. Voting across these parameter spaces finds the optimal rectangle. Based on the rectangles, sub-pixel detection is used to accurately determine the coordinates of the corner points.

9. The method for intelligent detection of nuclear power embedded parts installation based on machine vision according to claim 8, characterized in that: Step 3 specifically includes: Step 3-1: Extract the contour depth data within the corner point range from the laser radar point cloud data; Step 3-2: Calculate its physical size through 3D affine.

10. A machine vision-based intelligent detection platform for nuclear power embedded parts installation, characterized in that: include: Display screen, HD camera and LiDAR connected to the motherboard’s CPU; The modules running on the motherboard's CPU include: Calibration module, which is used to perform high-precision joint calibration of high-definition cameras and lidar; The detection module is used to identify nuclear power embedded parts in real time using a high-definition camera and detect the size and position of all nuclear power embedded parts using a calibrated high-definition camera and lidar; The splicing module is used to use a high-definition camera to take multiple photos and detect the wall surface of the building where the nuclear power embedded parts are located, and then splice the detection data to form the nuclear power embedded parts installation data of the complete wall surface; The comparison module is used to compare with the BIM data of the building where the nuclear power embedded parts are located, and identify abnormally installed nuclear power embedded parts.

Images