A virtual adapter-based pipeline reconstruction and detection method

Abstract

The application belongs to the technical field of three-dimensional measurement and digital reconstruction, and specifically discloses a pipeline reconstruction and detection method based on a virtual adapter, which comprises the following steps: a multi-view vision system collects multi-angle images of a pipeline to be detected, reconstructs the pipeline and determines a bending point; the pipeline is aligned and matched with the bending point, and an initial conversion matrix of a measurement and digital model coordinate system is obtained based on an SVD decomposition method; a digital model pipeline accessory is projected to a camera image, and an adjustment matrix is obtained by optimization of minimum pixel sum and maximum gradient; a threshold is set to screen edge points, and a final conversion matrix is obtained by optimization of maximum gradient after projection; the accessory is converted to the measurement coordinate system, and is spliced with the reconstructed pipeline to form a complete model, and the bending point and the accessory feature deviation are compared. The application reconstructs the pipeline with complex accessories such as flanges and valves through initial calibration of coordinate conversion, projection optimization and edge point fine adjustment, restores the pipeline and the accessory features completely, and improves the pipeline detection precision.

Description

TECHNICAL FIELD

[0001] The present application belongs to the technical field of three-dimensional measurement and digital reconstruction, and particularly relates to a pipeline reconstruction and detection method based on a virtual adapter. BACKGROUND

[0002] Pipeline accessories are key components in the pipeline system in addition to straight pipe sections, including flanges, valves, clamps, etc., which are crucial to ensuring the structural integrity, intelligent response and safe operation of the system. Among them, flanges realize modular assembly and sealing, and valves control flow and safety, and their accurate measurement has become a core requirement to ensure system operation.

[0003] Current main measurement methods include: three-coordinate equipment contact measurement, which is suitable for detachable complex accessories, has high precision but high operation requirements and long cycle; a structured light scanner can quickly obtain point clouds of complex accessories, but is affected by environmental light, large accessories need to be spliced multiple times and errors are accumulated; a laser scanner is suitable for on-site measurement of large accessories, has a wide range but takes time to scan and splice, and it is difficult to capture narrow spaces; a multi-view vision measurement system is suitable for large pipelines and can quickly reconstruct data, but is limited to fixed geometric pipeline appearances.

[0004] The existing technology has significant bottlenecks: a measurement blind area is formed due to the limitation of the camera view angle, resulting in incomplete feature information and a long measurement cycle; accumulated errors are generated due to light differences in multi-position shooting and splicing; it is difficult to completely restore the features of pipeline accessories; the multi-view vision system cannot adapt to pipelines containing complex accessories, and a targeted measurement method is urgently needed. SUMMARY

[0005] The purpose of the present application is to overcome the defects in the prior art and provide a pipeline reconstruction and detection method based on a virtual adapter.

[0006] The present application provides a pipeline reconstruction and detection method based on a virtual adapter, comprising the following steps:

[0007] S1, acquiring multi-angle images of the pipeline to be measured by a multi-view vision system, reconstructing the pipeline to be measured and determining the bending point data of the pipeline to be measured;

[0008] S2, determining the correspondence between the bending points of the pipeline to be measured and the bending points of the corresponding numerical model of the pipeline to be measured through pipeline alignment, and calculating the initial conversion matrix from the measurement coordinate system to the numerical model coordinate system based on the SVD decomposition method ;

[0009] S3, projecting the pipeline accessories in the numerical model to the image planes of each camera through the initial conversion matrix and camera parameters, and iteratively optimizing the objective function with the goal of minimizing the pixel sum of the projection points and maximizing the gradient sum, to obtain the adjustment matrix ;

[0010] S4. Set the threshold range and adjust the matrix accordingly. Edge points of pipe fittings that meet the threshold condition are selected, and these edge points are projected onto the image plane to optimize the adjustment matrix by maximizing the gradient sum of the edge points as the objective function. The final transformation matrix is ​​obtained. ;

[0011] S5. Connect the piping accessories in the digital model via... The model is then converted to a measurement coordinate system and spliced ​​with the reconstructed pipeline model from step S1 to form a complete pipeline model. The deviations in the bending point positions and the characteristic deviations of the pipeline accessories are compared between the pipeline model and the digital model.

[0012] A further solution is that, in step S1, the multi-view vision system needs to be calibrated before acquiring images. After calibration, the pipeline to be tested is placed in the measurement area, and the calibrated multi-view vision system is used to photograph the pipeline, obtaining images of the pipeline from all camera perspectives. The calibration process of the multi-view vision system is as follows:

[0013] A calibration plate matching the measurement area is used, the position and pose of the calibration plate are adjusted, and multiple sets of images are captured through a multi-view vision system;

[0014] Based on the principle of close-range photogrammetry, the intrinsic and extrinsic parameters of each camera are solved.

[0015] The intrinsic parameters include: lens focal length. Camera principal point deviation ( ), camera lens distortion deviation ( );

[0016] The extrinsic parameters include: the rotation matrix from the measurement coordinate system to the camera coordinate system. Translation matrix .

[0017] A further solution is that, in step S2, the process of analyzing the bending points of the pipeline under test is as follows:

[0018] The reconstructed tubular model is sampled along the central axis, and the local curvature is calculated with a fixed step size;

[0019] Distinguish between straight segments and bent segments by using a curvature change threshold;

[0020] Fit the center axis of each straight line segment, and find the intersection point of the axes of adjacent straight line segments as the bending point;

[0021] The curvature change threshold is a preset threshold. Sections with curvature changes less than the curvature change threshold are determined to be straight segments; sections with curvature changes greater than the curvature change threshold are determined to be bent segments.

[0022] A further solution is that, in step S2, the initial transformation matrix... The acquisition process is as follows:

[0023] The connection structure between the bending points of the pipeline under test obtained by actual measurement is analyzed according to their actual spatial distribution order.

[0024] For the predefined bending points in the digital model, analyze their connection structure relationships at the design level according to their planning sequence in the design phase;

[0025] By comparing the two connection structures, and on the basis that the connection structures of the two are consistent, the set of bending points of the pipeline under test is matched with the set of bending points of the digital model;

[0026] Based on the existing bend points of the pipeline under test and the corresponding bend points of the digital model, the SVD decomposition method is applied to determine the initial transformation matrix between the measurement coordinate system of the pipeline under test and the digital model coordinate system. The initial transformation matrix satisfy

[0027] ;

[0028] in, This is the set of bends in the pipeline under test. The set of bending points of the digital model; This is the rotation matrix from the digital model coordinate system to the measurement coordinate system. This is the translation matrix from the numerical model coordinate system to the measurement coordinate system.

[0029] A further solution is that, in step S3, the matrix is ​​adjusted. The calculation process is as follows:

[0030] Digital model piping accessory point set Through the initial transformation matrix Projected onto the camera image coordinate system:

[0031] ;

[0032] in, yes Image point coordinates from the digital model coordinate system to the two-dimensional image coordinate system; This indicates the specific transformation and projection matrix relationships. The included formula is expressed as follows:

[0033] ;

[0034] in, The set of point coordinates representing the piping accessories in the measurement coordinate system of the digital model;

[0035] The pipeline accessory is projected from the measurement coordinate system to the two-dimensional image coordinate system of the camera, and the projection process is expressed as:

[0036] ;

[0037] Wherein, is the position coordinate of a certain point on the pipeline accessory in the measurement coordinate system; is the point The actual image point coordinate projected from the measurement coordinate system to the two-dimensional image coordinate system, , , is the coordinate of the camera optical center in the measurement coordinate system, , is expressed as:

[0038] ;

[0039] Wherein, , , are all rotation components, used to describe the rotation relationship between the two coordinate systems, , , is the coordinate of the camera optical center in the measurement coordinate system, used to describe the translation relationship between the two coordinate systems;

[0040] The maximum gradient sum and the minimum pixel sum are taken as the objective function, and the iterative calculation is performed on the two-dimensional projection point set of all cameras The matrix conversion relationship from the digital model coordinate system to the measurement coordinate system is adjusted to obtain .

[0041] Further, in step S3, the construction process of the objective function is:

[0042] The coordinate point set of the pipeline accessory in the image coordinate system of a certain camera is: ;

[0043] The pixel sum and the gradient sum of the projection point set in each camera image are calculated respectively:

[0044] ;

[0045] The global objective function is constructed by combining the pixel sum and the gradient sum:

[0046] ;

[0047] In the formula, represents the image gradient amplitude in the camera image;

[0048] representing image pixel values in the camera image;

[0049] representing the number of image points of the tube attachment in the current camera image;

[0050] representing the number of all cameras in the vision system, .

[0051] Further, for each two-dimensional image point on the tube attachment, the image gradient is represented as:

[0052] ;

[0053] ;

[0054] wherein represents the gradient of the pixel value of the image in the horizontal direction; represents the gradient of the pixel value of the image in the height direction.

[0055] Further, in the step S4, the process of optimizing the adjustment matrix is as follows:

[0056] Based on the adjustment matrix , the point set of the tube attachment of the digital model is converted to the measurement coordinate system:

[0057] ;

[0058] wherein represents the point coordinate set of the tube attachment of the digital model in the measurement coordinate system;

[0059] By applying the matrix conversion relationship , the normal vector set of each point on the tube attachment of the digital model in the digital model coordinate system is converted to the measurement coordinate system, obtaining the corresponding normal vector set , and the expression is as follows:

[0060] ;

[0061] For each camera in the vision system, a light ray is constructed by connecting the optical center and a point in the image ;

[0062] The light ray​​ an angle between a corresponding normal vector ;

[0063] Setting an angle threshold range , , retaining points satisfying the angle in the range, and constructing an edge point candidate set ;

[0064] Projecting points in the edge point candidate set to the two-dimensional image plane of the corresponding camera to obtain a two-dimensional image point set ;

[0065] Setting a pixel threshold , filtering points with pixel values lower than in the two-dimensional image point set to obtain an effective edge point set ;

[0066] For each camera of the multi-view vision system, calculating the image gradient sum of the projected points in the effective edge point set , and iteratively optimizing the adjustment matrix with the maximum image gradient sum as the objective function, and outputting the final conversion matrix .

[0067] Further, the step S5 comprises:

[0068] Converting the pipeline accessory point cloud in the digital model to the measurement coordinate system through the final conversion matrix ;

[0069] Spatially splicing the converted pipeline accessory point cloud with the reconstructed pipeline point cloud to be measured in step S1 to form a complete pipeline three-dimensional model containing the pipeline accessories and the pipeline body;

[0070] Based on the complete pipeline three-dimensional model, the following detections are performed:

[0071] Comparing the spatial position deviation of the bending points of the pipeline to be measured with the bending points of the digital model;

[0072] Comparing the key feature size deviation of the pipeline accessories, including at least one of the flange hole position angle deviation, the valve mounting surface position deviation, or the clamp center distance deviation.

[0073] Compared with the prior art, the beneficial effects of the present application are:

[0074] ​The present application reconstructs the pipe body data through a multi-view vision system, combines digital model projection relationship optimization and edge point fine adjustment, breaks through the camera view angle limitation to eliminate the measurement blind area, does not need to paste mark points on the pipe surface, sprays matt paint and the like pretreatment, reduces the number of station conversion splicing, simultaneously through initial calibration of coordinate conversion, projection optimization, edge point fine adjustment to adapt to the pipe with flanges, valves and the like complex accessories, restores the pipe and accessory features completely, and improves the pipe detection precision.

[0075] The present application realizes the initial alignment of the bending point through the SVD decomposition method, combines the projection point gradient and the pixel and target function iterative optimization coordinate conversion relationship, and then through the edge point screening and gradient and maximum fine adjustment, effectively eliminates the splicing cumulative error caused by light and shadow changes, makes the pipe bending point position deviation and accessory feature deviation detection more accurate, and further ensures the reliability of the complete measurement data, and provides a high-precision basis for subsequent detection.

[0076] The present application omits the surface pretreatment step, reduces the number of station conversion splicing, ensures the complete measurement while improving the operation efficiency, breaks through the limitation that the existing multi-view vision system is only applicable to the cross-section symmetrical pipe, can effectively process the pipe with flange hole sites, valve installation surfaces, clamp center distances and the like complex features, converts the pipe accessories from the digital model coordinate system to the measurement coordinate system through the final conversion matrix, splices the pipe body data to form a complete three-dimensional model containing the pipe body and accessories, ensures the consistency of the measurement data in space, provides complete data support for the accurate detection of the bending point position deviation and accessory feature size deviation, and makes the complete measurement result more valuable. BRIEF DESCRIPTION OF DRAWINGS

[0077] The following drawings only schematically illustrate and explain the present application, and do not serve to limit the scope of the present application, in which:

[0078] Figure 1 : the reconstruction and detection flowchart of the present application;

[0079] Figure 2 : the digital model with part of pipe accessories (flanges);

[0080] Figure 3 : the camera imaging point residual error geometric model schematic diagram;

[0081] Figure 4 : the coordinate system projection relationship schematic diagram;

[0082] Figure 5 : the flange (pipe accessory) projected on the image of each camera;

[0083] Figure 6 : the camera image schematic diagram of the flange adjusted through the target function. DETAILED DESCRIPTION

[0084] In order to make the purpose, technical scheme, design method and advantages of the present application more clear, the present application is further described in detail below with specific examples in combination with the drawings. It should be understood that the specific examples described herein are only used to explain the present application and are not used to limit the present application.

[0085] As shown in the drawings, the present application provides a virtual adapter-based pipeline reconstruction and detection method, comprising the following steps: Figure 1 S1, a multi-view vision system is used to collect multi-angle images of the pipeline to be measured, reconstruct the pipeline to be measured and determine the bending point data of the pipeline to be measured; wherein the multi-view vision system is composed of at least two industrial cameras, which are fixed around the pipeline to be measured by a support to form a multi-view observation network; in the implementation of this step, the multi-view vision system is first calibrated to determine the internal and external parameters of the multi-view vision system, and a measurement coordinate system with the center of the shooting range of the multi-view vision system as the origin is established. The internal and external parameters of the multi-view vision system are important parameters for establishing the connection between the three-dimensional space coordinates and the two-dimensional image coordinates. Then, the calibrated vision system is used to shoot or scan the pipeline to be measured to obtain the image or point cloud data of the pipeline to be measured, reconstruct the pipe type data of the pipeline to be measured except for the pipeline accessories, and finally distinguish the straight line segment and the bending segment of the reconstructed pipe type model, and digitize the reconstructed pipeline to be measured to solve the intersection point of the adjacent straight line segments of the pipeline to be measured as the bending point;

[0086] S2, the correspondence between the bending points of the pipeline to be measured and the bending points of the corresponding numerical model of the pipeline to be measured is determined by pipeline alignment, and the initial conversion matrix from the measurement coordinate system to the numerical model coordinate system is calculated based on the SVD decomposition method

[0087] ; in the implementation of this step, the bending point data of the pipeline to be measured is first extracted, and the bending point data of the numerical model is analyzed; then, the correspondence between the bending points of the pipeline to be measured and the bending points of the numerical model is determined by pipeline alignment; finally, based on the existing bending points of the pipeline to be measured and the bending points of the corresponding numerical model, the SVD (singular value) decomposition method is applied to determine the initial conversion matrix from the measurement coordinate system where the pipeline to be measured is located to the numerical model coordinate system where the numerical model of the pipeline is located , wherein the projection relationship between the coordinate systems is as shown in the drawings ; Figure 4

[0088] S3, the pipeline accessories in the numerical model (for example, the numerical model of the flange as shown in the drawings) are projected to the image planes of each camera through the initial conversion matrix and the camera parameters, and the projection result is as shown in the drawings, for example, the image of the flange in each camera is displayed, and the target function is iteratively optimized to minimize the pixel sum of the projection points and to maximize the gradient sum to obtain the adjustment matrix Figure 2 Figure 5 , and the image of the adjusted flange in the camera is as​​​Figure 6 The plurality of images respectively correspond to different camera perspectives in the multi-view vision system, and the plurality of images are adjusted by the adjustment matrix The optimized flange projection images, each image corresponding to the observation perspective of a camera, and the plurality of images are adjusted by the adjustment matrix Figure 5 Compared with the prior art, Figure 6 The projection positions of the flanges in the images are more consistent with the actual image features after optimization by the minimum pixel sum, the maximum gradient, and the target function, so that the edges are clearer and more distinct from the background, and the accuracy of the coordinate conversion matrix is improved after optimization. Specifically, the matrix conversion relationship of the pipeline accessories in the digital model is established by the pipeline alignment , combined with the camera internal and external parameters determined in the calibration stage, and the matrix conversion relationship of the pipeline accessories in the digital model projected into each camera image is established. Then, the image point coordinates of the pipeline accessories in the digital model in all camera images are required as the target function of the maximum gradient sum and the minimum pixel sum, so as to adjust the matrix conversion relationship between the measurement coordinate system and the digital model coordinate system, and output the adjusted adjustment matrix ;

[0089] S4, set a threshold range, combine the adjustment matrix to screen the edge points of the pipeline accessories that meet the threshold condition, project the edge points to the image plane, and optimize the adjustment matrix to obtain the final conversion matrix ; specifically, set a threshold range, combine the existing adjustment matrix , and screen the edge points of the pipeline accessories that meet the threshold condition. Then, continue to apply the adjustment matrix , combined with the camera internal and external parameters determined in the calibration stage, to establish the matrix conversion relationship of the edge points of the pipeline accessories in the digital model projected into each camera image. It should be noted that before establishing the matrix conversion relationship, the edge points of the pipeline accessories screened need to be projected back to the camera image coordinate system first, and the pixel point threshold is set to filter and determine the edge projection points of the pipeline accessories. Finally, the image point coordinates of the edge points of the pipeline accessories in the digital model in all camera images are required as the target function of the maximum gradient sum, so as to adjust the matrix conversion relationship between the measurement coordinate system and the digital model coordinate system, and output the final conversion matrix .

[0090] S5, convert the pipeline accessories in the digital model to the measurement coordinate system by the final conversion matrix , and splice the reconstructed pipeline to be measured in step S1 to form a complete pipeline model, and compare the bending point position deviation and the pipeline accessory feature deviation of the pipeline to be measured with the digital model. Specifically, the pipeline accessory point cloud in the digital model is converted to the measurement coordinate system by the final conversion matrix Convert to measurement coordinate system; splice the converted pipeline accessory point cloud with the reconstructed pipeline point cloud in step S1 to form a complete pipeline three-dimensional model containing the pipeline accessories and the pipeline body; based on the complete pipeline three-dimensional model, the following detections are performed: comparing the spatial position deviation of the bend points of the pipeline to be measured and the bend points of the numerical model; comparing the key feature size deviation of the pipeline accessories, including at least one of the flange hole position angle deviation, the valve mounting surface position deviation, or the clamp center distance deviation.

[0091] In the above, the calibration method of the multi-view vision system comprises: using a calibration board matched with the measurement surface, continuously adjusting the pose of the calibration board, based on the close-range photogrammetry principle, the multi-view vision system captures the calibration board in different poses to solve the internal and external parameters of the multi-view vision system, and a measurement coordinate system with the center point of the calibration board as the origin is established. The internal parameters include: lens focal length , principal point deviation of the camera , distortion deviation of the camera lens ; the external parameters include: rotation matrix from the measurement coordinate system to the camera coordinate system . The method for solving the bend point comprises: sampling the reconstructed pipeline model along the center axis to calculate the local curvature at a fixed step length;

[0092] The straight line segment and the bend segment are distinguished by the curvature change threshold value;

[0093] The center axis of each straight line segment is fitted, and the intersection point of the axes of adjacent straight line segments is solved as the bend point;

[0094] The curvature change threshold value is a preset threshold value, and the segment with a curvature change less than the curvature change threshold value is determined as a straight line segment; the segment with a curvature change greater than the curvature change threshold value is determined as a bend segment.

[0095] In step S2, the process of determining the correspondence relationship between the bend points of the pipeline to be measured and the bend points of the corresponding numerical model of the pipeline to be measured through pipeline alignment is as follows: first, all the bend points of the pipeline to be measured are determined according to the pipeline digitization; at the same time, the numerical model corresponding to the pipeline to be measured is analyzed to determine all the bend points of the numerical model. Then, the correspondence relationship between all the bend points of the pipeline to be measured and the bend points of the numerical model is determined by applying the pipeline alignment method; finally, based on the existing bend points of the pipeline to be measured and the bend points of the corresponding numerical model, the SVD (singular value) decomposition method is applied to determine the initial conversion matrix between the measurement coordinate system in which the pipeline to be measured is located and the numerical model coordinate system in which the numerical model is located.

[0096] ;

[0097] wherein, , Respectively, the bending point set of the to-be-tested pipeline, the bending point set of the pipeline number module; 、 Respectively, the rotation matrix relationship and the translation matrix relationship of the number module coordinate system to the measurement coordinate system; The matrix conversion relationship of the number module coordinate system to the measurement coordinate system is represented.

[0098] In step S3, the initial conversion matrix is iteratively optimized with the minimum projection point pixel sum and the maximum gradient sum as the objective function The process is as follows:

[0099] Based on the initial conversion matrix of the pipeline alignment determination , combined with the camera internal and external parameters determined by calibration, the matrix conversion relationship of the pipeline accessory projection of the number module to each camera image is established, and the formula expression is as follows:

[0100] ;

[0101] Among them, is the point set on the pipeline accessory converted and projected from the number module coordinate system to the two-dimensional image coordinate system; The specific conversion and projection matrix relationship is represented, The actual formula expression is as follows:

[0102] ;

[0103] Among them, represents the point coordinate set of the pipeline accessory of the number module in the measurement coordinate system.

[0104] Combined with the camera internal and external parameters determined by calibration, the pipeline accessory is projected from the measurement coordinate system to the two-dimensional image coordinate system of the camera. The projection process is expressed as follows:

[0105] ;

[0106] Among them, is the position coordinate of a certain point on the pipeline accessory in the measurement coordinate system; as Figure 3 shown, is the actual image point coordinate of the point on the pipeline accessory projected from the measurement coordinate system to the two-dimensional image coordinate system, is the theoretical image point coordinate of the point on the pipeline accessory projected from the measurement coordinate system to the two-dimensional image coordinate system; is the image point deviation caused by the distortion of the camera lens; (the ) is the principal point deviation of the camera; is the focal length of the lens, which is also the principal point distance from the optical center of the lens to the image plane; , Let be the rotation matrix and translation matrix from the measurement coordinate system to the camera coordinate system, respectively, and:

[0107] ;

[0108] in, , ... These are all rotational components, used to describe the rotational relationship between two coordinate systems. , , The coordinates of the camera's optical center in the measurement coordinate system are used to describe the translation relationship between the two coordinate systems;

[0109] Then, using the maximum gradient sum and minimum pixel sum as objective functions, for all cameras... Iterative calculations are performed on the set of two-dimensional projection points to obtain the adjustment matrix from the digital model coordinate system to the measurement coordinate system. .

[0110] The objective function is expressed as follows:

[0111] The set of coordinate points of the pipeline accessories in the image coordinate system of a certain camera is as follows In the camera image, the sum of the image gradients corresponding to the image point coordinates of the pipeline accessory and the sum of the image pixels are calculated. Analysis shows that the matrix transformation relationship between the digital model coordinate system and the measurement coordinate system is optimal when all image points of the pipeline accessory satisfy the conditions of maximizing the sum of image gradients and minimizing the sum of pixels. Therefore, the following objective function is defined. :

[0112] ;

[0113] In the formula, Represents pixels and, Let the gradient sum be... express Image gradient magnitude in a camera image. express Image pixel values ​​in a camera image. This indicates the number of image points near the pipe in the current camera image. Apply all projection points of the pipeline accessories in the image coordinate system. The objective function is required to be present in all camera images. minimize, This represents the number of cameras in the vision system. , This represents the total number of cameras in the multi-view vision system. The optimization solution yields the adjustment matrix from the measurement coordinate system to the camera coordinate system. The image gradient represents the rate of change of pixel values ​​in the horizontal and vertical directions. For a given two-dimensional image point on a pipeline accessory, its image gradient It can be represented as:

[0114] ;

[0115] ;

[0116] In the formula, This represents the gradient of pixel values ​​in the horizontal direction of an image. This represents the gradient of pixel values ​​in the image along the height direction.

[0117] Considering that the central area of ​​pipeline accessories in actual engineering may have irregular structures, leading to adjustments in the matrix... Since it's impossible to completely transform the pipeline accessories from the measurement coordinate system to the digital model coordinate system, edge points of the pipeline accessories are extracted. Furthermore, the projection of these edge points back into the camera image must satisfy the maximum gradient sum, thereby adjusting the matrix transformation relationship between the measurement coordinate system and the digital model coordinate system. Specifically, in step S4, the final transformation matrix is ​​obtained. The process is as follows:

[0118] Set a threshold range and filter edge points of pipe fittings that meet the threshold conditions. Apply the initial optimized adjustment matrix. The set of points of the piping accessories in the digital model coordinate system When transformed to the measurement coordinate system, its expression is as follows:

[0119] ;

[0120] in, The set of point coordinates representing the piping accessories in the digital model's own digital model coordinate system; Represents the adjusted matrix The set of point coordinates of the pipeline accessories in the measurement coordinate system after conversion of the digital model. Simultaneously, an adjustment matrix is ​​applied. The set of normal vectors of each point on the pipeline accessory of the digital model in the digital model coordinate system. Transform to the measurement coordinate system to obtain the corresponding set of normal vectors. The expression is as follows:

[0121] ;

[0122] Select one camera in a multi-view vision system, and connect the camera's optical center position to the set of points. One point Construction point light , represents the number of space points of the pipeline accessories in the measurement coordinate system, . And the angle between the light and the corresponding normal vector is calculated , the upper and lower threshold values of the angle are set , , those who meet the threshold value are the edge points of the pipeline accessories under the current camera view; otherwise, they are non-edge region points of the pipeline accessories under the current camera view. The expression is as follows:

[0123] ;

[0124] In the formula, is the light vector corresponding to point in the measurement coordinate system; is the normal vector corresponding to point in the measurement coordinate system.

[0125] Then, the matrix conversion relationship is continuously applied, and the camera internal and external parameters determined in the calibration stage are combined to establish the matrix conversion relationship of the edge points of the pipeline accessories of the numerical model projected to each camera image. Before establishing the matrix conversion relationship from the numerical model coordinate system to the image coordinate system, the edge points of the pipeline accessories need to be removed twice. After the above removal process, the edge point set of the pipeline accessories in the measurement coordinate system that meets the requirements can be preliminarily determined. Considering that the pipeline accessories may have special-shaped features, the edge point set of the pipeline accessories in the measurement coordinate system obtained by removal is projected back to the two-dimensional camera image coordinate system to obtain the two-dimensional image point set . A pixel threshold value is set, and those whose pixel value exceeds the threshold value continue to be retained as edge region points; otherwise, they are determined as non-edge region points and are removed. The specific expression is as follows:

[0126] ;

[0127] After two rounds of removal, the edge point set of the pipeline accessories in the image coordinate system is determined. The matrix conversion relationship is established between the edge point set and the corresponding edge point set of the numerical model, that is:

[0128] ;

[0129] represents the matrix conversion relationship from the edge point numerical model coordinate system of the pipeline accessories to the measurement coordinate system and continues to project from the measurement coordinate system to the camera image coordinate system.

[0130] Then, using the maximum gradient sum and minimum pixel sum as objective functions, for all cameras... The set of two-dimensional projection points is iteratively calculated and adjusted to obtain the matrix transformation relationship from the digital model coordinate system to the measurement coordinate system. .

[0131] Finally, using the maximum gradient sum as the objective function, for all cameras... The set of two-dimensional projection points is iteratively calculated and adjusted to obtain the matrix transformation relationship from the digital model coordinate system to the measurement coordinate system. The specific solution process is as follows:

[0132] ;

[0133] In the formula, Represents a set of points a point in Image gradient magnitude in the camera image; Let be the objective function that maximizes the image coordinate gradients of all edge points in the camera image.

[0134] Apply the above optimized matrix transformation relation to all cameras to obtain the accurate matrix transformation relation. The expression for all cameras synchronously performing the above optimization process is:

[0135] ;

[0136] In the formula, Represents the first in a binocular vision system One camera, .

[0137] In step S5, given the final transformation matrix This method can transform the point cloud of pipeline accessories from the digital model coordinate system to the measurement coordinate system, thus completing the measurement of the pipeline accessories. Combined with the pipe body data determined by the pipeline measurement, complete pipeline measurement data can be output. Pipeline alignment is then applied to determine the measurement deviation of the bending points between the pipeline under test and the pipeline digital model. By comparing the feature data of the pipeline accessories, the measurement deviation of the pipeline accessories between the pipeline under test and the pipeline digital model is determined.

[0138] Having described various embodiments of the application, it is to be understood that the above description is meant to be illustrative only and not exhaustive, and is not limited to the disclosed embodiments. Many modifications and variations will be apparent to those of ordinary skill in the art, without departing from the scope and spirit of the described embodiments. The choice of words in this document is intended to best explain the principles of the embodiments, the practical application, or technical improvement over the prior art, or to enable others skilled in the art to understand the embodiments disclosed herein.

Claims

1. A method for virtual adapter based plumbing reconstruction and detection, the method comprising: The method comprises the following steps: (1) acquiring multi-angle images of the pipe to be measured by a multi-view vision system, reconstructing the pipe to be measured, and determining the bending point data of the pipe to be measured; (2) By pipeline alignment, the correspondence relationship between the bending point of the to-be-measured pipeline and the bending point of the corresponding numerical model of the to-be-measured pipeline is determined, and the initial conversion matrix from the measurement coordinate system to the numerical model coordinate system is calculated based on the SVD decomposition method ; (3) Project the pipe fittings in the digital model to the image planes of the cameras through the initial transformation matrix and camera parameters, and iteratively optimize the target function with the goal of minimizing the pixel error and maximizing the gradient sum to obtain the adjustment matrix ; (4) Set threshold range, combined with adjustment matrix Screening edge points of pipeline accessories meeting threshold conditions, projecting the edge points to the image plane to maximize the edge point gradient sum as the objective function to optimize the adjustment matrix , Get the final conversion matrix ; (5) The pipe accessories in the digital model are converted to the measurement coordinate system through the final conversion matrix and spliced with the reconstructed pipe under test in step (1) to form a complete pipe model, and the position deviation of the bending points and the characteristic deviation of the pipe accessories of the pipe under test are compared with those of the digital model. In step (4), the optimization adjustment matrix is ​​performed. The process is as follows: Adjustment matrix Collecting points of a digital-analog pipeline accessory Convert to measurement coordinate system: ； wherein, a set of point coordinates of the pipeline accessories in the measurement coordinate system represent the digital model. Applying matrix conversion relationship The normal vector set of each point on the pipeline accessories of the number model in the number model coordinate system Convert to the measurement coordinate system to obtain the corresponding normal vector set The expression is as follows: ； For each camera in the vision system, a line is constructed connecting the optical center and a point on the object that is being imaged . ; Computing the ray The angle between the corresponding normal vectors ;​ Setting a range of threshold values of the included angle , ] and retaining points satisfying the included angle in the range, to form a candidate set of edge points ; edge point candidate set The points in the image are projected onto the corresponding two-dimensional image plane of the camera to obtain a set of two-dimensional image points. ; Set pixel threshold Filtering a set of points in a two-dimensional image Medium pixel values ​​below The points are used to obtain the set of valid edge points. ; For each camera of the multi-view vision system, compute a set of valid edge points with the image gradient sum of the projected points, iteratively optimize the adjustment matrix with the maximum image gradient sum as the objective function , output the final transformation matrix .

2. A virtual adapter based plumbing reconstruction and detection method according to claim 1, wherein, In step (1), the multi-view vision system needs to be calibrated before image acquisition. After calibration, the pipe to be measured is placed in the measurement area, and the calibrated multi-view vision system is used to take pictures of the pipe to obtain pipe images from all camera angles. The calibration process of the multi-view vision system is as follows: A calibration board matching the measurement width is used, the position of the calibration board is adjusted, and multiple groups of images are taken by the multi-view vision system; Based on the principle of close-range photogrammetry, the internal and external parameters of each camera are solved; The intrinsic parameters include: lens focal length , principal point deviation of the camera , distortion deviation of the camera lens ; The extrinsic parameters include a rotation matrix of a measurement coordinate system to a camera coordinate system , a translation matrix .

3. A virtual adapter based plumbing reconstruction and detection method according to claim 2, wherein, In step (2), the bending point analysis process of the pipe to be measured is as follows: The reconstructed pipe model is sampled along the center axis to calculate the local curvature at a fixed step; Straight segments and bending segments are distinguished by a curvature change threshold; The center axis of each straight segment is fitted, and the intersection of the axes of adjacent straight segments is solved as a bending point; The curvature change threshold is a preset threshold. If the curvature change is less than the curvature change threshold, the segment is determined to be a straight segment. If the curvature change is greater than the curvature change threshold, the segment is determined to be a bending segment.

4. The virtual adapter based plumbing reconstruction and detection method of claim 3, wherein, In the step (2), the initial conversion matrix is obtained by the following process: The actual distribution order of the bending points of the pipe to be measured obtained through actual measurement is used to analyze the connection structure relationship between them; The bending points predefined in the numerical model are analyzed in terms of their planning order in the design stage to analyze the connection structure relationship at the design level; The two connection structure relationships are compared, and on the basis of consistent connection structure relationships, the bending point set of the pipe to be measured is matched with the bending point set of the numerical model; Based on the existing bending points of the to-be-tested pipeline and the corresponding bending points of the numerical model, an initial conversion matrix of a measurement coordinate system in which the to-be-tested pipeline is located and a numerical model coordinate system in which the numerical model is located is determined by using an SVD decomposition method , the initial conversion matrix satisfies ； wherein, is a set of bend points of the pipe to be measured, is a set of bend points of the number module; is a rotation matrix from the number module coordinate system to the measurement coordinate system, is a translation matrix from the number module coordinate system to the measurement coordinate system.

5. A virtual adapter based plumbing reconstruction and detection method according to claim 4, wherein, In step (3), the matrix is calculated as follows: Collecting points of a digital-analog pipeline accessory By an initial transformation matrix Projecting to camera image coordinate system: ； wherein is a pixel coordinate from a digital-analog coordinate system to a two-dimensional image coordinate system; denotes a specific conversion, projection matrix relationship, the formula contained is expressed as: ； wherein, represents a set of point coordinates of the digital-analog pipeline accessories in the measurement coordinate system; The pipe accessories are projected from the measurement coordinate system to the two-dimensional image coordinate system of the camera, and the projection process expression is as follows: ； wherein is the position coordinate of a certain point on the pipe accessory in the measuring coordinate system; is the point is the actual image point coordinate of the point on the pipe accessory in the two-dimensional image coordinate system projected from the measuring coordinate system, , , is the coordinate of the camera optical center in the measuring coordinate system, , are respectively: ； wherein, , , are all rotation components for describing the rotation relationship of the two coordinate systems, , , is the coordinate of the camera optical center in the measurement coordinate system for describing the translation relationship of the two coordinate systems; With the maximum gradient and the minimum pixel sum as the objective function, iterative calculation is performed on the two-dimensional projection point set to adjust the matrix conversion relationship from the digital model coordinate system to the measurement coordinate system two-dimensional projection point set to adjust the matrix conversion relationship from the digital model coordinate system to the measurement coordinate system .

6. A virtual adapter based plumbing reconstruction and detection method according to claim 5, wherein, In step (3), the construction process of the objective function is as follows: The coordinate point set of the pipe accessories in the image coordinate system of a certain camera is as follows: ； The pixel sum and gradient sum of each camera image are calculated respectively: ； The global objective function is constructed by combining the pixel sum and gradient sum: ； In the formula, denotes Image gradient magnitude in the camera image; representing image pixel values in the camera image; the number of image points representing the tube attachment in the current camera image; represents the number of all cameras in the vision system, .

7. A virtual adapter based plumbing reconstruction and detection method according to claim 6, wherein, For the image gradient of a certain two-dimensional image point on the pipe attachment is represented as: ； ； wherein represents a gradient of the pixel value of the image in the horizontal direction; represents a gradient of the pixel value of the image in the vertical direction.

8. A virtual adapter based plumbing reconstruction and detection method according to claim 7, wherein, Step (5) comprises: converting the pipe accessory point cloud in the digital model by the final transformation matrix to the measurement coordinate system; The converted pipe accessory point cloud is spatially spliced with the pipe to be measured point cloud reconstructed in step (1) to form a complete pipe three-dimensional model containing the pipe accessories and the pipe body; Based on the complete pipe three-dimensional model, the following detections are performed: The spatial position deviation of the bending points of the pipe to be measured and the bending points of the numerical model is compared; The key feature size deviation of the pipe accessories is compared, including at least one of the flange hole position angle deviation, the valve mounting surface position deviation, or the clamp center distance deviation.

Images