Large-scene three-dimensional surface measurement method

By arranging control points in large scenes and using structured light 3D sensors and wide field of view cameras for image acquisition and optimization model construction, the calibration problem of structured light 3D sensors in large-scale measurement is solved, and high precision and efficiency are achieved. Three-dimensional surface measurement.

CN118941751APending Publication Date: 2024-11-12SHENZHEN UNIV
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411013901.4
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-26
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In large-scale three-dimensional measurements, the calibration of structured light three-dimensional sensors is difficult, resulting in low accuracy and low efficiency, and there are large deviations in the reconstruction results.

Method used

By arranging multiple control points in the target scene, using structured light 3D sensors and wide field of view cameras to acquire images, extract control point pixel coordinates, build optimization models to output optimization parameters, perform three-dimensional reconstruction and stitching, ensuring the accuracy of global three-dimensional point clouds Sex and scale to real-life spatial scale.

Benefits of technology

The accuracy and efficiency of three-dimensional surface measurement in large scenes are improved, the use of large targets and fixed requirements for camera structural parameters are avoided, flexibility is enhanced, splicing errors are controlled, and high-precision three-dimensional reconstruction results are ensured.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118941751A_ABST
    Figure CN118941751A_ABST
Patent Text Reader

Abstract

The invention discloses a large-scene three-dimensional surface measurement method, and the method comprises the steps: obtaining a target scene, and arranging a plurality of control points for the target scene based on a structured light three-dimensional sensor; the method comprises the following steps: acquiring a target scene from different angles by using a structured light three-dimensional sensor and a wide-view-field camera to obtain a plurality of scene images containing at least one control point; extracting a pixel coordinate point of each control point in the scene image so as to establish a control point image coordinate set; constructing an optimization model by combining camera parameters of the structured light three-dimensional sensor and the wide-view-field camera, and outputting optimization parameters; performing three-dimensional reconstruction and splicing on scene images acquired by the structured light three-dimensional sensor by using the optimized parameters to obtain a global three-dimensional point cloud; and zooming the global three-dimensional point cloud to a real space to obtain a three-dimensional surface corresponding to the target scene. According to the invention, the problem that the structured light three-dimensional sensor is difficult to calibrate under the condition of a large field of view can be solved, and the precision and efficiency of three-dimensional surface measurement are improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The invention relates to the technical field of three-dimensional digital imaging, and in particular to a large-scene three-dimensional surface measurement method. Background Art

[0002] With the rapid development of the automotive, shipbuilding, aerospace and other manufacturing industries, there are requirements for high-precision, high-density, and large-scale surface profile measurements of tens to hundreds of meters. At present, among the large-scale three-dimensional measurement methods, the lidar-based method has the problem of low reconstruction and splicing accuracy, while the laser tracker-based method is difficult to achieve high-density measurement. In contrast, as a non-contact, full-field measurement three-dimensional imaging system, the structured light three-dimensional sensor can complete high-precision, high-density 3D scanning tasks within a few meters. However, when the measurement range is extended to tens of meters or even hundreds of meters, high-precision and high-density three-dimensional scanning based on structured light will cause certain problems. For example, it is difficult to calibrate a structured light three-dimensional sensor in a large space. Large reference targets with known benchmarks are not only bulky and difficult to operate during on-site calibration, but also prone to deformation, resulting in reduced calibration accuracy. For another example, the reconstruction of a structured light three-dimensional sensor depends on its triangular structure parameters. In the motion measurement of a large space, the relative structure of the structured light three-dimensional sensor is easily deformed by vibration and collision, which requires repeated recalibration, resulting in low calibration efficiency. At the same time, the effective 3D imaging field of view of the structured light 3D sensor is small. If it only relies on the overlapping areas between small fields of view for stitching, measurement errors will continue to accumulate during the stitching of large scenes, resulting in large deviations in the final reconstruction results. Summary of the invention

[0003] The embodiments of the present invention provide a large-scene three-dimensional surface measurement method, device, computer equipment and storage medium, which aim to solve the problem of difficult calibration of structured light three-dimensional sensors under large fields of view, thereby improving the accuracy and efficiency of large-scene three-dimensional surface measurement.

[0004] In a first aspect, an embodiment of the present invention provides a large-scene three-dimensional surface measurement method, comprising:

[0005] Acquire a target scene, and arrange a plurality of control points on the target scene based on a structured light three-dimensional sensor;

[0006] Based on the control point, the structured light three-dimensional sensor and the wide-field-of-view camera are used to capture the target scene from different angles to obtain a plurality of scene images each containing at least one control point;

[0007] For each of the scene images, extract the pixel coordinates of each of the control points in the scene image, thereby establishing a control point image coordinate set;

[0008] Based on the image coordinate set, an optimization model is constructed in combination with camera parameters of the structured light 3D sensor and the wide field of view camera, and the optimization model outputs optimization parameters;

[0009] Using the optimized parameters, three-dimensionally reconstruct and stitch the scene images collected by the structured light three-dimensional sensor to obtain a global three-dimensional point cloud corresponding to the target scene;

[0010] The global three-dimensional point cloud is scaled to a real space scale to obtain a three-dimensional surface corresponding to the target scene.

[0011] In a second aspect, an embodiment of the present invention provides a large-scene three-dimensional surface measurement device, comprising:

[0012] A control point arrangement unit, used for acquiring a target scene and arranging a plurality of control points on the target scene based on a structured light three-dimensional sensor;

[0013] An image acquisition unit, configured to acquire a plurality of scene images each containing at least one control point from different angles of the target scene using the structured light three-dimensional sensor and the wide-field-of-view camera based on the control point;

[0014] A coordinate set establishing unit, for extracting the pixel coordinate point of each control point in the scene image for each scene image, so as to establish a control point image coordinate set;

[0015] A parameter optimization unit, configured to construct an optimization model based on the image coordinate set and in combination with camera parameters of the structured light 3D sensor and the wide field of view camera, and output optimization parameters from the optimization model;

[0016] A reconstruction and stitching unit, used to perform three-dimensional reconstruction and stitching on the scene image collected by the structured light three-dimensional sensor using the optimization parameters to obtain a global three-dimensional point cloud corresponding to the target scene;

[0017] A scale scaling unit is used to scale the global three-dimensional point cloud to a real space scale to obtain a three-dimensional surface corresponding to the target scene.

[0018] In a third aspect, an embodiment of the present invention provides a computer device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the large-scene three-dimensional surface measurement method as described in the first aspect is implemented.

[0019] In a fourth aspect, an embodiment of the present invention provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the large-scene three-dimensional surface measurement method as described in the first aspect is implemented.

[0020] An embodiment of the present invention provides a large-scene three-dimensional surface measurement method, the method comprising: acquiring a target scene, arranging a plurality of control points on the target scene based on a structured light three-dimensional sensor; based on the control points, using the structured light three-dimensional sensor and a wide-field-of-view camera to acquire a plurality of scene images each containing at least one control point from different angles of the target scene; for each of the scene images, extracting the pixel coordinate points of each of the control points in the scene image, thereby establishing a control point image coordinate set; based on the image coordinate set, constructing an optimization model in combination with camera parameters of the structured light three-dimensional sensor and the wide-field-of-view camera, and outputting optimization parameters from the optimization model; using the optimization parameters to perform three-dimensional reconstruction and splicing on the scene images acquired by the structured light three-dimensional sensor, thereby acquiring a global three-dimensional point cloud corresponding to the target scene; scaling the global three-dimensional point cloud to a real space scale, thereby acquiring a three-dimensional surface corresponding to the target scene. The embodiment of the present invention controls the stitching error of large scenes by introducing a wide-field camera, thereby ensuring the overall stitching accuracy, and establishes a posterior global optimization model to directly solve the reconstruction parameters and stitching posture of each scanning position through the collected scene data, thereby completing the three-dimensional reconstruction of the entire scene. This can solve the problem of difficult calibration of structured light three-dimensional sensors under large fields of view, thereby improving the accuracy and efficiency of large-scene three-dimensional surface measurement. At the same time, all parameters in the embodiment of the present invention are calculated based on actual collected data. This post-processing method not only avoids the operation of using large known targets for calibration in the prior art, but also achieves the effect that the relative structural parameters between cameras do not need to be kept fixed during the scanning process, thereby making the large-scene three-dimensional surface measurement process very flexible. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying any creative work.

[0022] Figure 1 A schematic flow chart of a large-scene three-dimensional surface measurement method is provided for an embodiment of the present invention;

[0023] Figure 2 A schematic diagram of camera scanning coverage in a large-scene three-dimensional surface measurement method is provided for an embodiment of the present invention;

[0024] Figure 3 A schematic diagram of an optimized model space in a large-scene three-dimensional surface measurement method is provided for an embodiment of the present invention;

[0025] Figure 4A schematic block diagram of a large-scene three-dimensional surface measurement device is provided for an embodiment of the present invention. DETAILED DESCRIPTION

[0026] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0027] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0028] It should also be understood that the terms used in this specification of the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the specification of the present invention and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0029] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0030] See below Figure 1 , Figure 1 A schematic flow chart of a large-scene three-dimensional surface measurement method provided in an embodiment of the present invention specifically includes: steps S101 to S106.

[0031] Step S101, acquiring a target scene, and arranging a plurality of control points for the target scene based on a structured light three-dimensional sensor;

[0032] Step S102: Based on the control point, the structured light 3D sensor and the wide-field-of-view camera are used to capture the target scene from different angles to obtain a plurality of scene images each containing at least one control point;

[0033] Step S103, for each of the scene images, extracting the pixel coordinates of each of the control points in the scene image, thereby establishing a control point image coordinate set;

[0034] Step S104: constructing an optimization model based on the image coordinate set and combining camera parameters of the structured light 3D sensor and the wide field of view camera, and outputting optimization parameters from the optimization model;

[0035] Step S105, using the optimized parameters to perform three-dimensional reconstruction and splicing on the scene image collected by the structured light three-dimensional sensor to obtain a global three-dimensional point cloud corresponding to the target scene;

[0036] Step S106: scaling the global three-dimensional point cloud to a real space scale to obtain a three-dimensional surface corresponding to the target scene.

[0037] In this embodiment, a plurality of control points are first arranged for the target scene, so that pattern features that are easy to detect and locate can be provided for the subsequent three-dimensional surface measurement process, and image processing is convenient. Then, the structured light three-dimensional sensor and the wide-field-of-view camera are used to collect images of the target scene. Here, the field of view of the wide-field-of-view camera is larger than the field of view of the structured light three-dimensional sensor, and the fields of view captured by the wide-field-of-view camera overlap with each other. In this way, the independent views of each scanning position of the structured light three-dimensional sensor can be connected together, and sufficiently strong multi-view geometric constraints are provided for the joint optimization of integrated vision, thereby controlling the stitching error of the large field of view. Then, the pixel coordinates of the control points in the collected scene image are extracted, and all the pixel coordinates are summarized to obtain the control point image coordinate set. Subsequently, an optimization model is constructed in combination with the camera parameters. The corresponding optimization parameters can be obtained through the optimization model. The optimization parameters include not only the camera parameters, but also the three-dimensional spatial coordinates of the control points. In this way, the structured light 3D sensor can be calibrated according to the optimization parameters, and its scanned views can be reconstructed and spliced ​​to obtain the global 3D point cloud of the target scene. Finally, the global 3D point cloud is scaled to the real space to obtain the 3D surface measurement results of the target scene.

[0038] This embodiment controls the stitching error of large scenes by introducing a wide-field camera, thereby ensuring the overall stitching accuracy, and establishes a posteriori global optimization model to directly solve the reconstruction parameters and stitching posture of each scanning position through the collected scene data, thereby completing the three-dimensional reconstruction of the entire scene. This can solve the problem of difficult calibration of structured light three-dimensional sensors under large fields of view, thereby improving the accuracy and efficiency of large-scene three-dimensional surface measurement. At the same time, all parameters in this embodiment are calculated based on actual collected data. This post-processing method not only avoids the operation of using large known targets for calibration in the prior art, but also achieves the effect that the relative structural parameters between cameras do not need to be kept fixed during the scanning process, thereby making the large-scene three-dimensional surface measurement process very flexible.

[0039] In one embodiment, the acquiring the target scene, arranging a plurality of control points on the target scene based on a structured light three-dimensional sensor, includes:

[0040] Obtaining the field of view coverage of the structured light 3D sensor, and evenly arranging n control points with the same or different marking patterns within the field of view coverage; wherein n≥4;

[0041] An index number is set for each of the control points in sequence, and an index number set for the control points is established.

[0042] In this embodiment, combined with Figure 2 The structured light 3D sensor and the wide field camera collect images of the target scene at different positions, and the control points are arranged in the target scene according to the field size of the structured light 3D sensor (each field should observe more than 4 markers, that is, located at the four corners of the field). In addition, index numbers can be set for the control points, so that the control points can be quickly located according to the index numbers, improving the overall 3D surface measurement efficiency.

[0043] In one embodiment, for each of the scene images, extracting the pixel coordinate point of each of the control points in the scene image to establish a control point image coordinate set includes:

[0044] When the marking patterns of the control points are the same, the same feature detection and positioning algorithm is used to extract the coordinates of the control points in the scene image to obtain the pixel coordinates of the control points;

[0045] When the marking patterns of the control points are different, different feature detection and positioning algorithms are used to extract the coordinates of the control points in the scene image to obtain the pixel coordinate points of the control points.

[0046] Specifically, the marking pattern of the control point includes a cross mark and a circle center mark;

[0047] When the marking patterns of the control points are different, different feature detection and positioning algorithms are used to extract the coordinates of the control points in the scene image to obtain the pixel coordinate points of the control points, including:

[0048] When the marking pattern of the control point is a cross mark, a corner feature detection and positioning algorithm is used to extract the coordinates of the control point in the scene image to obtain the pixel coordinate point of the control point;

[0049] When the marking pattern of the control point is a circle center mark, an ellipse feature detection and positioning algorithm is used to extract the coordinates of the control point in the scene image to obtain the pixel coordinate point of the control point.

[0050] In this embodiment, the control points are feature detected and located by a feature detection and positioning algorithm, thereby obtaining the pixel coordinates of the control points in the scene image. Here, since the same marking pattern may be used as the control points in the actual application scene, or different marking patterns may be used as the control points, an appropriate feature detection and positioning algorithm may be selected specifically according to the marking pattern of the control points. For example, if the marking pattern of the control point is a cross mark, then the corner feature detection and positioning algorithm can be used to extract the pixel coordinates. For another example, if the marking pattern of the control point is a center mark, then the ellipse feature detection and positioning algorithm can be used to extract the pixel coordinates. Furthermore, in order to distinguish it from the background of the target scene, more unique marking patterns can be designed to improve the efficiency and accuracy of coordinate extraction.

[0051] In one embodiment, the constructing of an optimization model based on the image coordinate set and combining camera parameters of the structured light 3D sensor and the wide field of view camera, and outputting optimization parameters by the optimization model, includes:

[0052] The optimization model is constructed according to the following formula:

[0053]

[0054] Wherein, J(K, RT, X) represents the optimization function of the optimization model, K represents the optimized camera intrinsic parameters and distortion coefficients, RT represents the optimized camera extrinsic parameters, and X represents the optimized three-dimensional space coordinates of the control point. represents the pixel coordinates of the control point; m(·) represents the projection process of the pinhole imaging model, K L represents the camera intrinsic parameters and distortion coefficients of the structured 3D sensor, represents the camera extrinsics of the structured 3D sensor, represents the 3D spatial coordinates of the control points collected by the structured 3D sensor, K sk represents the camera intrinsic parameters and distortion coefficients of the wide field camera, represents the camera extrinsics of the wide-field camera, represents the 3D spatial coordinates of the control points collected by the wide-field camera, M represents the number of positions observed by the camera in the structured light 3D sensor, Q represents the number of cameras in the structured 3D sensor, P represents the number of positions observed by the camera in the wide-field camera, and Ω x Represents the index number set of control points in the x-th scene image;

[0055] The optimization function is minimized to obtain the optimization parameters.

[0056] In this embodiment, combined with Figure 3 ,in Figure 3 is with Figure 2The corresponding integrated visual joint constraint model represents the multi-view geometric constraints of the control points in the multi-view projection imaging process. This embodiment combines the views of cameras with different field of view sizes together, and the optimization parameters required for reconstruction and stitching can be obtained by minimizing the cost function established under the multi-view geometric constraints.

[0057] In some preferred embodiments, in order to obtain a stable solution, an accurate initial value can be used to construct an optimization model to obtain the optimization parameters. The specific steps of initial value estimation are as follows:

[0058] (1) Use the camera hardware parameters as the initial internal parameters and set the initial value of the distortion coefficient to 0;

[0059] (2) The first two frames of the wide-field camera are processed, and the corresponding geometric constraint matrix is ​​calculated through the pixel coordinates of the corresponding control points in their overlapping fields of view. For a planar scene, the homography matrix of the two views induced by the plane is calculated; for a non-planar scene, the basic matrix of the two views is calculated.

[0060] (3) Perform SVD decomposition and other processing on the matrix in the previous step (homography matrix or basic matrix) to obtain the posture of the two frame views, and then triangulate the corresponding control points based on the posture of the two frame views to obtain the three-dimensional spatial coordinates of the control points at the initial position.

[0061] (4) For subsequent viewing angles of the large-field-of-view camera, the Pnp (Perspective-n-Points, referring to the object motion positioning problem from 3D to 2D point pairs) pose estimation technology and triangulation reconstruction technology are continuously used to incrementally obtain the three-dimensional spatial coordinates of all control points and the camera extrinsics at all positions.

[0062] (5) For each camera in the structured light 3D sensor, the Pnp pose estimation technique is also used to calculate the external parameters of all scanning positions.

[0063] In one embodiment, the step of performing three-dimensional reconstruction and stitching of the scene image collected by the structured light three-dimensional sensor using the optimized parameters to obtain a global three-dimensional point cloud corresponding to the target scene includes:

[0064] Obtaining camera structure parameters in the structured light 3D sensor;

[0065] The camera structural parameters and the camera intrinsic parameters and distortion coefficients in the optimization parameters are set as reconstruction parameters, and the camera extrinsic parameters in the optimization parameters are set as stitching parameters;

[0066] Using the reconstruction parameters, control point matching and local three-dimensional reconstruction are performed on the scene image collected by the structured light three-dimensional sensor to obtain a local three-dimensional point cloud;

[0067] The local three-dimensional point cloud is stitched into a global coordinate system using the stitching parameters to obtain a global three-dimensional point cloud corresponding to the target scene.

[0068] Furthermore, scaling the global three-dimensional point cloud to a real space scale to obtain a three-dimensional surface corresponding to the target scene includes:

[0069] Based on a preset standard ruler, obtaining the three-dimensional distance between the three-dimensional space coordinates of the control points at both ends of the standard ruler;

[0070] The scale factor between the real space and the three-dimensional space is calculated according to the length of the standard pool and the three-dimensional distance;

[0071] The global three-dimensional point cloud is scaled to a real space scale according to the scale factor to obtain a three-dimensional surface corresponding to the target scene.

[0072] In this embodiment, after the optimization is completed, reconstruction parameters (camera intrinsic parameters, distortion coefficients, and structural parameters between cameras) and stitching parameters (camera extrinsic parameters) are obtained at each position of the structured light 3D sensor. Therefore, the reconstruction parameters can be used to perform dense corresponding point matching and local 3D reconstruction on the image collected by the structured light 3D sensor, and then the stitching parameters can be used to stitch the 3D point cloud obtained by the local 3D reconstruction into the global coordinate system, and finally the global 3D point cloud of the entire target scene can be obtained.

[0073] Afterwards, the scale factor is obtained based on the optimized three-dimensional spatial coordinates of the control points and the standard ruler of known length. That is, the distance between the three-dimensional spatial coordinates of the control points at both ends of the standard ruler is compared with the real distance of the standard ruler to obtain the scale factor. Then, the global three-dimensional point cloud is converted to the real space size through the scale factor to obtain the reconstructed three-dimensional surface of the target scene.

[0074] Figure 4 A schematic block diagram of a large-scene three-dimensional surface measurement device 400 provided in an embodiment of the present invention, the device 400 includes:

[0075] A control point arrangement unit 401 is used to acquire a target scene and arrange a plurality of control points on the target scene based on a structured light 3D sensor;

[0076] An image acquisition unit 402 is configured to acquire a plurality of scene images each containing at least one control point from different angles of the target scene using the structured light 3D sensor and the wide-field-of-view camera based on the control point;

[0077] A coordinate set establishing unit 403 is used to extract the pixel coordinate point of each control point in the scene image for each scene image, so as to establish a control point image coordinate set;

[0078] A parameter optimization unit 404 is used to construct an optimization model based on the image coordinate set and in combination with camera parameters of the structured light 3D sensor and the wide field of view camera, and output optimization parameters from the optimization model;

[0079] A reconstruction and stitching unit 405 is used to perform three-dimensional reconstruction and stitching on the scene image collected by the structured light three-dimensional sensor using the optimization parameters to obtain a global three-dimensional point cloud corresponding to the target scene;

[0080] The scale scaling unit 406 is used to scale the global three-dimensional point cloud to a real space scale to obtain a three-dimensional surface corresponding to the target scene.

[0081] In one embodiment, the control point arrangement unit 401 includes:

[0082] A range setting unit, used to obtain the field of view coverage of the structured light 3D sensor, and evenly arrange n control points with the same or different marking patterns within the field of view coverage; wherein n≥4;

[0083] The index setting unit is used to set an index number for each of the control points in sequence and establish an index number set for the control points.

[0084] In one embodiment, the coordinate set establishing unit 403 includes:

[0085] A first coordinate extraction unit, configured to extract the coordinates of the control points in the scene image using the same feature detection and positioning algorithm when the marking patterns of the control points are the same, so as to obtain the pixel coordinates of the control points;

[0086] The second coordinate extraction unit is used to extract the coordinates of the control points in the scene image by using different feature detection and positioning algorithms when the marking patterns of the control points are different, so as to obtain the pixel coordinate points of the control points.

[0087] In one embodiment, the marking pattern of the control point includes a cross mark and a circle center mark;

[0088] The second coordinate extraction unit comprises:

[0089] A corner point extraction unit, configured to extract the coordinates of the control point in the scene image by using a corner point feature detection and positioning algorithm when the marking pattern of the control point is a cross mark, so as to obtain the pixel coordinate point of the control point;

[0090] The ellipse extraction unit is used to extract the coordinates of the control points in the scene image by using an ellipse feature detection and positioning algorithm when the marking pattern of the control points is a circle center mark, so as to obtain the pixel coordinate points of the control points.

[0091] In one embodiment, the parameter optimization unit 404 includes:

[0092] The model building unit is used to build the optimization model according to the following formula:

[0093]

[0094] Wherein, J(K, RT, X) represents the optimization function of the optimization model, K represents the optimized camera intrinsic parameters and distortion coefficients, RT represents the optimized camera extrinsic parameters, and X represents the optimized three-dimensional space coordinates of the control point. represents the pixel coordinates of the control point; m(·) represents the projection process of the pinhole imaging model, K L represents the camera intrinsic parameters and distortion coefficients of the structured 3D sensor, represents the camera extrinsics of the structured 3D sensor, represents the 3D spatial coordinates of the control points collected by the structured 3D sensor, K Sk represents the camera intrinsic parameters and distortion coefficients of the wide field camera, represents the camera extrinsics of the wide-field camera, represents the 3D spatial coordinates of the control points collected by the wide-field camera, M represents the number of positions observed by the camera in the structured light 3D sensor, Q represents the number of cameras in the structured 3D sensor, P represents the number of positions observed by the camera in the wide-field camera, and Ω x Represents the index number set of control points in the x-th scene image;

[0095] The minimization unit is used to perform minimization processing on the optimization function to obtain the optimization parameter.

[0096] In one embodiment, the reconstruction and splicing unit 405 includes:

[0097] A parameter acquisition unit, used to acquire camera structure parameters in the structured light 3D sensor;

[0098] A parameter setting unit, used to set the camera structural parameters and the camera intrinsic parameters and distortion coefficients in the optimization parameters as reconstruction parameters, and to set the camera extrinsic parameters in the optimization parameters as stitching parameters;

[0099] A reconstruction unit, configured to perform control point matching and local three-dimensional reconstruction on the scene image collected by the structured light three-dimensional sensor using the reconstruction parameters to obtain a local three-dimensional point cloud;

[0100] A stitching unit is used to stitch the local three-dimensional point cloud into a global coordinate system using the stitching parameters to obtain a global three-dimensional point cloud corresponding to the target scene.

[0101] In one embodiment, the scale scaling unit 406 includes:

[0102] A distance acquisition unit, configured to acquire, based on a preset standard ruler, a three-dimensional distance between three-dimensional spatial coordinates of control points located at two ends of the standard ruler;

[0103] A factor calculation unit, used for calculating a scale factor between the real space and the three-dimensional space according to the length of the standard pool and the three-dimensional distance;

[0104] A three-dimensional reconstruction unit is used to scale the global three-dimensional point cloud to a real space scale according to the scale factor to obtain a three-dimensional surface corresponding to the target scene.

[0105] Since the embodiments of the apparatus part correspond to the embodiments of the method part, please refer to the description of the embodiments of the method part for the embodiments of the apparatus part, which will not be repeated here.

[0106] The embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed, the steps provided in the above embodiment can be implemented. The storage medium may include: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and other media that can store program codes.

[0107] The embodiment of the present invention also provides a computer device, which may include a memory and a processor, wherein a computer program is stored in the memory, and when the processor calls the computer program in the memory, the steps provided in the above embodiment may be implemented. Of course, the computer device may also include various network interfaces, power supplies and other components.

[0108] The various embodiments in the specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same and similar parts between the various embodiments can be referred to each other. For the system disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part description. It should be pointed out that for ordinary technicians in this technical field, without departing from the principles of this application, several improvements and modifications can be made to the present application, and these improvements and modifications also fall within the scope of protection of the claims of this application.

[0109] It should also be noted that, in this specification, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "comprises", "comprising" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the statement "comprising a ..." does not exclude the presence of other identical elements in the process, method, article or device including the element.

Claims

1. A large scene three-dimensional surface measurement method, characterized in that: include: Acquire a target scene, and arrange a plurality of control points on the target scene based on a structured light three-dimensional sensor; Based on the control point, the structured light three-dimensional sensor and the wide-field-of-view camera are used to capture the target scene from different angles to obtain a plurality of scene images each containing at least one control point; For each of the scene images, extract the pixel coordinates of each of the control points in the scene image, thereby establishing a control point image coordinate set; Based on the image coordinate set, an optimization model is constructed in combination with camera parameters of the structured light 3D sensor and the wide field of view camera, and the optimization model outputs optimization parameters; Using the optimized parameters, three-dimensionally reconstruct and stitch the scene images collected by the structured light three-dimensional sensor to obtain a global three-dimensional point cloud corresponding to the target scene; The global three-dimensional point cloud is scaled to a real space scale to obtain a three-dimensional surface corresponding to the target scene.

2. The large-scene three-dimensional surface measurement method according to claim 1, characterized in that: The acquiring of the target scene and arranging a plurality of control points on the target scene based on the structured light three-dimensional sensor include: Obtaining the field of view coverage of the structured light 3D sensor, and evenly arranging n control points with the same or different marking patterns within the field of view coverage; wherein n≥4; An index number is set for each of the control points in sequence, and an index number set for the control points is established.

3. The large-scene three-dimensional surface measurement method according to claim 2, characterized in that: For each of the scene images, extracting the pixel coordinate point of each of the control points in the scene image to establish a control point image coordinate set includes: When the marking patterns of the control points are the same, the same feature detection and positioning algorithm is used to extract the coordinates of the control points in the scene image to obtain the pixel coordinates of the control points; When the marking patterns of the control points are different, different feature detection and positioning algorithms are used to extract the coordinates of the control points in the scene image to obtain the pixel coordinate points of the control points.

4. The large-scene three-dimensional surface measurement method according to claim 3 is characterized in that: The marking pattern of the control point includes a cross mark and a circle center mark; When the marking patterns of the control points are different, different feature detection and positioning algorithms are used to extract the coordinates of the control points in the scene image to obtain the pixel coordinate points of the control points, including: When the marking pattern of the control point is a cross mark, a corner feature detection and positioning algorithm is used to extract the coordinates of the control point in the scene image to obtain the pixel coordinate point of the control point; When the marking pattern of the control point is a circle center mark, an ellipse feature detection and positioning algorithm is used to extract the coordinates of the control point in the scene image to obtain the pixel coordinate point of the control point.

5. The large-scene three-dimensional surface measurement method according to claim 1, characterized in that: The step of constructing an optimization model based on the image coordinate set and combining the camera parameters of the structured light 3D sensor and the wide field camera, and outputting optimization parameters by the optimization model, includes: The optimization model is constructed according to the following formula: Wherein, J(K, RT, X) represents the optimization function of the optimization model, K represents the optimized camera intrinsic parameters and distortion coefficients, RT represents the optimized camera extrinsic parameters, and X represents the optimized three-dimensional space coordinates of the control point. represents the pixel coordinates of the control point; m(·) represents the projection process of the pinhole imaging model, K L represents the camera intrinsic parameters and distortion coefficients of the structured 3D sensor, represents the camera extrinsics of the structured 3D sensor, represents the 3D spatial coordinates of the control points collected by the structured 3D sensor, K Sk represents the camera intrinsic parameters and distortion coefficients of the wide field camera, represents the camera extrinsics of the wide-field camera, represents the 3D spatial coordinates of the control points collected by the wide-field camera, M represents the number of positions observed by the camera in the structured light 3D sensor, Q represents the number of cameras in the structured 3D sensor, P represents the number of positions observed by the camera in the wide-field camera, and Ω x Represents the index number set of control points in the x-th scene image; The optimization function is minimized to obtain the optimization parameters.

6. The large-scene three-dimensional surface measurement method according to claim 5, characterized in that: The step of performing three-dimensional reconstruction and splicing on the scene image collected by the structured light three-dimensional sensor by using the optimized parameters to obtain a global three-dimensional point cloud corresponding to the target scene includes: Obtaining camera structure parameters in the structured light 3D sensor; The camera structural parameters and the camera intrinsic parameters and distortion coefficients in the optimization parameters are set as reconstruction parameters, and the camera extrinsic parameters in the optimization parameters are set as stitching parameters; Using the reconstruction parameters, control point matching and local three-dimensional reconstruction are performed on the scene image collected by the structured light three-dimensional sensor to obtain a local three-dimensional point cloud; The local three-dimensional point cloud is stitched into a global coordinate system using the stitching parameters to obtain a global three-dimensional point cloud corresponding to the target scene.

7. The large-scene three-dimensional surface measurement method according to claim 5, characterized in that: Scaling the global three-dimensional point cloud to a real space scale to obtain a three-dimensional surface corresponding to the target scene includes: Based on a preset standard ruler, obtaining the three-dimensional distance between the three-dimensional space coordinates of the control points at both ends of the standard ruler; The scale factor between the real space and the three-dimensional space is calculated according to the length of the standard pool and the three-dimensional distance; The global three-dimensional point cloud is scaled to a real space scale according to the scale factor to obtain a three-dimensional surface corresponding to the target scene.

8. A large-scene three-dimensional surface measurement device, characterized in that: include: A control point arrangement unit, used for acquiring a target scene and arranging a plurality of control points on the target scene based on a structured light three-dimensional sensor; An image acquisition unit, configured to acquire a plurality of scene images each containing at least one control point from different angles of the target scene using the structured light three-dimensional sensor and the wide-field-of-view camera based on the control point; A coordinate set establishing unit, for extracting the pixel coordinate point of each control point in the scene image for each scene image, so as to establish a control point image coordinate set; A parameter optimization unit, configured to construct an optimization model based on the image coordinate set and in combination with camera parameters of the structured light 3D sensor and the wide field of view camera, and output optimization parameters from the optimization model; A reconstruction and stitching unit, used to perform three-dimensional reconstruction and stitching on the scene image collected by the structured light three-dimensional sensor using the optimization parameters to obtain a global three-dimensional point cloud corresponding to the target scene; A scale scaling unit is used to scale the global three-dimensional point cloud to a real space scale to obtain a three-dimensional surface corresponding to the target scene.

9. A computer device, characterized in that: The method comprises a memory, a processor and a computer program stored in the memory and executable on the processor, wherein the processor implements the large-scene three-dimensional surface measurement method as claimed in any one of claims 1 to 7 when executing the computer program.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the large-scene three-dimensional surface measurement method according to any one of claims 1 to 7 is implemented.