Multi-view three-dimensional reconstruction method and device based on biaxial galvanometer scanning

Abstract

This invention relates to the field of laser scanning technology, specifically to a multi-view 3D reconstruction method and apparatus based on dual-axis galvanometer scanning. This method selects single-point, single-line, or surface scanning modes according to different measurement requirements and combines multi-view triangulation to obtain corresponding 3D spatial points or 3D point cloud data. This solves the problem of simultaneously handling fixed-point measurement, cross-sectional contour measurement, and large-area topography reconstruction within the same hardware architecture. Without introducing additional structured light projection devices, the existing point laser and dual-axis galvanometer hardware in the laser processing equipment can be reused to obtain 3D information at different levels of points, lines, and surfaces. This enables the laser processing equipment to possess real-time 3D perception, workpiece positioning, focus correction, and topography detection capabilities, significantly improving system integration, measurement flexibility, and the level of intelligence in the processing process.

Description

Technical Field

[0001] This invention relates to the field of laser scanning technology, and specifically to a multi-view three-dimensional reconstruction method and apparatus based on dual-axis galvanometer scanning. Background Technology

[0002] In laser engraving, marking, and welding applications, a dual-axis galvanometer and a point laser are typically used. The high-power laser point is scanned across the surface of the target object by controlling the galvanometer's deflection angle to complete the processing. However, these processing scenarios often require acquiring the three-dimensional pose and shape information of the target workpiece to determine the processing area, laser focusing distance, and processing trajectory parameters. Furthermore, three-dimensional quality inspection of the workpiece may be necessary after processing. However, current technologies typically rely on manual measurement and positioning, or planar measurement using a two-dimensional camera, which fails to achieve accurate and dense three-dimensional reconstruction of the target object.

[0003] To achieve rapid 3D measurement, existing technologies typically employ stand-alone structured light 3D scanning equipment. However, such equipment has significant limitations: First, the projection mode is fixed, making it difficult to simultaneously perform single-point ranging, cross-sectional contour measurement, and overall surface topography reconstruction within the same hardware architecture. Second, high-precision 3D reconstruction often relies on expensive global shutter industrial cameras and high-precision synchronous structured light projection devices; if low-cost rolling shutter cameras are used, image fragmentation and distortion are prone to occur during dynamic scanning. Third, while existing laser processing equipment possesses high-precision galvanometers and lasers, it lacks an integrated design with the 3D perception system, resulting in low system integration and difficulty in achieving integrated "processing-inspection." Summary of the Invention

[0004] In view of the aforementioned problems, this application is made to provide a multi-view three-dimensional reconstruction method and apparatus based on biaxial galvanometer scanning that overcomes or at least partially solves the aforementioned problems.

[0005] The method includes: A multi-view 3D reconstruction method based on biaxial galvanometer scanning, the method being executed by a multi-view 3D reconstruction device, the multi-view 3D reconstruction device comprising a point laser, a biaxial galvanometer, and at least two imaging units; the method comprising the following steps: Based on the current requirements of the 3D measurement task, the target scanning mode of the dual-axis galvanometer is determined. The target scanning mode is one of the following: single-point scanning mode, single-line scanning mode, or surface scanning mode. Based on the target scanning mode, the laser emitted by the point laser is deflected by the biaxial galvanometer and then irradiates the surface of the target object, and image data containing laser features is acquired by the at least two imaging units; Extract the pixel coordinate information corresponding to the laser feature from the image data acquired by each imaging unit; Based on the geometric relationship between the at least two imaging units and the pixel coordinate information, multi-view triangulation calculation is performed to determine the three-dimensional spatial point corresponding to the laser feature; if there are multiple sets of three-dimensional spatial points, then the three-dimensional point cloud data is determined based on the multiple sets of three-dimensional spatial points.

[0006] Preferably, determining the target scanning mode of the biaxial galvanometer based on the current 3D measurement task requirements includes: When the target scanning mode is single-point scanning mode, the fast axis and slow axis of the dual-axis galvanometer are controlled to remain stationary or deflect to a preset angle so that the laser emitted by the point laser forms a single laser point on the surface of the target object. Control at least two imaging units to acquire image data containing laser features; wherein, the laser features are single laser point features; Multi-view triangulation is performed based on the pixel coordinates of a single laser point feature in the image data to determine the single three-dimensional spatial coordinates; or, When the target scanning mode is single-line scanning mode, the fast axis of the dual-axis galvanometer is controlled to scan within the single exposure time of each imaging unit, while the slow axis maintains a fixed angle; so that the laser emitted by the point laser forms a continuous laser line along the surface of the target object within the exposure time. Control at least two imaging units to acquire image data containing laser features; wherein, the laser features are continuous laser line features; Multi-view triangulation is performed based on the pixel coordinates of continuous laser line features in image data to determine a set of three-dimensional spatial points; or, When the target scanning mode is area scanning mode, the slow axis of the dual-axis galvanometer is controlled to change the scanning position according to a preset step angle between adjacent exposure times; After multiple slow-axis steps, multiple sets of three-dimensional spatial points are determined, and the coordinates of these multiple sets of three-dimensional spatial points are unified and fused to generate three-dimensional point cloud data that characterizes the overall shape of the target object.

[0007] Preferably, when the target scanning mode is a single-line scanning mode, controlling the fast axis of the dual-axis galvanometer to scan within a single exposure time of each imaging unit includes: When the imaging unit is a rolling shutter imaging unit, the oscillation frequency of the fast axis in the biaxial galvanometer is... for:

[0008] in, For row scan cycles, To ensure effective exposure of rows, K is the laser line length coverage factor, and K is the scanning waveform coefficient: K=2 for reciprocating scanning and K=1 for unidirectional scanning. When the imaging unit is a global shutter imaging unit, the oscillation frequency of the fast axis in the dual-axis galvanometer is... for:

[0009] in, This refers to the exposure time per frame of the global shutter. K is the laser line length coverage factor, and K is the scanning waveform coefficient: K=2 for reciprocating scanning and K=1 for unidirectional scanning.

[0010] Preferably, extracting the pixel coordinate information corresponding to the laser feature from the image data acquired by each imaging unit includes: A sub-pixel center localization algorithm is used to extract the sub-pixel level center coordinate sequence of the laser line feature, and the pixel coordinate information includes the sub-pixel level center coordinate sequence.

[0011] Preferably, the step of performing multi-view triangulation calculations based on the geometric relationship between the at least two imaging units and the pixel coordinate information to determine the three-dimensional spatial point corresponding to the laser feature includes: The pixel coordinate information is subjected to distortion correction processing to obtain the corresponding distorted pixel coordinates; Based on the distortion-corrected pixel coordinates and the pose relationship of at least two imaging units relative to a unified reference coordinate system, a geometric projection constraint between two-dimensional pixel coordinates and three-dimensional spatial coordinates is constructed. Based on the geometric projection constraints, multi-view triangulation is performed on the pixel coordinates of different imaging units to determine the three-dimensional spatial points corresponding to the features of the continuous laser line.

[0012] Preferably, if the three-dimensional spatial points are in multiple sets, determining the three-dimensional point cloud data based on the multiple sets of three-dimensional spatial points includes: At each scanning position, multiple sets of three-dimensional spatial points calculated by different imaging units are transformed to the reference coordinate system of the main imaging unit. In the reference coordinate system, adjacent three-dimensional spatial points with a spatial distance less than a preset threshold are determined to be the same spatial point, thus obtaining a fused three-dimensional spatial point set corresponding to each scanning position; Multiple scan locations corresponding to the fused three-dimensional spatial point set are merged in the reference coordinate system of the main imaging unit to generate three-dimensional point cloud data.

[0013] Preferably, it further includes: Based on the intrinsic and extrinsic parameter matrices of the main imaging unit, each three-dimensional spatial point in the three-dimensional point cloud data is projected into the image data of the main imaging unit to determine the pixel position of each three-dimensional spatial point in the target image. Extract the color information of the corresponding pixel position and assign the color information to the corresponding three-dimensional spatial point to generate three-dimensional point cloud data with color information.

[0014] A multi-view 3D reconstruction device based on dual-axis galvanometer scanning is also provided. The device includes a point laser, a dual-axis galvanometer, and at least two imaging units, and further includes: The control module is used to determine the target scanning mode of the dual-axis galvanometer according to the current three-dimensional measurement task requirements. The target scanning mode is one of the following: single-point scanning mode, single-line scanning mode, or surface scanning mode. The acquisition module is used to control the laser emitted by the point laser to be deflected by the dual-axis galvanometer and then irradiate the surface of the target object based on the target scanning mode, and the at least two imaging units acquire image data containing laser features. The extraction module extracts the pixel coordinate information corresponding to the laser feature from the image data collected by each imaging unit; The determination module performs multi-view triangulation calculations based on the geometric relationship between the at least two imaging units and the pixel coordinate information to determine the three-dimensional spatial points corresponding to the laser features; if there are multiple sets of three-dimensional spatial points, the three-dimensional point cloud data is determined based on the multiple sets of three-dimensional spatial points.

[0015] A computer electronic device is also provided, including a processor, a memory, and a computer program stored in the memory and capable of running on the processor. When the computer program is executed by the processor, it implements the steps of the multi-view three-dimensional reconstruction method based on dual-axis galvanometer scanning as described above.

[0016] A computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements the steps of the multi-view three-dimensional reconstruction method based on dual-axis galvanometer scanning as described above.

[0017] This application has the following advantages: In the embodiments of this application, by selecting single-point, single-line, or surface scanning modes according to different measurement needs, and combining multi-view triangulation to directly obtain the corresponding three-dimensional spatial points or three-dimensional point cloud data, the problem of difficulty in simultaneously handling fixed-point measurement, cross-sectional contour measurement, and large-area topography reconstruction under the same hardware architecture is solved. Without introducing additional structured light projection devices, the original point laser and dual-axis galvanometer hardware in the laser processing equipment can be reused to obtain three-dimensional information at different levels of points, lines, and surfaces. This enables the laser processing equipment to have real-time three-dimensional perception, workpiece positioning, focus correction, and topography detection capabilities, significantly improving system integration, measurement flexibility, and the level of intelligence in the processing process. Attached Figure Description

[0018] To more clearly illustrate the technical solution of this application, the drawings used in the description of this application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0019] Figure 1 This is a flowchart illustrating the steps of a multi-view 3D reconstruction method based on dual-axis galvanometer scanning, provided in an embodiment of this application. Figure 2 This is a structural block diagram of a multi-view three-dimensional reconstruction device based on dual-axis galvanometer scanning provided in one embodiment of this application; Figure 3 This is a schematic diagram of the structure of a computer electronic device provided in an embodiment of the present invention; 1. Computer electronic device; 2. External device; 3. Processing unit; 4. Bus; 5. Network adapter; 6. I / O interface; 7. Display; 8. Memory; 9. Random access memory; 10. Cache memory; 11. Storage system; 12. Program / utility; 13. Program module. Detailed Implementation

[0020] To make the objectives, features, and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without inventive effort are within the scope of protection of this application.

[0021] The inventors discovered through analysis of existing technologies that multi-view 3D reconstruction compatible with different types of cameras can be achieved by utilizing the existing dual-axis galvanometer and single-point laser hardware architecture, and scanning imaging and 3D calculation can be completed in the same system, thereby improving system integration and lowering the hardware threshold.

[0022] Reference Figure 1 This application illustrates a multi-view 3D reconstruction method based on biaxial galvanometer scanning according to an embodiment of the present application: the method is executed by a multi-view 3D reconstruction device, which includes a point laser, a biaxial galvanometer, and at least two imaging units; The method includes the following steps: S110. Based on the current requirements of the three-dimensional measurement task, determine the target scanning mode of the dual-axis galvanometer. The target scanning mode is one of the following: single-point scanning mode, single-line scanning mode, or surface scanning mode. S120, based on the target scanning mode, the laser emitted by the point laser is controlled to be deflected by the dual-axis galvanometer and then irradiate the surface of the target object, and the image data containing laser features is acquired by the at least two imaging units; S130, extract the pixel coordinate information corresponding to the laser feature from the image data collected by each imaging unit; S140, based on the geometric relationship between the at least two imaging units and the pixel coordinate information, perform multi-view triangulation calculation to determine the three-dimensional spatial point corresponding to the laser feature; if the three-dimensional spatial point is in multiple sets, then determine the three-dimensional point cloud data based on the multiple sets of three-dimensional spatial points.

[0023] It should be noted that a three-dimensional point refers to the three-dimensional spatial coordinates of a certain feature on the surface of a target object.

[0024] In the embodiments of this application, by selecting single-point, single-line, or surface scanning modes according to different measurement needs, and combining multi-view triangulation to directly obtain the corresponding three-dimensional spatial points or three-dimensional point cloud data, the problem of difficulty in simultaneously handling fixed-point measurement, cross-sectional contour measurement, and large-area topography reconstruction under the same hardware architecture is solved. Without introducing additional structured light projection devices, the original point laser and dual-axis galvanometer hardware in the laser processing equipment can be reused to obtain three-dimensional information at different levels of points, lines, and surfaces. This enables the laser processing equipment to have real-time three-dimensional perception, workpiece positioning, focus correction, and topography detection capabilities, significantly improving system integration, measurement flexibility, and the level of intelligence in the processing process.

[0025] The following will further describe a multi-view three-dimensional reconstruction method based on biaxial galvanometer scanning in this exemplary embodiment.

[0026] In one embodiment of the present invention, the specific process of step S110, "determining the target scanning mode of the biaxial galvanometer according to the current three-dimensional measurement task requirements, wherein the target scanning mode is one of a single-point scanning mode, a single-line scanning mode, or a surface scanning mode, can be further described in conjunction with the following description.

[0027] In this embodiment, the current 3D measurement task requirements can originate from function call instructions of the machining control system or from measurement targets set by the manual operation interface. The 3D measurement tasks include at least: fixed-point measurement tasks to acquire single-point 3D coordinate information of a specific spatial location of the target object; cross-sectional measurement tasks to acquire the contour curve of a cross section of the target object; and full-field 3D reconstruction tasks to acquire overall surface topography information of the target object. Different tasks correspond to different scanning modes. Once a specific 3D measurement task is determined, the corresponding scanning mode control parameters are output, thereby determining the target scanning mode of the dual-axis galvanometer.

[0028] Specifically, when the three-dimensional measurement task is a fixed-point measurement task, it is not necessary to form spatial line or surface data. It is only necessary to obtain the three-dimensional coordinate information of a specified spatial position of the target object. At this time, the target scanning mode of the dual-axis galvanometer is determined to be the single-point scanning mode.

[0029] When the target scanning mode is single-point scanning mode, the fast axis and slow axis of the dual-axis galvanometer are controlled to remain stationary or deflect to a preset angle so that the laser emitted by the point laser forms a single laser point on the surface of the target object. Control at least two imaging units to acquire image data containing laser features; wherein, the laser features are single laser point features; Multi-view triangulation is performed based on the pixel coordinate information corresponding to the single laser point feature in the image data to determine the single three-dimensional spatial coordinates.

[0030] A point laser forms a single, stable spot on the surface of a target object. A multi-view imaging unit synchronously acquires this spot and calculates its corresponding single three-dimensional spatial coordinates through multi-view triangulation. This mode can be used for laser focus correction, machining height calibration, and reference point positioning.

[0031] When performing the cross-sectional measurement task of the three-dimensional measurement task of the cross-sectional contour curve, it is necessary to obtain the continuous spatial curve information of a certain cross-section of the target object, but there is no need to perform slow axis stepping expansion. At this time, the target scanning mode of the dual-axis galvanometer is determined to be the single-line scanning mode.

[0032] When the target scanning mode is single-line scanning mode, the fast axis of the dual-axis galvanometer is controlled to scan within the single exposure time of each imaging unit, while the slow axis maintains a fixed angle; so that the laser emitted by the point laser forms a continuous laser line along the surface of the target object within the exposure time. Control at least two imaging units to acquire image data containing laser features; wherein, the laser features are continuous laser line features; Multi-view triangulation calculations are performed based on the pixel coordinate information corresponding to the continuous laser line features in the image data to determine a set of three-dimensional spatial points.

[0033] Because the imaging unit performs integral imaging of the incident light during exposure, multiple spatial locations illuminated by the point laser at different times accumulate in the same frame image, presenting a continuous laser line feature. This achieves the synthesis of a continuous spatial line feature from temporal scanning. Subsequently, through multi-view triangulation calculation, a set of three-dimensional spatial points is obtained, forming the three-dimensional cross-sectional contour curve of the target object. This mode is used for rapid positioning, coarse positioning, or pre-scanning of the trajectory before processing.

[0034] When the three-dimensional measurement task is a full-field three-dimensional reconstruction task of the overall surface topography information, it is necessary to obtain three-dimensional data of different scanning positions on the entire surface of the target object. At this time, the target scanning mode of the biaxial galvanometer is determined to be the surface scanning mode.

[0035] When the target scanning mode is area scanning mode, the slow axis of the dual-axis galvanometer is controlled to change the scanning position according to a preset step angle between adjacent exposure times; After multiple slow-axis steps, multiple sets of three-dimensional spatial points are determined, and the coordinates of these multiple sets of three-dimensional spatial points are unified and fused to generate three-dimensional point cloud data that characterizes the overall shape of the target object.

[0036] Through slow-axis stepping control, a continuous laser line translates line by line in space, achieving a scanning expansion from line to surface. By superimposing multiple scans, three-dimensional point cloud data covering the entire surface of the target object is finally obtained. This mode can be used for overall shape detection, workpiece pose recognition, and post-processing quality assessment.

[0037] Specifically, in area scan mode, within each exposure cycle, the fast-axis scan, combined with the imaging unit's exposure integration, can only form a single continuous laser line, yielding local cross-sectional contour information rather than complete surface or volume data. The slow-axis movement occurs between adjacent exposure times; that is, after the current frame exposure is complete, the imaging unit stops integrating and reads the image, and the fast-axis scan also stops. Before the next exposure, the slow-axis performs a preset step angle change. This timing arrangement effectively avoids the slow-axis movement affecting the spatial consistency of the fast-axis scan, ensuring that each frame exposure corresponds to only one fixed slow-axis position, and that each image data contains only a single continuous laser line feature.

[0038] It should be noted that the slow-axis stepping motion in this embodiment is a constant-angle stepping motion. Understandably, each step of the slow axis changes the projection position of the point laser on the target object's surface, causing the continuous laser line formed by the next fast-axis scan to spatially shift relative to the previous line.

[0039] The above embodiments, under the same hardware architecture, realize the acquisition of three spatial information methods—point, line, and surface—by switching control modes, and achieve integrated processing, measurement, and detection in the processing equipment.

[0040] Furthermore, in single-line scanning mode, the fast axis of the dual-axis galvanometer is controlled to scan within the single exposure time of each imaging unit (i.e., camera). The fast axis refers to the scanning axis in the dual-axis galvanometer used for high-speed reciprocating or unidirectional oscillation, namely the X-axis. By applying a preset drive signal to this fast axis, a single-point laser is made to move continuously along a certain path on the surface of the target object, thereby forming a scanning trajectory that changes with time in space.

[0041] Because the imaging unit accumulates and images the incident light signal during the exposure time—that is, it integrates the received light energy during the exposure—when the point laser passes through different spatial positions multiple times during the exposure time, the image sensor accumulates and records the laser reflections from these different time points in the same frame image, thus forming a continuous laser line feature on the image. This laser line is not a physical line light source existing simultaneously in space, but rather a laser line feature synthesized by the time integration effect.

[0042] Imaging units can be divided into rolling shutter imaging units and global shutter imaging units.

[0043] When the imaging unit is a rolling shutter imaging unit, the oscillation frequency of the fast axis in the biaxial galvanometer is... for:

[0044] in, For row scan cycles, To ensure effective exposure of rows, The laser line length coverage factor is the ratio of the actual scanned laser line length to the maximum scannable laser line length within the imaging unit's field of view; K is the scanning waveform coefficient: K=2 for reciprocating scanning and K=1 for unidirectional scanning. This ensures that the laser trajectory completely covers the photosensitive window in the time domain, eliminating breakage distortion. The system can adaptively adjust the K value according to the actual scanning waveform.

[0045] In the above process, by controlling the oscillation frequency of the fast-axis scanning motion, it is ensured that the laser point passes through at least once within each line exposure time window, thereby forming a continuous laser line feature in the image. Specifically, the line scanning period refers to the time interval between the start of exposure of two adjacent lines of pixels in the rolling shutter imaging unit, and the effective exposure line number refers to the exposure time of a single line of pixels in the imaging unit. The corresponding equivalent number of rows, i.e., the exposure time of the imaging unit. With row scan cycle The ratio is For example, based on the currently set exposure time, the critical frequency for fast-axis scanning can be quickly calculated. In actual configuration, the oscillation frequency of the fast axis is controlled. Critical frequency The energy density is N times that of lasers (e.g., 5 to 10 times). Tests have shown that under these parameter matching conditions, the laser scanning trajectory achieves multiple energy integrations within a single row of pixel photosensitive windows, eliminating feature breaks caused by the roller blind effect. For surface scanning reconstruction of a 10mm high standard gauge block, the height direction measurement accuracy can reach ±0.09mm.

[0046] Understandably, rolling shutters capture light line by line, with different image lines corresponding to different time windows. If the fast-axis scan fails to capture the corresponding position within the exposure window of certain lines, laser lines will be missing in those lines, manifesting as broken or jagged lines. Matching the fast-axis scan with the line exposure windows ensures that the integrated result of the point laser reflection light is recorded within the exposure window of each line, thus forming continuous laser line features in a single frame image and improving the integrity and stability of 3D reconstruction under rolling shutter conditions.

[0047] On the other hand, when the imaging unit is a global shutter imaging unit, the oscillation frequency of the fast axis in the dual-axis galvanometer... for:

[0048] in, This refers to the exposure time per frame of the global shutter. The laser line length coverage factor is the ratio of the actual scanned laser line length to the maximum scannable laser line length within the imaging unit's field of view; K is the scanning waveform coefficient: K=2 for reciprocating scanning and K=1 for unidirectional scanning. This ensures that the laser point completes at least one full-range scan within the exposure cycle, thereby synthesizing a continuous virtual laser line within a single frame image. Understandably, a global shutter provides synchronized exposure across the entire frame: the entire frame shares the same exposure time window. If the fast axis does not achieve sufficient coverage within this window, the line feature may exhibit localized uneven brightness or incomplete line segments. By defining the matching relationship between the fast axis scan and the exposure time, the point laser can complete the coverage of the target line segment during a single frame exposure, thereby forming a more uniform and continuous laser line feature in a single frame image, improving the quality of laser feature extraction and the accuracy of subsequent multi-view geometric calculations.

[0049] In one embodiment of the present invention, the specific process of "extracting the pixel coordinate information corresponding to the laser feature from the image data acquired by each imaging unit" in step S130 can be further described in conjunction with the following description. It includes: A sub-pixel center localization algorithm is used to extract the sub-pixel level center coordinate sequence of the laser line feature, and the pixel coordinate information includes the sub-pixel level center coordinate sequence.

[0050] Before 3D reconstruction, it is necessary to extract the subpixel-level center coordinates of the laser feature from the acquired image data. These coordinates can represent the pixel position of the center of the laser feature on the corresponding image.

[0051] In addition to laser reflections, image data may also contain interference from ambient light, multiple reflections, and target object texture noise. Therefore, before extracting the sub-pixel-level center coordinate sequence, the image needs to be filtered to enhance the contrast between the continuous laser line features and the background image, facilitating further image segmentation.

[0052] Specifically, the filtering methods mentioned above include, but are not limited to, Gaussian filtering, median filtering, and bandpass filtering, which will not be elaborated here.

[0053] Furthermore, after feature enhancement, the image is segmented to determine the pixel regions where the laser features are located. Segmentation methods include, but are not limited to, global thresholding, adaptive thresholding, gradient-based edge detection segmentation, and region extraction based on connected component analysis. The segmentation results in one or more candidate laser regions, providing constraints for subsequent center localization.

[0054] Next, after obtaining the candidate region of the laser line, the center coordinates of the laser feature are determined by a sub-pixel localization algorithm. Sub-pixel localization methods include, but are not limited to: intensity distribution weighting method, Gaussian function fitting method, polynomial or parabolic fitting method, edge symmetry analysis method, ridge detection method, etc.

[0055] The following description uses a continuous laser line as an example to illustrate laser characteristics. Employing a intensity distribution weighted method, a row of pixels can be selected from the segmented laser region. The grayscale values ​​of all pixels belonging to the continuous laser region within that row are statistically analyzed. Using these grayscale values ​​as weights, a weighted average position is calculated. The calculated floating-point coordinates are then used as the sub-pixel center position of the continuous laser line. This process is repeated row by row along the entire continuous laser line, forming a continuous sequence of sub-pixel-level center coordinates. This sequence of sub-pixel-level center coordinates can then be used as direct input data for subsequent triangulation.

[0056] After determining the subpixel-level center coordinate sequence, it is also necessary to perform homology matching, that is, to determine which subpixel-level center coordinates in different imaging unit images correspond to the same physical point on the surface of the target object, so as to ensure that the pixel coordinate pairs used in subsequent triangulation have physical homology.

[0057] Specifically, multi-view homology matching typically starts with geometric constraints to establish a strongly constrained search range. Since the pose (extrinsic parameters) of each imaging unit relative to one of the master imaging units is already calibrated, the system can establish epipolar geometric relationships between the feature pixel positions in the image corresponding to one imaging unit and the imaging plane of another imaging unit: when a candidate laser feature point appears in the image corresponding to one imaging unit, it cannot appear at an arbitrary position in the image corresponding to another imaging unit, but should fall near the corresponding epipolar line. Therefore, searching for candidate points that meet the conditions within the constrained region near the epipolar line significantly reduces the number of mismatch combinations, providing more accurate pixel coordinates for subsequent triangulation.

[0058] It should be noted that the main imaging unit refers to the reference camera used as the coordinate system reference in a multi-camera system.

[0059] In situations where pseudo-matches may still exist even when using only epipolar geometry, point lasers form continuous laser lines during fast-axis scanning. Different positions of this line in a single frame of the image correspond to different time slices or phases of the scan trajectory. If a candidate corresponding point falls near the epipolar line, but its relative order or phase consistency in the scan sequence does not conform to the scanning pattern, it can be judged as a pseudo-feature and eliminated. On the other hand, the continuous laser line should present a continuous, smooth, or at least locally coherent central trajectory on the image. Therefore, homologous matching should have a continuous changing relationship between adjacent sampling points. Isolated jump points or discontinuous correspondences can also be used to eliminate pseudo-matches. Through the above methods, a set of reliable multi-view homologous pixel coordinate sequences that can be directly used for triangulation can be output, thereby ensuring the accuracy and stability of 3D reconstruction.

[0060] In one embodiment of the present invention, the specific process of step S140, which involves "performing multi-view triangulation calculations based on the geometric relationship between the at least two imaging units and the pixel coordinate information to determine the three-dimensional spatial points corresponding to the laser features; if the three-dimensional spatial points are multiple sets, then determining the three-dimensional point cloud data based on the multiple sets of three-dimensional spatial points," can be further explained in conjunction with the following description.

[0061] In this embodiment, the step of performing multi-view triangulation calculations based on the geometric relationship between the at least two imaging units and the pixel coordinate information to determine the three-dimensional spatial point corresponding to the laser feature includes: The pixel coordinate information is subjected to distortion correction processing to obtain the corresponding distorted pixel coordinates; Based on the distortion-corrected pixel coordinates and the pose relationship of at least two imaging units relative to a unified reference coordinate system, a geometric projection constraint between two-dimensional pixel coordinates and three-dimensional spatial coordinates is constructed. Based on the geometric projection constraints, multi-view triangulation is performed on the pixel coordinates of different imaging units to determine the three-dimensional spatial points corresponding to the features of the continuous laser line.

[0062] It should be noted that the pixel coordinates of different imaging units are pixel coordinates of the same origin. Pixel coordinates of the same origin refer to the pixel coordinates of the corresponding pixel points obtained by different imaging units imaging the same scan trajectory spatial point at the same exposure time.

[0063] Specifically, distortion correction processing is performed on the pixel coordinate information to obtain the corresponding distortion-free pixel coordinates. Based on step S130, the sub-pixel-level center coordinates of the continuous laser line features in the image data corresponding to each imaging unit are obtained. However, due to radial and tangential distortion of the imaging unit's lens, directly using these sub-pixel-level center coordinates for triangulation will lead to accumulated spatial calculation errors. Therefore, distortion correction is needed for these sub-pixel-level center coordinates. The general process is as follows: The subpixel-level center coordinates are mapped to the normalized image plane by inverse transformation of the intrinsic parameter matrix. A nonlinear distortion mapping relationship is established using radial distortion coefficients k1, k2, k3 and tangential distortion coefficients p1, p2. Normalized distortion-free pixel coordinates are obtained by using numerical iteration or pre-computed lookup tables. .

[0064] First, the sub-pixel-level center coordinates are mapped to the normalized image plane, and then the normalized distortion-free pixel coordinates are... Subpixel-level center coordinates affected by distortion Satisfying the following nonlinear mapping:

[0065]

[0066] in, .

[0067] In the above formula, These are relatively ideal pixel coordinates for distortion correction and normalization. These are the sub-pixel-level center coordinates normalized to the image plane, affected by distortion. The parameters are initially mapped using the inverse transformation of the intrinsic parameter matrix Ki. k1, k2, and k3 are radial distortion coefficients. They are used to correct barrel or pincushion distortion caused by lens shape defects and are usually proportional to a high power of the distance r from the pixel to the center point. p1 and p2 are tangential distortion coefficients used to correct geometric distortion caused by the non-parallelism between the lens plane and the image sensor (CCD / CMOS) plane. The squared distance from a point in the normalized plane to the origin (i.e. This reflects the evolution of the distortion variable as the field of view increases.

[0068] Specifically, based on the distorted pixel coordinates and the pose relationships of at least two imaging units relative to a unified reference coordinate system, geometric projection constraints between the two-dimensional pixel coordinates and the three-dimensional spatial coordinates are constructed. After distortion correction, the distorted pixel coordinates are obtained. And it is represented as a homogeneous pixel coordinate vector, the expression of which is:

[0069] Let be the sub-pixel homogeneous coordinate vector observed by the i-th imaging unit. This variable is derived from the sub-pixel level center coordinates. The distortion-corrected pixel coordinates are obtained after distortion correction preprocessing.

[0070] Next, the intrinsic parameter matrices of each imaging unit are used. and its extrinsic matrix relative to the imaging unit (unified reference coordinate system) which serves as the main imaging unit. Construct the projection matrix:

[0071] Here, the 3×4 projection matrix of the i-th imaging unit is given. This matrix is ​​the intrinsic parameter matrix of the imaging unit. With the extrinsic parameter matrix (pose matrix) The product of these two projection matrices describes the geometric mapping between points in three-dimensional space and points in two-dimensional subpixels.

[0072] Based on three-dimensional space points Homogeneous pixel coordinate vector Construct the geometric constraint projection relationship, the expression of which is:

[0073] in, Let be the homogeneous coordinate vector of the three-dimensional space point to be solved, representing the real spatial position of the feature point on the surface of the target workpiece in the main imaging unit coordinate system (world coordinate system); S represents the projection scale factor of the i-th imaging unit. Physically, it corresponds to the optical axis depth value of the spatial point Pw in the coordinate system of the i-th imaging unit. The projection scale factor S will be eliminated in subsequent 3D point calculations. The subscript i is the imaging unit index (i=1,2,…,M), which indicates that the above geometric constraint relationship holds for any imaging unit within the system that observes the same feature point.

[0074] Specifically, based on the geometric projection constraints, multi-view triangulation is performed on the pixel coordinates of different imaging units to determine the three-dimensional spatial points corresponding to the continuous laser line features. Specifically, multi-view triangulation means that when at least two imaging units complete homogeneous matching for the same laser feature in their corresponding image data, two or more pairs of pixel coordinates satisfying homogeneity can be obtained. Using the cross product property m×(MPw)=0, the geometric constraint projection relationship constructed above is transformed into a homogeneous linear equation system, the expression of which is:

[0075] in, and Refers to the sub-pixel homogeneous coordinate vectors extracted from the image corresponding to the i-th imaging unit and the image corresponding to the j-th imaging unit; and These are the 3×4 projection matrices for the images corresponding to the i-th and j-th imaging units, respectively. These matrices couple the intrinsic parameters and spatial pose information of the imaging unit; × vector cross product operator. Based on... Homogeneous pixel coordinate vector The vectors are geometrically collinear with the vectors of the projected spatial points, so their cross product is always a zero vector.

[0076] It should be noted that the above system of equations constitutes an overdetermined constraint system. By using the least squares method (such as singular value decomposition (SVD)) to find the intersection point in space that minimizes the error, the three-dimensional spatial point can be solved. This is used to complete the three-dimensional curve calculation of continuous laser line features in two-dimensional image data.

[0077] Under the constraints of multiple imaging units, the positions of real spatial points are solved through geometric consistency, realizing the inverse calculation from two-dimensional pixel coordinates to three-dimensional physical space coordinates. On the other hand, when the system contains M imaging units, K pairs of effective imaging units can be selected, multiple triangulation measurements can be performed, and the results can be weighted and fused to further reduce random errors.

[0078] In this embodiment, when there are multiple sets of three-dimensional spatial points, it is understood that these multiple sets of three-dimensional spatial points can be obtained in surface scanning mode. Determining three-dimensional point cloud data based on multiple sets of three-dimensional spatial points includes: At each scanning position, multiple sets of three-dimensional spatial points calculated by different imaging units are transformed to the reference coordinate system of the main imaging unit. In the reference coordinate system, adjacent three-dimensional spatial points with a spatial distance less than a preset threshold are determined to be the same spatial point, thus obtaining a fused three-dimensional spatial point set corresponding to each scanning position; Multiple scan locations corresponding to the fused three-dimensional spatial point set are merged in the reference coordinate system of the main imaging unit to generate three-dimensional point cloud data.

[0079] In the above embodiments, after completing the multi-view triangulation calculation, the system obtains the set of three-dimensional spatial points corresponding to different scanning positions after slow axis stepping and different imaging units are solved respectively. In order to generate complete and spatially consistent three-dimensional point cloud data, it is necessary to perform layered fusion processing on the multiple sets of three-dimensional spatial points.

[0080] Before fusion, all 3D spatial points at each scanning position must be uniformly transformed to the reference coordinate system of the main imaging unit. Specifically, the extrinsic parameter matrix of the main imaging unit is used as a unified spatial reference. The coordinates of the 3D spatial points calculated by each imaging unit are transformed so that they are all expressed in the same physical coordinate system, eliminating the spatial offset caused by the difference in coordinate reference between different imaging unit pairs.

[0081] Furthermore, in the reference coordinate system of the main imaging unit, the Euclidean distance between each 3D spatial point is calculated. When the spatial distance between two or more 3D spatial points is less than a preset threshold, these 3D spatial points are determined to correspond to the same physical spatial point. For multiple 3D spatial points determined to be the same physical spatial point, a weighted average or least squares optimization method can be used to merge them to obtain a fused representative 3D spatial point. Through this process, random errors caused by image noise, pixel quantization errors, and viewpoint differences can be suppressed, while compensating for observation inconsistencies in local occlusion areas, thereby obtaining the fused 3D spatial point set corresponding to the scanning position.

[0082] Subsequently, the system obtains a set of fused 3D spatial points corresponding to multiple scanning positions. Since the slow axis changes the scanning position according to a preset step angle between adjacent exposure times, each scanning position corresponds to a different spatial section of the target object's surface. The set of fused 3D spatial points corresponding to each scanning position is spatially merged, that is, the spatial point data of different scanning rows (or different sections) are superimposed under a unified coordinate system, so that the continuous spatial section data expands line by line in the scanning dimension, and finally the 3D point cloud data covering the entire surface of the target object is obtained.

[0083] Specifically, the point cloud data of the entire venue. It consists of the set of three-dimensional spatial points at each scan position generated by the triangulation results of K groups of effective imaging units under N slow-axis step scans, and its expression is:

[0084] in, Full-field synthetic point cloud refers to the set of three-dimensional point clouds with complete geometric shape and unified coordinate system output after a complete scanning cycle; N is the total number of scan steps, and its value is determined by the number of scan lines input. It represents the total number of discrete samples performed by the slow axis in space. n is the scan step index (n=1,2,…,N), representing the specific time slice or specific scan line currently being processed; K is the total number of effective imaging unit pairs. If the imaging unit pair is a binocular imaging unit, its theoretical maximum value is the number of combinations. (i.e., a combination of any 2 units selected from M imaging units), the actual value depends on the number of imaging unit combinations that have overlapping fields of view and meet baseline constraints; K is the imaging unit pair index (k=1,2,…,K), representing the specific imaging unit combination currently involved in the calculation; Local 3D point cloud data refers to the set of 3D coordinate points calculated by the triangulation algorithm from the kth image unit pair at the nth scan step.

[0085] In this embodiment, each three-dimensional spatial point is located in the reference coordinate system of the main imaging unit, has a unified spatial reference, and has undergone redundant fusion processing, resulting in high spatial consistency and measurement stability. This achieves the orderly integration of redundant solution results from multiple imaging units and data from multiple scanning positions, ensuring the integrity, accuracy, and spatial continuity of the three-dimensional reconstruction results.

[0086] In another embodiment, the color image data acquired by the main imaging unit can be used to map each three-dimensional spatial point and its corresponding image pixel color information (RGB value) into a three-dimensional point cloud, so that each three-dimensional point not only contains spatial coordinate information but also color information, forming color three-dimensional point cloud data with physical consistency.

[0087] Based on the intrinsic and extrinsic parameter matrices of the main imaging unit, each three-dimensional spatial point in the three-dimensional point cloud data is projected into the image data of the main imaging unit to determine the pixel position of each three-dimensional spatial point in the target image. Extract the color information of the corresponding pixel position and assign the color information to the corresponding three-dimensional spatial point to generate three-dimensional point cloud data with color information.

[0088] The expression for mapping the above color image data to three-dimensional space is:

[0089] RGB represents the color information assigned to a point in space. Pixel index mapping of color image data for the main imaging unit; The intrinsic parameter matrix of the main imaging unit; The extrinsic parameter matrix of the main imaging unit is set as an identity matrix and a zero vector; The homogeneous coordinates of a spatial point in the reference coordinate system of the main imaging unit.

[0090] Understandably, after the point cloud data has been unified into the reference coordinate system of the main imaging unit, for each three-dimensional spatial point, it is used as the homogeneous coordinate of the spatial point in the reference coordinate system of the main imaging unit and input into the intrinsic parameter matrix of the main imaging unit and projected and mapped to obtain the pixel position of the three-dimensional spatial point in the image data of the main imaging unit. The RGB color value of the corresponding pixel is read through the pixel index mapping of the image data of the main imaging unit, and the RGB color value is associated with the three-dimensional spatial point to generate three-dimensional point cloud data with color information.

[0091] In summary, this embodiment reuses the existing point laser and dual-axis galvanometer in the laser processing equipment, switching the scanning mode before or during processing. This allows the point laser to form a scanning trajectory on the target workpiece surface that can be used for 3D reconstruction, and a multi-view camera captures the corresponding images to complete 3D reconstruction and in-situ colorization. By feeding the reconstructed 3D point cloud data back to the processing control system, workpiece positioning, adaptive adjustment of processing parameters, and online quality monitoring are achieved. Thus, without adding additional projection hardware, an integrated application of laser processing and 3D perception is realized.

[0092] As the device embodiment is basically similar to the method embodiment, the description is relatively simple, and relevant parts can be found in the description of the method embodiment.

[0093] Reference Figure 2 This diagram illustrates a structural block diagram of a multi-view 3D reconstruction device based on dual-axis galvanometer scanning according to an embodiment of this application. The multi-view 3D reconstruction device includes a point laser, a dual-axis galvanometer, and a multi-view camera composed of at least two imaging units. Specifically, it also includes the following modules: Also includes: The control module 110 is used to determine the target scanning mode of the dual-axis galvanometer according to the current three-dimensional measurement task requirements. The target scanning mode is one of the following: single-point scanning mode, single-line scanning mode, or surface scanning mode. The acquisition module 120 is used to control the laser emitted by the point laser to be deflected by the dual-axis galvanometer and then irradiate the surface of the target object based on the target scanning mode, and the at least two imaging units acquire image data containing laser features. Extraction module 130 extracts the pixel coordinate information corresponding to the laser feature from the image data collected by each imaging unit; The determination module 140 performs multi-view triangulation calculations based on the geometric relationship between the at least two imaging units and the pixel coordinate information to determine the three-dimensional spatial points corresponding to the laser features; if there are multiple sets of three-dimensional spatial points, then the three-dimensional point cloud data is determined based on the multiple sets of three-dimensional spatial points.

[0094] Reference Figure 3 The illustration shows a computer electronic device for implementing a multi-view three-dimensional reconstruction method based on dual-axis galvanometer scanning, which may specifically include the following: The aforementioned computer electronic device 1 is manifested in the form of a general-purpose computing device. The components of the computer electronic device 1 may include, but are not limited to: one or more processors or processing units 3, memory 8, and a bus 4 connecting different system components (including memory 8 and processing unit 3).

[0095] Bus 4 represents one or more of several bus architectures, including memory buses or memory controllers, peripheral buses, graphics acceleration ports, processors, or local buses using any of the various bus architectures. For example, these architectures include, but are not limited to, the Industry Standard Architecture (ISA) bus, the Micro Channel Architecture (MAC) bus, the Enhanced ISA bus, the Audio / Video Electronics Standards Association (VESA) local bus, and the Peripheral Component Interconnect (PCI) bus.

[0096] Computer electronic device 1 typically includes a variety of computer system readable media. These media can be any available media that can be accessed by computer electronic device 1, including volatile and non-volatile media, removable and non-removable media.

[0097] Memory 8 may include computer system readable media in the form of volatile memory, such as random access memory 9 and / or cache memory 10. Computer electronic device 1 may further include other removable / non-removable, volatile / non-volatile computer system storage media. By way of example only, storage system 11 may be used to read and write non-removable, non-volatile magnetic media (commonly referred to as a "hard disk drive"). Although Figure 3 As not shown, a disk drive for reading and writing to a removable non-volatile disk (such as a "floppy disk") and an optical disk drive for reading and writing to a removable non-volatile optical disk (such as a CD-ROM, DVD-ROM, or other optical media) may be provided. In these cases, each drive may be connected to bus 4 via one or more data media interfaces. The memory may include at least one program product having a set (e.g., at least one) of program modules 13 configured to perform the functions of the embodiments of this application.

[0098] A program / utility 12 having a set (at least one) of program modules 13 may be stored, for example, in memory. Such program modules 13 include—but are not limited to—an operating system, one or more application programs, other program modules 13, and program data. Each or some combination of these examples may include an implementation of a network environment. Program modules 13 typically perform the functions and / or methods described in the embodiments of this application.

[0099] The computer electronic device 1 can also communicate with one or more external devices 2 (e.g., keyboard, pointing device, display 7, camera, etc.), and with one or more devices that enable an operator to interact with the computer electronic device 1, and / or with any device that enables the computer electronic device 1 to communicate with one or more other computing devices (e.g., network card, modem, etc.). This communication can be performed through the I / O interface 6. Furthermore, the computer electronic device 1 can also communicate with one or more networks (e.g., local area network (LAN)), wide area network (WAN), and / or public networks (e.g., the Internet) through the network adapter 5. Figure 3 As shown, network adapter 5 communicates with other modules of computer electronic device 1 via bus 4. It should be understood that, although... Figure 3 Not shown, it may be combined with other hardware and / or software modules, including but not limited to: microcode, device drivers, redundant processing unit 3, external disk drive array, RAID system, tape drive and data backup storage system 11, etc.

[0100] The processing unit 3 executes various functional applications and data processing by running programs stored in memory 8, such as implementing a method for multi-view three-dimensional reconstruction based on dual-axis galvanometer scanning provided in the embodiments of this application.

[0101] That is, when the processing unit 3 executes the above program, it performs the following: Based on the current 3D measurement task requirements, it determines the target scanning mode of the dual-axis galvanometer, which is one of a single-point scanning mode, a single-line scanning mode, or a surface scanning mode; based on the target scanning mode, it controls the laser emitted by the point laser to be deflected by the dual-axis galvanometer and then illuminate the surface of the target object, and the at least two imaging units acquire image data containing laser features; it extracts the pixel coordinate information corresponding to the laser features from the image data acquired by each imaging unit; based on the geometric relationship between the at least two imaging units and the pixel coordinate information, it performs multi-view triangulation calculation to determine the 3D spatial point corresponding to the laser feature; if there are multiple sets of 3D spatial points, it determines the 3D point cloud data based on the multiple sets of 3D spatial points.

[0102] In the embodiments of this application, a computer-readable storage medium is also provided, on which a computer program is stored, which, when executed by a processor, implements a method for multi-view three-dimensional reconstruction based on dual-axis galvanometer scanning as provided in all embodiments of this application.

[0103] That is, when the program is executed by the processor, it performs the following: Based on the current 3D measurement task requirements, it determines the target scanning mode of the dual-axis galvanometer, which is one of a single-point scanning mode, a single-line scanning mode, or a surface scanning mode; based on the target scanning mode, it controls the laser emitted by the point laser to be deflected by the dual-axis galvanometer and then illuminate the surface of the target object, and the at least two imaging units acquire image data containing laser features; it extracts the pixel coordinate information corresponding to the laser features from the image data acquired by each imaging unit; based on the geometric relationship between the at least two imaging units and the pixel coordinate information, it performs multi-view triangulation calculations to determine the 3D spatial point corresponding to the laser feature; if there are multiple sets of 3D spatial points, it determines 3D point cloud data based on the multiple sets of 3D spatial points.

[0104] Any combination of one or more computer-readable media may be used. A computer-readable medium can be a computer-readable signal medium or a computer-readable storage medium. A computer-readable storage medium can be, for example—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of computer-readable storage media include: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in connection with an instruction execution system, apparatus, or device.

[0105] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including—but not limited to—electromagnetic signals, optical signals, or any suitable combination thereof. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, capable of transmitting, propagating, or transmitting programs for use by or in connection with an instruction execution system, apparatus, or device.

[0106] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof. These programming languages ​​include object-oriented programming languages ​​such as Java, Smalltalk, and C++, as well as conventional procedural programming languages ​​such as C or similar languages. The program code can be executed entirely on the operator's computer, partially on the operator's computer, as a standalone software package, partially on the operator's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the operator's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider). The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably.

[0107] Although preferred embodiments of the present application have been described, those skilled in the art, upon learning the basic inventive concept, can make other changes and modifications to these embodiments. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments as well as all changes and modifications falling within the scope of the embodiments of the present application.

[0108] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes said element.

[0109] The above provides a detailed description of the multi-view three-dimensional reconstruction method and apparatus based on dual-axis galvanometer scanning provided in this application. Specific examples have been used to illustrate the principles and implementation methods of this application. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of this application. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of this application. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. A multi-view 3D reconstruction method based on dual-axis galvanometer scanning, characterized in that, The method is performed by a multi-view 3D reconstruction device, which includes a point laser, a biaxial galvanometer, and at least two imaging units; the method includes the following steps: Based on the current requirements of the 3D measurement task, the target scanning mode of the dual-axis galvanometer is determined. The target scanning mode is one of the following: single-point scanning mode, single-line scanning mode, or surface scanning mode. Based on the target scanning mode, the laser emitted by the point laser is deflected by the biaxial galvanometer and then irradiates the surface of the target object, and image data containing laser features is acquired by the at least two imaging units; Extract the pixel coordinate information corresponding to the laser feature from the image data acquired by each imaging unit; Based on the geometric relationship between the at least two imaging units and the pixel coordinate information, multi-view triangulation calculation is performed to determine the three-dimensional spatial point corresponding to the laser feature; if there are multiple sets of three-dimensional spatial points, then the three-dimensional point cloud data is determined based on the multiple sets of three-dimensional spatial points.

2. The multi-view three-dimensional reconstruction method based on dual-axis galvanometer scanning according to claim 1, characterized in that, The step of determining the target scanning mode of the dual-axis galvanometer based on the current requirements of the 3D measurement task includes: When the target scanning mode is single-point scanning mode, the fast axis and slow axis of the dual-axis galvanometer are controlled to remain stationary or deflect to a preset angle so that the laser emitted by the point laser forms a single laser point on the surface of the target object. Control at least two imaging units to acquire image data containing laser features; wherein, the laser features are single laser point features; Multi-view triangulation is performed based on the pixel coordinates of a single laser point feature in the image data to determine the single three-dimensional spatial coordinates; or, When the target scanning mode is single-line scanning mode, the fast axis of the dual-axis galvanometer is controlled to scan within the single exposure time of each imaging unit, while the slow axis maintains a fixed angle; so that the laser emitted by the point laser forms a continuous laser line along the surface of the target object within the exposure time. Control at least two imaging units to acquire image data containing laser features; wherein, the laser features are continuous laser line features; Multi-view triangulation is performed based on the pixel coordinates of continuous laser line features in image data to determine a set of three-dimensional spatial points; or, When the target scanning mode is area scanning mode, the slow axis of the dual-axis galvanometer is controlled to change the scanning position according to a preset step angle between adjacent exposure times; After multiple slow-axis steps, multiple sets of three-dimensional spatial points are determined, and the coordinates of these multiple sets of three-dimensional spatial points are unified and fused to generate three-dimensional point cloud data that characterizes the overall shape of the target object.

3. The multi-view three-dimensional reconstruction method based on dual-axis galvanometer scanning according to claim 2, characterized in that, When the target scanning mode is single-line scanning mode, controlling the fast axis of the dual-axis galvanometer to scan within a single exposure time of each imaging unit includes: When the imaging unit is a rolling shutter imaging unit, the oscillation frequency of the fast axis in the biaxial galvanometer is... for: ; in, For row scan cycles, To ensure effective exposure of rows, K is the laser line length coverage factor, and K is the scanning waveform coefficient: K=2 for reciprocating scanning and K=1 for unidirectional scanning; When the imaging unit is a global shutter imaging unit, the oscillation frequency of the fast axis in the dual-axis galvanometer is... for: ; in, This refers to the exposure time per frame of the global shutter. K is the laser line length coverage factor, and K is the scanning waveform coefficient: K=2 for reciprocating scanning and K=1 for unidirectional scanning.

4. The multi-view three-dimensional reconstruction method based on biaxial galvanometer scanning according to claim 1, characterized in that, Extracting the pixel coordinate information corresponding to the laser feature from the image data acquired by each imaging unit includes: A sub-pixel center localization algorithm is used to extract the sub-pixel level center coordinate sequence of the laser line feature, and the pixel coordinate information includes the sub-pixel level center coordinate sequence.

5. The multi-view three-dimensional reconstruction method based on dual-axis galvanometer scanning according to claim 1, characterized in that, The step of performing multi-view triangulation calculations based on the geometric relationship between the at least two imaging units and the pixel coordinate information to determine the three-dimensional spatial point corresponding to the laser feature includes: The pixel coordinate information is subjected to distortion correction processing to obtain the corresponding distorted pixel coordinates; Based on the distortion-corrected pixel coordinates and the pose relationship of at least two imaging units relative to a unified reference coordinate system, a geometric projection constraint between two-dimensional pixel coordinates and three-dimensional spatial coordinates is constructed. Based on the geometric projection constraints, multi-view triangulation is performed on the pixel coordinates of different imaging units to determine the three-dimensional spatial points corresponding to the features of the continuous laser line.

6. The multi-view three-dimensional reconstruction method based on dual-axis galvanometer scanning according to claim 1, characterized in that, If the three-dimensional spatial points are in multiple sets, then the three-dimensional point cloud data is determined based on the multiple sets of three-dimensional spatial points, including: At each scanning position, multiple sets of three-dimensional spatial points calculated by different imaging units are transformed to the reference coordinate system of the main imaging unit. In the reference coordinate system, adjacent three-dimensional spatial points with a spatial distance less than a preset threshold are determined to be the same spatial point, thus obtaining a fused three-dimensional spatial point set corresponding to each scanning position; Multiple scan locations corresponding to the fused three-dimensional spatial point set are merged in the reference coordinate system of the main imaging unit to generate three-dimensional point cloud data.

7. The multi-view three-dimensional reconstruction method based on biaxial galvanometer scanning according to claim 6, characterized in that, Also includes: Based on the intrinsic and extrinsic parameter matrices of the main imaging unit, each three-dimensional spatial point in the three-dimensional point cloud data is projected into the image data of the main imaging unit to determine the pixel position of each three-dimensional spatial point in the target image. Extract the color information of the corresponding pixel position and assign the color information to the corresponding three-dimensional spatial point to generate three-dimensional point cloud data with color information.

8. A multi-view three-dimensional reconstruction device based on dual-axis galvanometer scanning, characterized in that, The device includes a point laser, a biaxial galvanometer, and at least two imaging units, and also includes: The control module is used to determine the target scanning mode of the dual-axis galvanometer according to the current three-dimensional measurement task requirements. The target scanning mode is one of the following: single-point scanning mode, single-line scanning mode, or surface scanning mode. The acquisition module is used to control the laser emitted by the point laser to be deflected by the dual-axis galvanometer and then irradiate the surface of the target object based on the target scanning mode, and the at least two imaging units acquire image data containing laser features. The extraction module is used to extract the pixel coordinate information corresponding to the laser feature from the image data acquired by each imaging unit; The determination module is used to perform multi-view triangulation calculations based on the geometric relationship between the at least two imaging units and the pixel coordinate information to determine the three-dimensional spatial points corresponding to the laser features; if there are multiple sets of three-dimensional spatial points, then the three-dimensional point cloud data is determined based on the multiple sets of three-dimensional spatial points.

9. A computer electronic device, characterized in that, It includes a processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein when the computer program is executed by the processor, it implements the steps of the multi-view three-dimensional reconstruction method based on biaxial galvanometer scanning as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, which, when executed by a processor, implements the steps of the multi-view three-dimensional reconstruction method based on biaxial galvanometer scanning as described in any one of claims 1 to 7.

Images