Photogrammetry control method and device based on laser galvanometer and electronic equipment

Abstract

The invention provides a photogrammetry control method and device based on a laser galvanometer and electronic equipment, and belongs to the technical field of photogrammetry, and the method comprises the steps: controlling the laser galvanometer to deflect according to a preset scanning path in a single exposure period of a camera, so as to form n laser gratings in a single frame image collected by the camera; extracting feature information of each laser grating from the single-frame image, wherein the feature information comprises pixel coordinates in a space coordinate system; and calculating three-dimensional coordinate points of the surface of the object to be measured according to the real-time deflection angles of the laser galvanometer when the laser galvanometer projects the laser gratings, the feature information of the laser gratings and pre-calibrated camera parameters, and summarizing to generate three-dimensional point cloud data. According to the invention, the precision of photogrammetry is improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer, in particular to a photogrammetry control method and device based on laser galvanometer and electronic equipment. BACKGROUND

[0002] Photogrammetry technology is an important method for obtaining three-dimensional geometric information of an object by analyzing its surface image. In the fields of industrial detection, reverse engineering and quality control, three-dimensional measurement methods based on active vision are widely used due to their high precision and fast speed. Among them, the structured light measurement method, especially the technology of projecting laser gratings, has become a mainstream technology due to its high light bar brightness, strong directionality and easy extraction. The basic principle of this method is to project a specific laser pattern (such as a dot, a line or a grating) onto the surface of the measured object. Due to the height variation of the object surface, the laser pattern captured by the camera from another angle will be deformed. By processing this deformation and using the principle of triangulation, the three-dimensional appearance of the object can be reconstructed.

[0003] However, the existing photogrammetry technology based on laser gratings, especially the typical system architecture using fixed lasers and cameras, has several inherent limitations. First, the system calibration process is complex. In order to obtain high-precision measurement results, the relative position and attitude of each laser and each camera in the system must be calibrated extremely accurately. This not only requires strict process control during production and assembly, but also in field applications, the calibration process is tedious and time-consuming, and the long-term stability of the equipment is easily affected by environmental temperature, vibration and other factors, resulting in the need for frequent recalibration. Second, the system has poor flexibility and adaptability. Once the position and angle of the laser and camera are fixed, the effective measurement field of view and depth of field are also fixed. When the size and shape of the measured object exceed the preset range, or the surface geometric features are complex, a single measurement cannot cover completely, and often requires mechanical movement of the equipment or multiple sensor stitching to complete the measurement, which greatly increases the complexity and cost of the system and reduces the measurement efficiency. Third, there is a bottleneck in measurement speed and dynamic characteristics. Due to the dependence on fixed projection patterns, the amount of data obtained by single acquisition is limited. For large-area objects, multiple scans are required by driving the equipment or workpiece through a stepping motor, which takes a long time. More importantly, such systems are difficult to accurately measure moving objects. During camera exposure, there is relative motion between the object and the projected fixed grating pattern, which will cause motion blur in the captured image, severely reducing the feature extraction accuracy, and even leading to measurement failure, so it is usually only suitable for static or quasi-static scenes.

[0004] In summary, the fixed laser grating photogrammetry system in the prior art has obvious deficiencies in calibration complexity, flexibility, measurement efficiency and measurement ability of dynamic objects, resulting in inaccurate measurement results. SUMMARY

[0005] Therefore, it is necessary to provide a laser galvanometer-based photogrammetry control method, device and electronic equipment to solve the technical problem of inaccurate photogrammetry results caused by calibration errors of a multi-camera system in the prior art.

[0006] To solve the above technical problems, in a first aspect, the present application provides a laser galvanometer-based photogrammetry control method, comprising: controlling the laser galvanometer to deflect according to a preset scanning path within a single exposure period of the camera, so as to form n a plurality of laser gratings in a single frame image collected by the camera, n wherein n is a positive integer greater than 1; extracting feature information of each laser grating from the single frame image, the feature information including pixel coordinates in a spatial coordinate system; calculating three-dimensional coordinate points of the surface of the object to be measured according to the real-time deflection angle of the laser galvanometer when projecting each laser grating, the feature information of each laser grating, and the pre-calibrated camera parameters, and generating three-dimensional point cloud data by summarizing.

[0007] In a possible implementation, before controlling the laser galvanometer to deflect according to the preset scanning path, the method further comprises: obtaining appearance feature information of the object to be measured, the appearance feature information including object size and surface shape; generating the preset scanning path adapted to the object to be measured according to the appearance feature information by using a path planning algorithm.

[0008] In a possible implementation, the step of controlling the laser galvanometer to deflect according to the preset scanning path comprises: controlling the laser galvanometer to deflect the laser beam to n a plurality of predetermined positions in sequence, and controlling the laser beam to remain stable at each predetermined position; completing the exposure of the single frame image by the camera during the stable residence of the laser beam at the plurality of predetermined positions, so as to obtain a single frame image containing n a plurality of laser gratings.

[0009] In a possible implementation, the step of forming a plurality of laser gratings within a single exposure period of the camera comprises: According to the performance parameters of the camera and the laser galvanometer, the number of laser raster stripes in a single frame image is determined by the following formula n : T≥ n×(T 1 +T 2 ) wherein, T T is the exposure time of a single frame image, T 1 is the residence time of a single raster stripe at a predetermined position, T 2 is the rotation time of the laser galvanometer between adjacent raster stripes.

[0010] In a possible implementation, the step of controlling the laser galvanometer to deflect according to the preset scanning path further includes: controlling the laser galvanometer to deflect during the acquisition of a plurality of continuous frames of images; using a fixed step size to cause the laser raster stripes in adjacent frames of images to have an increment in position on the surface of the object to be measured.

[0011] In a possible implementation, the object to be measured is in a state of motion, and the method further includes: acquiring motion trajectory information of the object to be measured in the state of motion; adjusting the deflection angle of the laser galvanometer in real time according to the motion trajectory information, so that the n laser raster stripes formed in a single exposure period of the camera can be projected onto the target region on the surface of the object to be measured following the motion of the object to be measured; extracting feature information and calculating three-dimensional coordinate points from the acquired single frame of image containing the projected laser raster stripes, to generate three-dimensional point cloud data of the object in the state of motion.

[0012] In a possible implementation, the step of calculating the three-dimensional coordinate points on the surface of the object to be measured according to the real-time deflection angle of the laser galvanometer when each laser raster stripe is projected, the feature information of each laser raster stripe, and the camera parameters calibrated in advance includes: determining the direction vector of the laser beam in the spatial coordinate system according to the real-time deflection angle; determining the direction vector of the corresponding camera imaging ray in the spatial coordinate system according to the pixel coordinates and the camera parameters; based on the direction vector of the laser beam and the ray vector of the camera imaging, the three-dimensional coordinate points corresponding to the feature points of the laser raster stripe are calculated by solving the intersection point of the two vectors in space.

[0013] In a second aspect, the present application further provides a photogrammetry control device based on a laser galvanometer, comprising an image acquisition module, configured to control the laser galvanometer to deflect according to a preset scanning path in a single exposure period of a camera, so as to form a plurality of laser gratings in a single frame of image collected by the camera n a laser grating, n is a positive integer greater than 1; a feature extraction module, configured to extract feature information of each laser grating from the single frame of image, the feature information including pixel coordinates in a spatial coordinate system; a three-dimensional mapping module, configured to calculate three-dimensional coordinate points of a surface of a to-be-measured object according to real-time deflection angles of the laser galvanometer when projecting each laser grating, the feature information of each laser grating, and pre-calibrated camera parameters, and generate three-dimensional point cloud data by summarizing.

[0014] In a third aspect, the present application further provides an electronic device, comprising a memory and a processor, wherein the memory is configured to store a program; the processor is coupled to the memory and configured to execute the program stored in the memory, so as to implement the steps of the laser galvanometer-based photogrammetry control method in any of the implementation manners described above.

[0015] In a fourth aspect, the present application further provides a computer-readable storage medium for storing computer-readable programs or instructions, which can implement the steps of the laser galvanometer-based photogrammetry control method in any of the implementation manners described above when executed by a processor.

[0016] The laser galvanometer-based photogrammetry control method provided by the present application controls the laser galvanometer to deflect rapidly within a single exposure time of a camera, so that a single laser beam dynamically forms a plurality of gratings on the surface of an object, and a single frame of image containing these gratings is collected synchronously, and then the three-dimensional coordinates of the surface of the object are directly calculated in combination with the real-time deflection angles of the galvanometer and the camera parameters. The method realizes the use of single exposure to obtain multiple gratings, thereby improving the data acquisition efficiency, and the direct calculation method based on accurate geometric relationship avoids the matching error in a multi-camera system, which is conducive to improving the efficiency and accuracy of three-dimensional measurement. BRIEF DESCRIPTION OF DRAWINGS In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without any creative effort based on these drawings.

[0017] Figure 1 An embodiment flowchart of the laser galvanometer-based photogrammetry control method provided by the present application; Figure 2This is a schematic flowchart of another embodiment of the photogrammetry control method based on a laser galvanometer of the present invention; Figure 3 For the present invention Figure 1 A schematic diagram of an embodiment of S101; Figure 4 This is an imaging timing diagram of an embodiment of the photogrammetric control method based on a laser galvanometer of the present invention; Figure 5 For the present invention Figure 1 Another embodiment of S101 is illustrated in the flowchart. Figure 6 This is a schematic flowchart of another embodiment of the photogrammetry control method based on a laser galvanometer of the present invention; Figure 7 For the present invention Figure 1 A schematic diagram of an embodiment of S103; Figure 8 A schematic diagram of an embodiment of the photogrammetry control device based on a laser galvanometer provided by the present invention; Figure 9 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0019] In the description of the embodiments of the present invention, unless otherwise stated, "multiple" means two or more. "And / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone.

[0020] The terms "first," "second," etc., used in the embodiments of this invention are for descriptive purposes only and should not be construed as indicating or implying their relative importance or implicitly specifying the number of technical features indicated. Therefore, a technical feature defined with "first" or "second" may explicitly or implicitly include at least one of that feature.

[0021] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0022] Before demonstrating the embodiments, the following terms will be explained.

[0023] Photogrammetry ( Photogrammetry (3D) is a technique that extracts measurement information from multiple images to obtain the three-dimensional coordinates of an object, and is often used in 3D reconstruction and measurement.

[0024] Laser galvanometer ( Laser Galvanometer ), a method utilizing galvanometers ( galvanometer A device that rapidly deflects a laser beam, enabling precise scanning and positioning of the laser beam on the surface of an object.

[0025] Image acquisition module ( Image Acquisition Module The system is responsible for acquiring images of the surface of an object illuminated by a laser pattern, and typically includes a camera and optical components.

[0026] Galvanometer control ( Galvanometer Control The laser beam scans the surface of an object by controlling the deflection angle of the galvanometer through the drive circuit.

[0027] Synchronous control ( Synchronization Control It precisely coordinates the timing of laser projection and camera exposure to ensure that image acquisition is completed at the moment the laser pattern stabilizes, thus avoiding motion blur.

[0028] 3D computation ( 3D Calculation Based on camera parameters and laser deflection angle, the three-dimensional coordinates of points on the object's surface are calculated using triangulation or forward intersection principles.

[0029] Point cloud generation ( Point Cloud Generation The calculated 3D points are collected into point cloud data for subsequent 3D modeling and analysis.

[0030] Triangulation ( Triangulation (This involves) calculating the three-dimensional position of a point by measuring angles or distances and combining them with geometric relationships.

[0031] grating ( Grating Laser stencils (or laser beams) refer to the stripes or patterns formed on the surface of an object, used to characterize the object's shape.

[0032] Calibration ( CalibrationThis is the process of determining the camera and system's internal parameters (such as focal length and distortion) and external parameters (such as position and attitude) to ensure measurement accuracy.

[0033] Dynamic characteristics ( Dynamic Characteristics The system's ability to measure moving objects, including response speed and resistance to motion ambiguity.

[0034] System calibration ( System Calibration ), for the entire measurement system (including camera, laser galvanometer, etc.).

[0035] Exposure time ( Exposure Time The length of time it takes for the camera sensor to receive light affects image brightness and motion blur.

[0036] Scan path ( Scanning Path The preset trajectory of the laser beam on the surface of the object is controlled by a galvanometer.

[0037] Point cloud data ( Point Cloud Data (), represents the set of three-dimensional spatial points on the surface of an object, and is the basic data for three-dimensional reconstruction.

[0038] 3D model reconstruction ( 3D Model Reconstruction ( ) is the process of generating a 3D model of an object from point cloud data.

[0039] Synchronization timing ( Synchronization Timing The time-coordinated sequence between laser projection and camera exposure ensures the accuracy of data acquisition.

[0040] Motion blur ( Motion Blur Image blurring caused by the relative motion of objects or cameras affects measurement accuracy.

[0041] Joint calibration ( Joint Calibration This allows for the simultaneous calibration of multiple components in the system, such as cameras and laser galvanometers, to simplify the process and improve accuracy.

[0042] This invention provides a photogrammetric control method, apparatus, electronic device, and storage medium based on a laser galvanometer. The technical solutions of the embodiments of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0043] Figure 1 This is a schematic flowchart of an embodiment of the photogrammetry control method based on a laser galvanometer provided by the present invention, as shown below. Figure 1 As shown, the photogrammetric control method based on laser galvanometers includes: S101. Within a single exposure cycle of the camera, control the laser galvanometer to deflect along a preset scanning path to form a single frame image acquired by the camera. n A laser grating, n It is a positive integer greater than 1.

[0044] The core of step S101 in this embodiment lies in controlling the laser galvanometer to complete a specific deflection motion within a single camera exposure time. Specifically, the system controls the galvanometer deflection motor through a drive circuit, causing the incident laser beam to be sequentially projected onto multiple predetermined positions on the surface of the object under test along a preset scanning path. At each predetermined position, the laser beam will maintain a brief, stable dwell time (e.g., the dwell time is...). T 1), then quickly jump to the next position (e.g., jump time is...). T 2). During this process, the control unit precisely synchronizes the timing of laser projection and camera exposure, ensuring that the camera sensor continuously receives light while the laser beam remains stable at each position, thereby completely recording multiple laser gratings projected at different times into the same image frame. The timing relationship of this process can be found in the appendix. Figure 4 As shown.

[0045] For example, when using a frame rate of 100 Hz When using industrial cameras, the single-frame exposure time T Up to 7 ms If the dwell time of a single grating is set... T 1 is 0.05 ms galvanometer position jump time T 2 is 0.05 ms According to the formula T ≥ n ×( T 1+ T 2) Calculate the maximum amount of space a single frame image can hold. n =70 independent gratings. This number far exceeds the projection capability of traditional fixed lasers, reducing calibration processes and errors between multiple devices, which helps improve measurement efficiency and accuracy, while also saving costs.

[0046] This embodiment achieves the technical effect of using a single-line laser to equivalently replace a multi-line laser by transforming dynamic deflection in the time dimension into a spatial grating distribution in a single-frame image. This method not only avoids the calibration complexity caused by the different installation positions of each laser in traditional fixed multi-laser systems, but also significantly improves the system's adaptability to objects of different sizes and shapes through programmed control of the scanning path. Furthermore, since high-density grating data can be acquired in a single exposure, the time required for a complete scan is greatly reduced, improving the efficiency of 3D measurement.

[0047] S102. Extract the feature information of each laser grating from a single frame image. The feature information includes pixel coordinates in the spatial coordinate system.

[0048] Step S102 processes the acquired single-frame image containing multiple laser gratings to extract precise feature information for each grating. Specifically, image processing algorithms are used to identify the position of each laser grating in the image, and its centerline coordinates are extracted as key feature information. Since the light stripes formed by laser gratings on the object surface usually have high contrast, sub-pixel extraction algorithms such as the geometric center method or the gray-scale centroid method can be used to calculate the sub-pixel precision pixel coordinates of each grating feature point in the image coordinate system.

[0049] For example, for a laser grating that appears as a curved shape in an image, by performing grayscale analysis along the normal direction of the light stripe, the precise coordinates of its center position can be calculated point by point, and finally a continuous coordinate sequence of the entire grating can be obtained, with a coordinate accuracy of up to the sub-pixel level (such as 0.05 pixels).

[0050] This embodiment extracts the precise position information of multiple gratings directly from a single-frame image, providing complete input data for subsequent 3D calculations. Since all grating coordinates originate from a single-frame image acquired at the same time, temporal consistency of the data is ensured, avoiding matching errors caused by differences in image acquisition time in traditional multi-camera systems. Simultaneously, the sub-pixel level extraction precision, combined with the high data density of multiple gratings within a single frame, provides accurate data support for generating high-quality 3D point clouds, thus improving the accuracy and reliability of 3D reconstruction.

[0051] S103. Based on the real-time deflection angle of the laser galvanometer when projecting each laser grating, the characteristic information of each laser grating, and the pre-calibrated camera parameters, calculate the three-dimensional coordinate points on the surface of the object to be measured, and summarize and generate three-dimensional point cloud data.

[0052] This embodiment reconstructs the three-dimensional shape of the object under test through geometric calculations based on the laser grating feature information and system parameters obtained in the preceding steps. Specifically, for each extracted laser grating feature point, the system simultaneously acquires two key parameters: the sub-pixel precision pixel coordinates of the feature point in the image coordinate system, and the real-time deflection angle of the laser galvanometer when projecting the corresponding grating. Combined with the pre-calibrated camera intrinsic and extrinsic parameters (including focal length, principal point coordinates, distortion coefficients, etc.), a complete measurement model is established.

[0053] In the specific calculation process, firstly, based on the real-time deflection angle of the laser galvanometer, the precise direction vector of the corresponding laser beam in the spatial coordinate system is determined through a calibrated spatial mapping relationship. Simultaneously, based on the pixel coordinates of the feature point and camera parameters, the direction vector of the corresponding camera imaging ray in the spatial coordinate system is calculated through back projection. Then, using the forward intersection method from the triangulation principle, the intersection operation of the laser beam direction vector and the camera imaging ray vector is performed in three-dimensional space to calculate the three-dimensional coordinates of the feature point on the surface of the measured object.

[0054] For example, for a given grating feature point, when the laser galvanometer deflection angle is 0.1 radians, the corresponding laser beam direction vector is (0.995, 0.010, 0.100). The imaging ray vector calculated based on pixel coordinates (500.2, 300.5) and camera intrinsic parameters is (0.100, -0.200, 0.975). By solving for the intersection of these two spatial lines, the three-dimensional coordinates of the point (25.36, 15.82, 50.15 mm) can be obtained. After traversing all grating feature points and completing the calculations, all obtained three-dimensional coordinate points are summarized to form a complete three-dimensional point cloud data describing the surface morphology of the object.

[0055] This embodiment avoids the feature matching difficulties in traditional multi-camera stereo vision by directly combining high-precision galvanometer deflection angles with image feature information. Since each 3D point is independently calculated based on precise geometric relationships, and all data originates from a single frame image at the same time, high accuracy and spatiotemporal consistency of the measurement results are ensured. Simultaneously, the multi-grating features contained in a single frame image improve the efficiency and accuracy of 3D measurement, providing reliable data support for subsequent model reconstruction and quality inspection.

[0056] In this embodiment, by controlling the rapid deflection of the laser galvanometer within a single camera exposure time, multiple gratings are dynamically formed on the object surface by a single laser beam. Simultaneously, a single-frame image containing these gratings is acquired. Combined with the real-time deflection angle of the galvanometer and camera parameters, the three-dimensional coordinates of the object surface are directly calculated. This achieves the replacement of the traditional fixed layout of multiple lasers with a single galvanometer system, simplifying the system calibration process and enabling the acquisition of high-density three-dimensional data in a single acquisition. Furthermore, the programmable scanning path significantly improves the adaptability and flexibility of the measurement system, thus enhancing the efficiency and accuracy of three-dimensional measurement.

[0057] In some embodiments of the present invention, please refer to Figure 2 Before step S101, that is, before controlling the laser galvanometer to deflect according to a preset scanning path, the method further includes: S201. Obtain the appearance feature information of the object to be tested, including the object size and surface shape. S201. A path planning algorithm is used to generate a preset scanning path that is adapted to the object under test based on the appearance feature information.

[0058] The appearance feature information is obtained in various ways, such as by reading directly from the product's CAD model or by preliminary scanning through a preset measurement program.

[0059] The path planning algorithms include, but are not limited to, the grid method, the artificial potential field method, the genetic algorithm, and the feature analysis method based on CAD models. The grid method plans the path by discretizing the scanning area into a regular grid, which is suitable for objects with regular shapes; the artificial potential field method guides the scanning path by constructing a virtual potential field, which can effectively avoid invalid areas; the genetic algorithm optimizes the path globally by simulating the natural evolution process, which is suitable for path planning on complex curved surfaces; the feature analysis method based on CAD models directly uses the object's three-dimensional model data and automatically generates the optimal scanning path by analyzing its geometric features (such as curvature and boundaries).

[0060] Specifically, this embodiment determines the coverage area of ​​the laser scan based on the three-dimensional boundary dimensions of the object, and optimizes the distribution density and orientation of the gratings according to the surface curvature variation characteristics. For example, for workpieces with complex curved surface features, the grating density can be automatically increased in areas with greater curvature, while the number of gratings can be appropriately reduced in relatively flat areas, thereby achieving optimal allocation of measurement resources.

[0061] Taking a specific implementation as an example, when the measurement dimension is 300... mm ×200 mm ×50 mm When dealing with complex curved surface workpieces, the path planning algorithm first determines the required scanning area of ​​300 based on the workpiece's three-dimensional dimensions. mm ×200 mm Then, the surface curvature distribution was analyzed, and in areas with a curvature radius less than 10... mm The area with the grating spacing set to 1 mm In regions with gentle curvature, the grating spacing is increased to 3. mm The scanning path generated in this way contains 125 gratings, which ensures the measurement accuracy of key features while effectively controlling the overall data acquisition volume.

[0062] This embodiment achieves intelligent and adaptive measurement by programmatically generating scanning paths adapted to specific objects under test. Compared to traditional fixed laser projection systems, this method can flexibly adjust the measurement strategy according to the actual geometric features of the object under test, avoiding redundant scanning of invalid areas while ensuring the measurement accuracy of key feature areas. This significantly improves the system's applicability and measurement efficiency, making it particularly suitable for 3D inspection needs in flexible production lines with small batches and multiple product varieties.

[0063] In some embodiments of the present invention, please refer to Figure 3 The step S101, which involves controlling the laser galvanometer to deflect according to a preset scanning path, includes: S301, Control the laser galvanometer to sequentially deflect the laser beam to... n A predetermined position is established, and the laser beam is controlled to remain stably stationary at each predetermined position; S302, Using a camera, the exposure of a single frame image is completed while the laser beam is stably stationary at multiple predetermined positions, in order to obtain an image containing... n A single frame image of a laser grating.

[0064] Specifically, the laser galvanometer is controlled to deflect the laser beam sequentially to n The system identifies predetermined locations and controls the laser beam to remain stably stationary at each location. This step involves controlling a galvanometer deflection motor via a drive circuit, precisely deflecting the laser beam to the target position according to a preset sequence. At each predetermined location, the system maintains the laser beam stably stationary for a preset duration. T 1. Ensure that the camera sensor can fully receive the laser energy at that location. For example, when scanning a planar workpiece, the galvanometer deflects the laser beam sequentially to 20 equally spaced angular positions, maintaining a stable dwell time of 0.05 milliseconds at each position.

[0065] Furthermore, by using a camera, the exposure of a single frame image is completed while the laser beam is stably stationary at multiple predetermined positions, in order to obtain an image containing... n A single frame image of a laser grating. In this step, the control unit precisely coordinates the timing relationship between the galvanometer deflection and the camera exposure to ensure the camera operates throughout the entire exposure cycle. T The camera continuously acquires images, integrating the laser spots at each stationary position to create a complete grating pattern. Specifically, the camera's exposure begins at the stable dwell time of the first predetermined position and ends at the completion of the dwell time at the last predetermined position.

[0066] To illustrate with a specific implementation example, when setting the single-frame exposure time... T =7 ms Dwell time of a single grating T 1 = 0.05 ms galvanometer rotation timeT 2 = 0.05 ms At that time, according to the formula T ≥ n ×( T 1+ T 2) Calculations show that a single frame image can form a maximum of 70 independent gratings. During this process, the galvanometer is sequentially deflected to 70 predetermined positions, remaining stably at each position for 0.05 seconds. ms The camera throughout the 7 ms During the exposure period, continuous data acquisition resulted in 70 clear laser gratings in a single frame image.

[0067] This embodiment achieves the technical effect of simultaneously capturing multiple laser gratings in a single frame image by organically combining a time-division projection method with single-frame integral imaging. Since the position of each grating is acquired under stable beam conditions, the impact of motion blur on image quality is effectively avoided, ensuring the accuracy of feature extraction. Simultaneously, the method of acquiring multiple gratings in a single exposure significantly improves data acquisition efficiency. Compared to traditional fixed multi-laser systems, this method achieves adjustable grating number, spacing, and distribution through program control, greatly enhancing the flexibility and adaptability of photogrammetry.

[0068] In some embodiments of the present invention, the step of forming multiple laser gratings within a single exposure cycle of the camera in step S101 includes: Based on the performance parameters of the camera and laser galvanometer, the number of laser grating lines in a single frame image is determined using the following formula. n : T≥ n×(T 1 +T 2 ) in, T The exposure time for a single frame of an image. T 1 represents the dwell time of a single grating at a predetermined position. T 2 represents the rotation time of the laser galvanometer between adjacent gratings.

[0069] Specifically, please refer to Figure 4 , Figure 4 The diagram illustrates the imaging timing of one embodiment of the photogrammetric control method based on a laser galvanometer. In this embodiment, the maximum number of gratings that can be formed within a single frame image is determined based on system hardware performance parameters. Specifically, a relational formula is used based on the performance parameters of the camera and the laser galvanometer. T ≥ n×(T 1 +T 2 ) Perform parameter optimization. Among them, the first... N Frame and the N+1 These are two adjacent frames as an example.

[0070] In practical applications, taking a typical industrial camera as an example, when using 100... Hz At frame rate, single frame exposure time T The maximum value can be set to 7. ms Considering the minimum step response time of mainstream industrial galvanometers T 2 is approximately 0.05 ms Meanwhile, to ensure the imaging quality of the grating, the dwell time of a single grating is... T 1. It must be no less than 0.05 ms Substituting these parameters into the formula yields: n ≤ T / ( T 1 +T 2) = 7 / (0.05+0.05) = 70. Therefore, under this configuration, a single frame image can form a maximum of 70 independent raster lines.

[0071] This embodiment provides a theoretical basis for system parameter configuration by establishing a precise mathematical model. Quantitative analysis based on hardware performance ensures the optimal setting of the number of gratings, making full use of the camera's exposure time while guaranteeing the imaging quality of each grating. This enables the system to automatically optimize operating parameters under different hardware configurations, avoiding image quality degradation caused by an excessive number of gratings and maximizing measurement data density. This significantly improves 3D data acquisition efficiency while maintaining measurement accuracy.

[0072] In some embodiments of the present invention, please refer to Figure 5 The step S101, which involves controlling the laser galvanometer to deflect according to a preset scanning path, further includes: S501, Control the laser galvanometer to deflect during the acquisition of multiple consecutive frames of images; S502. A fixed step size is used to increase the position of the laser grating on the surface of the object under test in adjacent frames.

[0073] Specifically, step S501 coordinates the camera acquisition frequency and the galvanometer deflection rhythm through the control unit to establish a continuous acquisition working mode. Step S502 adopts a fixed step size control method to make the position of the laser grating on the surface of the object under test in adjacent frame images produce regular increments. Specifically, after acquiring each frame image, the laser galvanometer is controlled to deflect by a fixed angle, so that the grating pattern in the next frame image has a definite displacement relative to the previous frame.

[0074] To illustrate with a specific implementation example, when measuring a large workpiece, the galvanometer is configured to operate with a fixed step size during 100 consecutive frame acquisitions. Each frame contains 50 laser gratings, and the inter-frame deflection step size is set to 0.001 radians. In this configuration, the first frame records the 50 gratings at their initial position, the second frame records the 50 gratings offset by 0.001 radians, and so on. In this way, 5000 independent grating data points can be accumulated over 100 frames, and these gratings are evenly distributed over a larger scanning area.

[0075] This embodiment achieves complete coverage of large objects by a small field-of-view measurement system through a continuous multi-frame incremental scanning mechanism. The fixed step size control strategy ensures the uniformity and integrity of data acquisition, avoiding data loss due to motion errors in traditional mechanical scanning methods. Simultaneously, by accumulating data from multiple frames, a 3D point cloud with a density far exceeding that of a single frame can be obtained, which is beneficial for improving the accuracy of the 3D reconstruction results.

[0076] In some embodiments of the present invention, please refer to Figure 6 The object to be measured is in motion, and the method further includes: S601. Obtain the motion trajectory information of the object under test that is in motion; S602. Based on the motion trajectory information, adjust the deflection angle of the laser galvanometer in real time to ensure that the image formed within a single exposure cycle of the camera... n A laser grating that can follow the movement of the object being measured and project onto the target area on its surface; S603. Extract feature information and calculate three-dimensional coordinate points from the acquired single-frame image containing the following laser grating to generate three-dimensional point cloud data of the object in motion.

[0077] In this embodiment, steps S601 to S603 provide a specific implementation method for performing three-dimensional measurement on a moving object. This implementation method first obtains the motion trajectory information of the object to be measured in motion through step S601. This motion trajectory information can come from real-time position and speed data provided by a production line encoder, robot control system, or external motion capture device.

[0078] In step S602, the deflection angle of the laser galvanometer is calculated and adjusted in real time based on the acquired motion trajectory information. Specifically, the control unit predicts the target position of the object during the camera exposure based on its current motion state and adjusts the galvanometer deflection angle accordingly to ensure that the image formed within a single camera exposure cycle is within the target position. nA laser grating can accurately follow a moving object and continuously project onto the target area on its surface. For example, when measuring a workpiece moving linearly at a speed of 1 m / s, the system calculates the position offset in real time based on the speed of movement and adjusts the galvanometer deflection angle accordingly, so that the laser grating can accurately cover the surface of the moving workpiece at each exposure moment.

[0079] Step S603 processes the image data acquired after the projection. Feature information is extracted from the single-frame image containing the laser grating in motion, and combined with the corresponding real-time deflection angle and camera parameters, the three-dimensional coordinates of the moving object are calculated, ultimately generating three-dimensional point cloud data that accurately reflects the shape of the object in motion.

[0080] This embodiment effectively overcomes the image blurring and measurement errors caused by object movement in traditional photogrammetry by employing an active tracking projection strategy. Because the laser grating remains synchronized with the moving object, the relative stability of the grating pattern on the object's surface is ensured during camera exposure, resulting in clear image data. This significantly expands the application scope of 3D measurement technology, enabling its effective application in industrial scenarios such as moving workpiece inspection on production lines and robot guidance, thereby improving the accuracy and reliability of dynamic 3D measurement.

[0081] In some embodiments of the present invention, please refer to Figure 7 In step S103, the step of calculating the three-dimensional coordinate points of the surface of the object under test based on the real-time deflection angle of the laser galvanometer when projecting each laser grating, the characteristic information of each laser grating, and the pre-calibrated camera parameters includes: S701. Determine the direction vector of the laser beam in the spatial coordinate system based on the real-time deflection angle; S702. Based on the pixel coordinates and camera parameters, determine the direction vector of the corresponding camera imaging ray in the spatial coordinate system; S703. Based on the direction vector of the laser beam and the ray vector of the camera imaging, the three-dimensional coordinates of the feature points of the laser grating are calculated by solving the intersection of the two vectors in space.

[0082] Specifically, steps S701 to S703 describe the detailed implementation process of calculating three-dimensional coordinate points based on the principle of triangulation.

[0083] In step S701, the system determines the precise direction vector of the laser beam in the spatial coordinate system based on the real-time deflection angle of the laser galvanometer and the pre-established mapping relationship between the galvanometer deflection angle and the spatial direction of the laser beam through system calibration. This direction vector completely describes the propagation path of the laser beam in three-dimensional space.

[0084] Step S702 synchronously processes image information. Based on the pixel coordinates of the laser grating feature points extracted from a single frame image, and the intrinsic and extrinsic parameters obtained through camera calibration (including focal length, principal point coordinates, lens distortion coefficient, and camera pose), the direction vector of the corresponding camera imaging ray in the spatial coordinate system is determined through back projection calculation. This step eliminates the influence of lens distortion on measurement accuracy and ensures the accuracy of the imaging ray direction.

[0085] In step S703, the previously obtained laser beam direction vector and camera imaging ray vector are subjected to spatial intersection calculation. Specifically, the laser beam direction vector is represented as a spatial straight line equation, and the camera imaging ray vector is represented as another spatial straight line equation. By solving for the intersection of these two spatial lines, the three-dimensional coordinates of the laser grating feature point on the surface of the object being measured can be obtained. For example, when the laser galvanometer deflection angle is 0.15 radians, the corresponding laser beam direction vector is (0.988, 0.010, 0.155), while the camera imaging ray vector calculated based on the pixel coordinates (400.5, 300.2) is (0.120, -0.085, 0.989). By performing spatial straight line intersection calculation, the three-dimensional coordinates of the feature point (25.36, 15.82, 50.15) millimeters can be obtained.

[0086] This embodiment achieves accurate conversion from 2D images to 3D coordinates by organically combining 2D image information with the spatial direction of the laser beam through precise spatial geometric calculations. Since each 3D point is independently calculated based on precise geometric relationships, feature matching errors inherent in traditional multi-camera stereo vision are avoided, ensuring high accuracy of the measurement results. Simultaneously, directly utilizing the real-time deflection angle of the laser galvanometer as a spatial direction reference effectively reduces system calibration complexity and improves measurement stability and reliability.

[0087] To better implement the photogrammetric control method based on a laser galvanometer in the embodiments of the present invention, based on the photogrammetric control method based on a laser galvanometer, correspondingly, as follows: Figure 8 As shown, this embodiment of the invention also provides a photogrammetry control device based on a laser galvanometer. The photogrammetry control device 800 based on a laser galvanometer includes: The image acquisition module 801 is used to control the laser galvanometer to deflect along a preset scanning path within a single exposure cycle of the camera, so as to form an image in a single frame acquired by the camera. n A laser grating, n It is a positive integer greater than 1; The feature extraction module 802 is used to extract feature information of each laser grating from a single frame image. The feature information includes pixel coordinates in the spatial coordinate system. The 3D rendering module 803 is used to calculate the 3D coordinate points on the surface of the object under test based on the real-time deflection angle of the laser galvanometer when projecting each laser grating, the feature information of each laser grating, and the pre-calibrated camera parameters, and to generate 3D point cloud data.

[0088] The photogrammetric control device 800 based on a laser galvanometer provided in the above embodiments can realize the technical solutions described in the above embodiments of the photogrammetric control method based on a laser galvanometer. The specific implementation principles of each module or unit can be found in the corresponding content in the above embodiments of the photogrammetric control method based on a laser galvanometer, and will not be repeated here.

[0089] like Figure 9 As shown, the present invention also provides an electronic device 900. The electronic device 900 includes a processor 901, a memory 902, and a display 903. Figure 9 Only some components of the electronic device 900 are shown, but it should be understood that it is not required to implement all of the components shown, and more or fewer components may be implemented instead.

[0090] In some embodiments, processor 901 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 902 or process data, such as the photogrammetric control method based on laser galvanometer in this invention.

[0091] In some embodiments, processor 901 may be a single server or a group of servers. The server group may be centralized or distributed. In some embodiments, processor 901 may be local or remote. In some embodiments, processor 901 may be implemented on a cloud platform. In one embodiment, the cloud platform may include a private cloud, public cloud, hybrid cloud, community cloud, distributed cloud, intranet, multi-cloud, etc., or any combination thereof.

[0092] In some embodiments, memory 902 may be an internal storage unit of electronic device 900, such as a hard disk or memory of electronic device 900. In other embodiments, memory 902 may also be an external storage device of electronic device 900, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 900.

[0093] Furthermore, the memory 902 may include both internal storage units of the electronic device 900 and external storage devices. The memory 902 is used to store application software and various types of data installed on the electronic device 900.

[0094] In some embodiments, display 903 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 903 is used to display information from electronic device 900 and to display a visual user interface. Components 901-903 of electronic device 900 communicate with each other via a system bus.

[0095] In one embodiment, when the processor 901 executes the laser galvanometer-based photogrammetry control program in the memory 902, the following steps can be implemented: During a single exposure cycle of the camera, the laser galvanometer is controlled to deflect along a preset scanning path to form a single frame image captured by the camera. n A laser grating, n It is a positive integer greater than 1; The feature information of each laser grating is extracted from a single frame image. The feature information includes pixel coordinates in the spatial coordinate system. Based on the real-time deflection angle of the laser galvanometer when projecting each laser grating, the characteristic information of each laser grating, and the pre-calibrated camera parameters, the three-dimensional coordinate points on the surface of the object under test are calculated and the three-dimensional point cloud data is generated.

[0096] It should be understood that when the processor 901 executes the photogrammetry control program based on the laser galvanometer in the memory 902, in addition to the functions mentioned above, it can also perform other functions, as detailed in the description of the corresponding method embodiments above.

[0097] Furthermore, the embodiments of the present invention do not specifically limit the type of the electronic device 900 mentioned. The electronic device 900 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the present invention, the electronic device 900 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0098] Accordingly, embodiments of this application also provide a computer-readable storage medium for storing computer-readable programs or instructions. When the programs or instructions are executed by a processor, they can implement the steps or functions of the photogrammetric control method based on laser galvanometer provided in the above-described method embodiments.

[0099] Those skilled in the art will understand that all or part of the processes of the methods described in the above embodiments can be implemented by a computer program instructing related hardware (such as a processor, controller, etc.), and the computer program can be stored in a computer-readable storage medium. The computer-readable storage medium may be a disk, optical disk, read-only memory, or random access memory, etc.

[0100] The above provides a detailed description of the photogrammetric control method, device, electronic equipment, and storage medium based on laser galvanometer provided by the present invention. Specific examples have been used to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core ideas of the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation methods and application scope based on the ideas of the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. A laser galvanometer-based photogrammetric control method, characterized in that, The method comprises: During a single exposure period of a camera, a laser galvanometer is controlled to deflect according to a pre-set scan path to form a n a line laser raster, n is a positive integer greater than 1; extracting feature information of each laser raster from the single frame image, the feature information comprising pixel coordinates in a spatial coordinate system; calculating three-dimensional coordinate points on the surface of the object to be measured according to the real-time deflection angle of the laser galvanometer when projecting each laser raster, the feature information of each laser raster, and the pre-calibrated camera parameters, and generating three-dimensional point cloud data by summarizing.

2. Laser galvanometer-based photogrammetry control method according to claim 1, characterized in that, Before controlling the laser galvanometer to deflect according to the preset scanning path, the method further comprises: obtaining appearance feature information of the object to be measured, the appearance feature information comprising object size and surface shape; generating the preset scanning path adapted to the object to be measured by using a path planning algorithm according to the appearance feature information.

3. The laser galvanometer-based photogrammetry control method of claim 1, wherein, The step of controlling the laser galvanometer to deflect according to the preset scanning path comprises: controlling the laser galvanometer to deflect the laser beam to n a predetermined position one by one, and controlling the laser beam to keep stable stay at each of the predetermined positions; Exposure of a single frame image is accomplished by the camera during the stable dwelling of the laser beam at the predetermined positions to obtain a single frame image containing n a line laser raster.

4. The laser galvanometer-based photogrammetry control method of claim 1, wherein, The step of forming multiple laser rasters in a single exposure period of the camera comprises: According to the performance parameters of the camera and the laser galvanometer, the number of laser raster strips in a single frame image is determined by using the following formula n : T ≥ n ×( T 1+ T 2) wherein, T 1 is the dwell time of a single raster at a predetermined position, T 1 is the dwell time of a single raster at a predetermined position, T 2 is the rotation time of the laser galvanometer between adjacent rasters.

5. The laser galvanometer-based photogrammetry control method of claim 1, wherein, The step of controlling the laser galvanometer to deflect according to the preset scanning path comprises: controlling the laser galvanometer to deflect during the acquisition of continuous multiple frames of images; using a fixed step size to cause the laser rasters in adjacent frames of images to have an increment in position on the surface of the object to be measured.

6. The laser galvanometer-based photogrammetry control method of claim 1, wherein, When the object to be measured is in a state of motion, the method further comprises: obtaining motion trajectory information of the object to be measured in the state of motion; According to the motion trajectory information, the deflection angle of the laser galvanometer is adjusted in real time, so that the n A linear laser raster can be projected on the target area of the surface of the object to be measured to follow the motion of the object. extracting feature information from the single frame image containing the projected laser rasters and calculating three-dimensional coordinate points to generate three-dimensional point cloud data of the object in the state of motion.

7. Laser galvanometer-based photogrammetry control method according to any of claims 1 to 6, characterized in that, The step of calculating three-dimensional coordinate points on the surface of the object to be measured according to the real-time deflection angle of the laser galvanometer when projecting each laser raster, the feature information of each laser raster, and the pre-calibrated camera parameters comprises: determining a direction vector of the laser beam in the spatial coordinate system according to the real-time deflection angle; determining a direction vector of the corresponding camera imaging light ray in the spatial coordinate system according to the pixel coordinates and the camera parameters; calculating the three-dimensional coordinate point corresponding to the feature point of the laser raster by solving the intersection point of the two vectors in space based on the direction vector of the laser beam and the light ray vector of the camera imaging.

8. A laser galvanometer-based photogrammetric control device, characterized in that The method comprises: The image acquisition module is configured to control the laser galvanometer to deflect according to a preset scanning path in a single exposure period of the camera, so as to form a n a laser raster, n is a positive integer greater than 1. a feature extraction module configured to extract feature information of each laser raster from a single frame image, the feature information comprising pixel coordinates in a spatial coordinate system; a three-dimensional rendering module configured to calculate three-dimensional coordinate points on the surface of an object to be measured according to a real-time deflection angle of a laser galvanometer when projecting each laser raster, the feature information of each laser raster, and pre-calibrated camera parameters, and generate three-dimensional point cloud data by summarizing.

9. An electronic device, comprising: The method comprises a memory and a processor, wherein the memory is configured to store a program; the processor is coupled to the memory and is configured to execute the program stored in the memory to implement the steps of the photogrammetry control method based on a laser galvanometer according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer readable program or instructions for storing are provided, which can implement the steps of the photogrammetry control method based on a laser galvanometer according to any one of claims 1 to 7 when executed by a processor.

Images