Multi-point displacement measurement method, system and electronic device based on single optical source

Abstract

The application relates to the technical field of optical distance measuring, in particular to a multi-point displacement measuring method and system based on a single optical source and electronic equipment, the method comprising the following steps: acquiring an original image, wherein the original image comprises a measured object and a plurality of laser points; extracting the laser points in the original image to obtain the distortion 2D coordinates of each laser point; establishing a corresponding relationship between each laser point and a transmitted light beam; based on the pre-calibrated camera parameters and laser parameters, the distortion 2D coordinates of each laser point and the corresponding relationship between the laser points and the transmitted light beam, the 3D coordinates of each laser point are calculated; and the 3D coordinates of the plurality of laser points are outputted, the synchronous multi-point displacement measurement of the single optical source is adapted to the monitoring scene of discrete measuring points, multiple debugging on hardware is not needed, and the measuring efficiency and accuracy are improved.

Description

Technical Field

[0001] This application relates to the field of optical ranging technology, and in particular to a method, system and electronic device for multi-point displacement measurement based on a single optical source. Background Technology

[0002] In existing technologies, displacement measurement mainly relies on single-point triangulation laser sensors or high-resolution 3D scanners. Both types of devices are based on laser triangulation, a technology used for non-contact displacement or distance measurement. This technology projects a laser onto the target surface, captures the reflection point at a fixed angle using a camera or photodetector, and then derives the displacement through ensemble triangulation.

[0003] While high-resolution 3D scanners can provide three-dimensional inspection of the entire surface, they suffer from resource waste and inefficiency in displacement monitoring scenarios that require only a limited number of discrete measurement points. The scanning and data processing speeds are slow, and a large amount of unnecessary redundant data is generated, increasing the computational complexity of data processing.

[0004] At this point, a single-point triangulation laser sensor is usually selected. However, a single-point sensor can only detect the displacement of a single measurement point. If multi-point measurement is required, multiple sensor units must be deployed, which makes the overall equipment bulky. In addition, each sensor needs to be calibrated separately, which is time-consuming and prone to errors due to manual operation, thus reducing measurement efficiency and accuracy. Summary of the Invention

[0005] In displacement monitoring scenarios requiring only a limited number of discrete measurement points, this application provides a multi-point displacement measurement method, system, and electronic device based on a single optical source to improve measurement efficiency and accuracy.

[0006] Firstly, this application provides a multi-point displacement measurement method based on a single optical source, employing the following technical solution:

[0007] A method for multi-point displacement measurement based on a single optical source, the method comprising:

[0008] Acquire an original image, wherein the original image includes the object under test and multiple laser points;

[0009] Extract the laser points from the original image to obtain the distorted 2D coordinates of each laser point;

[0010] Establish the correspondence between each laser point and the emitted beam;

[0011] Based on pre-calibrated camera and laser parameters, the distorted 2D coordinates of each laser point, and the correspondence between laser points and emitted beams, the 3D coordinates of each laser point are calculated.

[0012] Output the 3D coordinates of multiple laser points.

[0013] In one possible implementation, laser points are extracted from the original image to obtain the distorted 2D coordinates of each laser point, including:

[0014] Distinguish between light spot regions and background regions in the original image based on brightness thresholds;

[0015] Mark multiple light spots in the light spot area;

[0016] Calculate the centroid of each light spot to obtain the laser point;

[0017] The distorted 2D coordinates of each laser point are obtained based on its position in the original image.

[0018] In one possible implementation, when the column of multiple laser points is perpendicular to the sensor plane formed by the camera optical axis and the laser axis of the emitted beam array, establishing the correspondence between each laser point and the emitted beam includes:

[0019] Determine the number of laser points in each row area based on the preset row area;

[0020] Determine if the number of laser points in each row region is the preset value.

[0021] If so, the column order of the laser points is determined according to the distorted 2D coordinates of multiple laser points in each row region;

[0022] The correspondence between laser points and emitted beams is determined based on the row regions and column order of the laser points.

[0023] In one possible implementation, when the column of multiple laser points is not perpendicular to the sensor plane formed by the camera optical axis and the laser axis of the emitted beam array, establishing the correspondence between each laser point and the emitted beam includes:

[0024] Directly match the laser point with the corresponding emitted beam, wherein the emitted beam is an independent line in the original image.

[0025] In one possible implementation, based on pre-calibrated camera and laser parameters, the distorted 2D coordinates of each laser point, and the correspondence between laser points and emitted beams, the 3D coordinates of each laser point are calculated, including:

[0026] The distorted 2D coordinates of each laser point are corrected according to the camera parameters to obtain the undistorted 2D coordinates;

[0027] Based on the camera parameters, the distortion-free 2D coordinates are converted into a normalized direction vector of the camera's line of sight;

[0028] The intersection point is determined based on the propagation trajectory of the emitted beam corresponding to the laser point and the normalized direction vector to obtain the 3D coordinates of the laser point.

[0029] In one possible implementation, the 3D coordinates of multiple laser points are output, including:

[0030] The 3D coordinates of multiple laser points are arranged according to the corresponding distorted 2D coordinates, and when the number of laser points in each row region is not a preset value, the output of the 3D coordinates of the laser points in the corresponding row region is invalid.

[0031] Secondly, this application provides a multi-point displacement measurement system based on a single optical source, employing the following technical solution:

[0032] A multi-point displacement measurement system based on a single optical source, the system comprising:

[0033] A laser source is a source beam used to emit a single beam of light.

[0034] A diffraction beam splitter is used to split a single source beam into an array of emitted beams.

[0035] A camera is used to acquire the original image of the laser point on the object being measured, wherein the optical axis of the camera is set at a preset angle with the laser axis of the emitted beam array;

[0036] The controller, coupled to the laser source and the camera respectively, is used to control the laser source to emit a source beam, receive the raw image output by the camera, and calculate and output the 3D coordinates of multiple laser points.

[0037] In one possible implementation, the system further includes:

[0038] A narrowband filter is placed in the imaging optical path between the camera and the object being measured, and its center wavelength is the same as the laser beam wavelength. It is used to filter out ambient stray light.

[0039] Thirdly, this application provides an electronic device that adopts the following technical solution:

[0040] An electronic device includes: a memory and a processor;

[0041] The memory is used to store computer programs / instructions;

[0042] The processor is configured to execute the computer program / instructions stored in the memory, causing the processor to perform the first aspect and / or various possible implementations of the first aspect as described above.

[0043] Fourthly, this application provides a computer-readable storage medium, which adopts the following technical solution:

[0044] A computer-readable storage medium storing a computer program / instructions, which, when executed by a processor, are used to implement the first aspect and / or various possible embodiments of the first aspect as described above.

[0045] This application provides a method, system, and electronic device for multi-point displacement measurement based on a single optical source. The method acquires an original image containing laser points using a camera, calculates and extracts the distorted 2D coordinates of the laser points, establishes a correspondence between the laser points and the emitted beam based on the perpendicular relationship between the column of laser points and the sensor plane, and then corrects the distorted coordinates and converts them into normalized direction vectors by combining calibrated camera parameters and laser parameters. Finally, it obtains 3D coordinates by combining the propagation trajectory of the emitted beam, realizing synchronous multi-point displacement measurement of a single optical source. This method is suitable for monitoring scenarios with discrete measurement points, eliminates the need for multiple hardware adjustments, and improves measurement efficiency and accuracy.

[0046] The system includes a laser source, a diffraction beam splitter, a camera, and a controller. A narrowband filter of the same wavelength can be added to filter out stray light. The camera optical axis is set at an angle to the laser axis. The system is controlled and calculated by the controller. The structure is simple and easy to deploy. Attached Figure Description

[0047] Figure 1a A schematic diagram of the structure of a multi-point displacement measurement system based on a single optical source is provided in one embodiment of this application.

[0048] Figure 1b A schematic diagram of the structure of a multi-point displacement measurement system based on a single optical source is provided in one embodiment of this application.

[0049] Figure 2 This is a flowchart illustrating a multi-point displacement measurement method based on a single optical source, provided as an embodiment of this application.

[0050] Figure 3 This is a flowchart illustrating a multi-point displacement measurement method based on a single optical source, provided as an embodiment of this application.

[0051] Figure 4 This is a schematic diagram of the structure of an electronic device provided in an embodiment of this application. Detailed Implementation

[0052] To better understand the purpose, technical solutions, and advantages of this application, it has been described and illustrated below with reference to the accompanying drawings and embodiments. However, those skilled in the art should understand that this application can be implemented without these details. In some cases, to avoid obscuring various aspects of this application due to unnecessary description, well-known methods, processes, systems, components, and / or circuits already described at a higher level will not be elaborated upon. It will be apparent to those skilled in the art that various modifications can be made to the embodiments disclosed in this application, and the general principles defined in this application can be applied to other embodiments and application scenarios without departing from the principles and scope of this application. Therefore, this application is not limited to the illustrated embodiments, but conforms to the broadest scope consistent with the scope of protection claimed in this application.

[0053] It should be noted that the descriptions of these embodiments are for the purpose of aiding understanding the present invention, but do not constitute a limitation thereof. Furthermore, the technical features involved in the various embodiments of the present invention described below can be combined with each other as long as they do not conflict with each other.

[0054] It should be understood that the embodiments described herein may be implemented in hardware, software, firmware, middleware, microcode, or any combination thereof. For hardware implementations, the processor may be implemented as one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field-programmable gate arrays (FPGAs), processors, controllers, microcontrollers, microprocessors, other electronic units designed to perform the functions described herein, or combinations thereof.

[0055] When an embodiment is implemented as software, firmware, middleware, or microcode, program code, or code segments, it may be stored in a machine-readable medium, such as a storage component. A code segment may represent a procedure, function, subroutine, program, routine, subroutine, module, software package, class, or any combination of instructions, data structures, or program statements. One code segment can be coupled to another code segment or hardware circuitry by passing and / or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., can be passed, forwarded, or transmitted using any suitable means, including memory sharing, messaging, token passing, network transmission, etc.

[0056] For software implementations, the techniques described herein can be implemented using modules (e.g., programs, functions, etc.) that perform the functions described herein. The software code can be stored in memory units and executed by a processor. The memory units can be implemented within or outside the processor; in the latter case, the memory units can be communicatively coupled to the processor via various methods known in this art.

[0057] In the description of this application, "several" means one or more, "more than" means two or more, "greater than," "less than," and "exceeding" are understood to exclude the stated number, while "above," "below," and "within" are understood to include the stated number. The use of "first" and "second" in the description is merely for distinguishing technical features and should not be construed as indicating or implying relative importance, or implicitly indicating the number of indicated technical features, or implicitly indicating the order of the indicated technical features.

[0058] In the description of this application, the terms "one embodiment," "some embodiments," "illustrative embodiment," "example," "specific example," or "some examples," etc., refer to specific features, structures, materials, or characteristics described in connection with that embodiment or example, which are included in at least one embodiment or example of this application. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any one or more embodiments or examples.

[0059] In existing technologies, when conducting displacement monitoring for a limited number of discrete measurement points, although high-resolution 3D scanners can achieve full-surface three-dimensional detection, they have not been adapted and optimized for discrete multi-point displacement monitoring scenarios. During the scanning process, a large amount of redundant data is generated, which not only results in high hardware costs but also increases the computational load of data processing, leading to low detection and processing efficiency.

[0060] When using multiple single-point triangulation laser sensors to complete multi-point measurements, multiple sets of sensor hardware need to be deployed, making the overall equipment bulky, and each sensor needs to be calibrated individually. Especially when measuring 20-30 or more points, calibrating each sensor one by one is not only time-consuming and labor-intensive, but also prone to errors due to manual operation. At the same time, the complexity of wiring and the difficulty of data synchronization increase significantly with the number of sensors, seriously affecting measurement efficiency and accuracy.

[0061] To address the aforementioned technical problems, this application provides a method, system, and electronic device for multi-point displacement measurement based on a single optical source. This method can simultaneously measure multiple three-dimensional displacement points using a single optical source, effectively improving the efficiency and accuracy of discrete displacement monitoring.

[0062] The technical solution of this application and how the technical solution of this application solves the above-mentioned technical problems are described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments. The embodiments of this application will be described below with reference to the accompanying drawings.

[0063] Figure 1a and Figure 1b A schematic diagram of a multi-point displacement measurement system based on a single optical source provided in an embodiment of this application; as shown below. Figure 1a and Figure 1b As shown in the figure, this application discloses a multi-point displacement measurement system based on a single optical source, the system comprising:

[0064] The outer casing 101 can be made of different materials depending on the working conditions and requirements.

[0065] The laser source 102 is installed inside the housing 101 and is used to emit a single source beam.

[0066] The diffraction beam splitter 103, mounted on the housing 101, is used to split a single source beam into an array of emitted beams.

[0067] Camera 104, installed inside housing 101, is used to acquire the original image of the laser point on the object being measured, wherein the optical axis of the camera is set at a preset angle α with respect to the laser axis of the emitted beam array.

[0068] The controller 105 is installed inside the housing 101 and is coupled to the laser source 102 and the camera 104 respectively. It is used to control the laser source 102 to emit the source beam, receive the original image output by the camera 104, and calculate and output the 3D coordinates of multiple laser points.

[0069] A narrowband filter 106 is mounted on the housing 101 and placed in the imaging optical path between the camera 104 and the object being measured. Its center wavelength is the same as the laser beam wavelength, and it is used to filter ambient stray light.

[0070] After the laser source 102 is turned on, it emits a single source beam. This source beam is diffracted and split by the diffractor beam splitter 103 into a regularly arranged array of emitted beams. For example, the emitted beam array is a 5×5 array. A single diffractor beam splitter 103 can achieve multi-point beam projection from a single optical source. The source beam passes through the diffractor beam splitter 103 and is split into 25 emitted beams diverging at an angle θ, where θ is approximately 11°. After the emitted beam array is projected onto the surface of the object being measured, it forms a laser point array corresponding to the emitted beam array. The camera 104 captures an image of the object being measured with the projected laser point array, acquiring an original image containing the object and all laser points on its surface. The controller 105 receives this original image, analyzes and processes it, and ultimately calculates the 3D coordinates of all laser points, completing the synchronous measurement of multi-point displacement and outputting the results.

[0071] Narrowband filter 106 is arranged in the imaging optical path of camera 104. It can significantly attenuate and shield stray light in the environment that has a wavelength different from that of the laser beam, such as natural light, industrial lighting light, and various interference light such as reflections from equipment surfaces. This prevents stray light from forming false light spots when the camera 104 is imaging, effectively preventing stray light from being confused with the laser point. This provides a precise image basis for the subsequent laser point recognition, coordinate extraction and 3D coordinate calculation of controller 105, and improves the accuracy of measurement results.

[0072] The housing 101 serves as the hardware integration carrier of the system and can be made of suitable materials according to the actual application requirements of the industrial site. The laser source 102, diffraction beam splitter 103, camera 104, controller 105 and narrowband filter 106 are all integrated and installed in the preset position of the housing 101. The relative positions of each hardware component are fixed after precise calibration, realizing the integrated design of the entire system. This ensures the installation accuracy of the optical components, reduces the impact of component offset on the measurement results, simplifies the on-site deployment process of the system, eliminates the need for multiple on-site debugging, effectively improves the hardware deployment efficiency of the system in industrial scenarios, and enhances the overall structural stability and environmental adaptability of the system.

[0073] The preset angle α between the camera's optical axis and the laser axis of the emitted beam array is determined comprehensively based on factors such as the optical geometry of laser triangulation, system measurement accuracy requirements, working range requirements, and hardware installation requirements.

[0074] 1. The preset angle is a basic condition for the camera to acquire the original image. The laser axis and the camera optical axis must form an intersecting angle so that the 3D displacement of the measured surface can be deduced by combining the position change of the laser point in the original image and the geometric relationship. If there is no angle or the angle is 0°, the camera will not be able to capture the position shift of the light point, and the core logic of triangulation will fail. Therefore, the angle must be an effective angle greater than 0°.

[0075] 2. The size of the preset angle determines the effective measurement distance of the system. If the angle is too large, the measurement range will be reduced, and the offset of the laser point in the original image will easily exceed the camera's field of view. If the angle is too small, the imaging sensitivity of the laser point position offset will be reduced, making it difficult to achieve accurate measurement. The optimal angle needs to be matched according to the preset working range to ensure that the laser point is always clearly imaged within the working range and the offset can be accurately captured.

[0076] 3. The setting of the preset angle must ensure that the minute displacement of the measured surface can be converted into a recognizable laser point pixel offset in the original image, i.e., the imaging magnification ratio. If the angle is too small, the pixel offset corresponding to the minute displacement is insufficient and is easily masked by noise in the original image. If the angle is too large, although the pixel offset is large, the effective coverage of the camera's field of view will be limited.

[0077] 4. Since the laser source 102, diffraction beam splitter 103, camera 104, controller 105 and narrowband filter 106 are all integrated and installed in the housing 101, the preset included angle must meet the mechanical size limit of the housing 101 to ensure that the emitted beam array formed after the source beam passes through the diffraction beam splitter 103 can completely fall into the field of view of the camera.

[0078] In this preferred embodiment, a 532nm, ~10mW laser source 102 can be selected, paired with a diffraction beam splitter 103 (DOE) that can generate 25 laser points in a 5×5 grid. The camera uses a 1440×1080 monochromatic sensor, installed at a preset angle of about 20°, with a measurement range (nominal working range) of 35-45cm, and a narrowband filter 106 with a center wavelength of 532nm is added, which not only meets the geometric requirements of triangulation, but also adapts to the compact hardware installation design of the system.

[0079] After the system hardware is built, a single calibration scheme using laser projection and camera imaging coupling is adopted to replace the separate calibration of multiple sensors in the existing technology. This allows for the simultaneous acquisition of camera and laser parameters, which can then be directly used for subsequent calculations.

[0080] A calibration board with a known geometry is selected, such as a checkerboard calibration board with a known grid spacing. Then, a darkroom environment is set up to capture pure laser point images and eliminate interference from the calibration board image. The specific calibration steps are as follows:

[0081] (1) Place the calibration plate in N different positions / spatial directions (N≥10) within the measurement space to cover the nominal working range of the system;

[0082] (2) Capture two images for each position j (j=1,2,...,N) on the calibration plate:

[0083] Image a: Original image of the calibration plate under normal lighting, showing the clearly visible geometry of the calibration plate;

[0084] Image b: A laser spot image taken in a dark room environment. Ambient light was turned off and only the laser source was turned on. Only the laser spot projected by the diffraction beam splitter onto the calibration plate is visible in the image. There is no calibration plate pattern.

[0085] (3) Import the original images of the calibration board at all locations (i.e., image a) into the OpenCV standard camera calibration routine. Input the world 3D coordinates of the grid points on the calibration board and the pixel 2D coordinates of the grid points in the image. Solve to obtain the camera parameters, which include:

[0086] (1) Camera matrix (Internal Reference):

[0087]

[0088] and This represents the camera's focal length in pixels, both horizontally and vertically.

[0089] These are the camera principal point coordinates, the pixel coordinates of the center of the image plane, in pixels;

[0090] (2) Camera distortion coefficient Including radial distortion and tangential distortion , in the form of This is used for subsequent image coordinate distortion correction;

[0091] (3) Rotation matrix of the calibration plate at each position Translation vector This describes the spatial position and orientation of the calibration plate in the world coordinate system;

[0092] (4) According to the rotation matrix of the calibration plate at each position Translation vector The 3D plane equation corresponding to the calibration plate at position j is derived as follows:

[0093]

[0094] in, ; ; ; ; For the plane normal vector, by Derivation, perpendicular to the plane of the calibration plate, Let be the plane intercept, and all coefficients satisfy . ;

[0095] (5) For each location of the laser point image (i.e., image b), identify the distorted 2D coordinates of all laser points in the image. (i is the laser point index, i=1,2,...), the superscript d indicates "distortion";

[0096] (6) Using OpenCV distortion removal routines, combined with camera matrix and camera distortion coefficient Convert distorted 2D coordinates to dedistorted 2D coordinates The formula is:

[0097]

[0098] The distortion-free coordinates eliminate the optical distortion of the camera lens and are the true pixel coordinates;

[0099] (7) Combining camera parameters and the 3D plane equation of the calibration board, the distortion-free 2D coordinates are... Converted to 3D world coordinates of laser points on the calibration plate Specifically:

[0100]

[0101]

[0102] After deformation, we get:

[0103]

[0104]

[0105] That is, to , Substitute Solving for the given information yields the following results. Then substitute back to calculate , Finally, the 3D position of each laser point on the calibration plate is obtained;

[0106] Example, known =720, =1200, =540, =1195, the 2D coordinates of a certain laser point after distortion correction are: =840, =659.5, Calibration plate 3D plane equation =40cm, then:

[0107] = =4cm, =4cm, which is the 3D coordinate of the laser point. cm.

[0108] (8) All laser points i at position j are projected by the same laser line. For the same laser line i, summarize its 3D coordinates at N positions. The least squares method was used to fit the spatial linear equation of the laser point, and the origin of the ray was obtained by solving the equation. and unit direction vector (satisfy The parametric equation of a straight line in space is: (t≥0), where, The length parameter of the light ray. For any 3D point on the ray; minimize all points by matrix inversion or SVD decomposition. Find the sum of squared distances to the line that minimizes the error. and .

[0109] (9) Summarize all camera parameters and laser line parameters, complete a single calibration, and store all camera parameters and laser line parameters in the controller for later use.

[0110] The multi-point displacement measurement method based on a single optical source provided in this application embodiment can be executed by the multi-point displacement measurement system based on a single optical source provided in this application embodiment.

[0111] The technical solutions of this application will be described in detail below with specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0112] Figure 2 This is a flowchart illustrating a multi-point displacement measurement method based on a single optical source, provided in an embodiment of this application. Figure 1a and Figure 1b The technical implementation of the illustrated embodiment is applied to a controller, such as... Figure 2 As shown, the method in this embodiment may include the following steps:

[0113] S201. Obtain the original image.

[0114] The controller sends a coordinated working command to the laser source and the camera. After receiving the command, the laser source emits a single source beam. After being diffracted and split by the diffractor, the source beam forms a regularly arranged array of emitted beams and is projected onto the surface of the object being measured, forming a laser dot array on the surface of the object being measured that corresponds one-to-one with the array of emitted beams.

[0115] The camera captures images of the object under test containing the laser dot array at the same time or after the beam array is projected, using preset shooting parameters. It acquires an original image containing the entire object under test and all the laser dots, and transmits the original image to the controller in real time.

[0116] S202. Extract the laser points from the original image to obtain the distorted 2D coordinates of each laser point.

[0117] After receiving the original image, the controller preprocesses the image to determine the laser points. This process involves removing image noise using Gaussian filtering, separating the laser points from the background using binarization thresholding, and finally identifying the light point regions through contour detection. For the identified light point regions, sub-pixel-level positioning is performed using the centroid method or Gaussian fitting algorithm to improve the coordinate accuracy from the pixel level to the sub-pixel level, such as 0.1 pixels, thus filtering out the laser points.

[0118] After extracting the laser points from the original image, the horizontal and vertical coordinates of each laser point in the original image are recorded according to the camera's pixel coordinate system to obtain the distorted 2D coordinates of each laser point. Due to the optical distortion of the camera lens, the distorted 2D coordinates have a slight positional deviation, which can achieve coarse positioning of the laser points.

[0119] S203. Establish the correspondence between each laser point and the emitted beam.

[0120] Since the original image captured by the camera is a planar 2D image, the intelligent system records the pixel coordinates of the laser points in the original image. By establishing the correspondence between each laser point and the emitted beam, the system realizes the correspondence between the 2D pixel coordinates of each laser point in the original image and the 3D spatial coordinates of the surface of the object being measured. Furthermore, since different emitted beams correspond to different spatial propagation trajectories, when calculating the 3D coordinates of a certain laser point, it is necessary to know which emitted beam the laser point was emitted from. That is, it is necessary to call the direction vector of the emitted beam corresponding to the laser point. Only by combining the propagation trajectory of the emitted beam and the camera's line of sight trajectory to find the intersection point through triangulation can the 3D coordinates of the laser point be obtained.

[0121] S204. Based on the pre-calibrated camera and laser parameters, the distorted 2D coordinates of each laser point, and the correspondence between the laser point and the emitted beam, calculate the 3D coordinates of each laser point.

[0122] Since the controller stores calibrated camera and laser parameters, and combines this with the established correspondence between laser points and emitted beams, the 3D coordinates of the laser points can be calculated.

[0123] First, the distorted 2D coordinates of the laser point are corrected using the camera distortion coefficient to eliminate the optical distortion of the camera lens and obtain the distorted 2D coordinates.

[0124] Subsequently, based on the pixel focal length and principal point coordinates in the camera matrix, the distortion-free 2D coordinates are converted into a normalized direction vector of the camera's line of sight, which reflects the spatial propagation direction of the camera's optical center pointing to the laser point.

[0125] Finally, for each laser point, the spatial propagation trajectory parameters of its corresponding emitted beam are retrieved. By combining the propagation trajectory of the emitted beam with the normalized direction vector of the camera's line of sight, the optimal spatial intersection point of the two is solved using the least squares method. The spatial coordinates of this intersection point are the 3D coordinates of the laser point in the real world.

[0126] S205 outputs the 3D coordinates of multiple laser points.

[0127] After calculating the 3D coordinates of multiple laser points, all 3D coordinates are output. Furthermore, when outputting the 3D coordinates, the 3D coordinates can be sorted according to the position of the laser points to achieve a one-to-one correspondence between the laser points and their corresponding 3D coordinates, maintaining consistency with the laser point positions in the original image.

[0128] The output of multiple 3D coordinates can be encapsulated in industrially common data formats (such as CSV, JSON, etc.) and the dataset containing 3D coordinates can be synchronously output to external devices such as host computers and industrial control systems through preset communication interfaces (such as RS485, Ethernet, USB, etc.) to complete the output of the results of this multi-point displacement measurement. If continuous monitoring is required, steps S201-S205 can be executed repeatedly.

[0129] The multi-point displacement measurement method based on a single optical source provided in this application uses a controller to coordinate the synchronous operation of the laser source and the camera. The single source beam is divided into a regular emission beam array by a diffractor beam splitter. A single projection can form a multi-laser point array on the surface of the object being measured. The camera synchronously acquires the original image containing the complete laser point array. This eliminates the need for multiple projections and shooting, enabling the one-time acquisition of multi-point data and significantly shortening the measurement cycle.

[0130] Then, by accurately extracting the distorted 2D coordinates of the laser point, and combining them with preset rules to establish a unique correspondence between the laser point and the emitted beam, and relying on pre-calibrated camera parameters and laser parameters, coordinate distortion correction, line-of-sight vector transformation and spatial trajectory solving are completed. The 3D coordinates of the laser point are obtained through the least squares method, which effectively eliminates errors such as lens distortion and beam offset, improves the accuracy of point measurement, and ensures the synchronization and accuracy of multi-point displacement measurement.

[0131] Figure 3 A flowchart illustrating a multi-point displacement measurement method based on a single optical source provided in an embodiment of this application is shown below. Figure 3 As shown, based on the above embodiments, this embodiment includes the following method:

[0132] S301. Obtain the original image.

[0133] For a detailed description of step S301, please refer to step S201.

[0134] S302. Distinguish between light spot areas and background areas in the original image based on the brightness threshold.

[0135] The brightness threshold is usually preset, and it can be adaptive or fixed.

[0136] If an adaptive brightness threshold is used, the grayscale segmentation thresholds for the light spot region and background region containing the laser point are calculated by statistically analyzing the grayscale histogram of the original image. For example, the threshold range is usually 150-220 for an 8-bit grayscale image. Then, the original image is divided into local regions of a specific pixel grid, such as 10*10 pixels, and the brightness threshold is calculated independently for each region. Highlighted regions with an area < 3 pixels (set by the user) can be identified as noise and non-light spot regions.

[0137] By adopting an adaptive threshold, different lighting environments can be adapted, such as workshop lighting and indoor natural light, reducing the missed and false detections of laser points caused by fixed thresholds, and further improving the recognition purity of laser points.

[0138] S303, Mark multiple light spots in the light spot area.

[0139] After thresholding, the binary image can be used to extract all closed highlight contours using contour detection algorithms (such as OpenCV's findContours). By using preset contour circularity conditions, the contours of light spots that meet the conditions can be filtered out, and the boundaries of the light spots can be marked with rectangles to record the pixel coordinate range of the light spot contours.

[0140] For example, the spot selection criteria can be a circularity of the outline ≥ 0.7 and an outline area of ​​5-50 pixels. The irregular stray light outline can be filtered out by the circularity of the outline exceeding 0.7, while the laser point dispersion range can be limited by the outline area.

[0141] S304. Calculate the centroid of each light spot to obtain the laser point.

[0142] For the marked light spot, the precise position can be calculated using the centroid method. Specifically, the centroid calculation formula is as follows:

[0143]

[0144] in, , The coordinates of the pixels within the light spot. This represents the pixel grayscale value.

[0145] If the surface of the object being measured is relatively rough, the light spot may be slightly diffused. In this case, a Gaussian fitting algorithm can be selected to fit a two-dimensional Gaussian function to the gray-scale distribution of the light spot, and the extreme points of the function can be taken as the laser points.

[0146] The coordinates of the laser point can be obtained with sub-pixel accuracy using the centroid method or Gaussian fitting algorithm.

[0147] S305. Obtain the distorted 2D coordinates of each laser point based on its position in the original image.

[0148] With the camera pixel coordinate system as the reference, the origin is the top left corner of the original image, the u-axis is horizontal to the right, and the v-axis is vertical to the bottom.

[0149] Record the centroid coordinates of each laser point The superscript d indicates "distortion", and the coordinate values ​​can be retained as 1 decimal.

[0150] Additionally, the distorted 2D coordinates of all laser points can be arranged in the order they appear in the original image, for example, numbered from left to right and from top to bottom.

[0151] S306. Determine whether the column of multiple laser points is perpendicular to the sensor plane formed by the camera optical axis and the laser axis of the emitted beam array.

[0152] For the distorted 2D coordinates of each laser point, extract the v-axis coordinates of the laser points in the same column and calculate the column direction vector. , The difference in v-axis between adjacent laser points within a column.

[0153] If the included angle is approximately 90° (error ≤ 3°), then it is determined that "the column of laser points is perpendicular to the sensor plane"; otherwise, it is determined that "it is not perpendicular".

[0154] When the diffraction beam splitter is not rotated, the v-axis coordinates of the first column of the 5×5 laser dot array are 420.3, 480.5, 540.2, 599.8, and 659.7, and the column direction vector is (0, 60.2). The angle between the vector and the sensor plane normal vector is 88.5°, which is determined to be "perpendicular".

[0155] S307. If vertical, determine the number of laser points in each row area according to the preset row area.

[0156] Based on the number of rows in the emitted beam array, for example, if the diffraction beam splitter is split into a 5*5 emitted beam array, then the image along the v-axis will be divided into 5 non-overlapping row regions G1-G5. The boundaries of the regions are defined during calibration, such as G1: v∈[400,460], G2: v∈[461,520], and so on.

[0157] Iterate through the distorted 2D coordinates of all laser points, and count the number N of laser points in each row region based on their v-axis coordinates. k Referring to the example above, k=1-5, corresponding to G1-G5; therefore, the statistical results of the number of laser points in each row area can be output, for example, G1: 5; G2: 4; G3: 5; G4: 5; G5: 5.

[0158] In a certain measurement scenario, due to the obstruction of the measured object in the G2 row area, only 4 laser points will be detected in that row area.

[0159] S308. Determine whether the number of laser points in each row area is the preset value.

[0160] The preset value is determined based on the number of columns in the emitted beam array. When the diffraction beam splitter is split into a 5*5 emitted beam array, the preset value is 5.

[0161] The number of laser points N in each row region k Compare with the preset value to determine N. k Is it equal to the preset value?

[0162] N k If the value is equal to the preset value, then the row area is marked as a "valid row". k If the value is not equal to the default value, it is marked as "invalid row".

[0163] S309. If so, the column order of the laser points is determined according to the distorted 2D coordinates of multiple laser points in each row region.

[0164] For each "valid row", extract the u-axis coordinates of the distorted 2D coordinates of all laser points in that row, and sort them in ascending order of u-axis coordinates (left to right). The sorted laser points correspond to column order C1-C5 in sequence, such that the smallest u corresponds to C1 and the largest u corresponds to C5.

[0165] Record the "row region + column order" identifier for each laser point, such as G1C1, G1C2.

[0166] By sorting the columns using the horizontal axis, precise column matching between the laser point and the emitted beam can be achieved.

[0167] For example, the u-axis coordinates of the effective laser points in row G1 are 800.2, 840.5, 880.1, 920.3, and 960.6. After being sorted from smallest to largest, they correspond to C1-C5 respectively, where the laser point with u=840.5 is labeled as G1C2.

[0168] S310. Determine the correspondence between the laser points and the emitted beam based on the row region and column order of the laser points.

[0169] The emitted beam array is pre-indexed with a unique "row region-column" index, such as R1C1 corresponding to beam 1, R1C2 corresponding to beam 2, ..., R5C5 corresponding to beam 25; the "row region Gk" of the laser point is mapped to the "row Rk" of the emitted beam (G1→R1, G2→R2, and so on).

[0170] By combining the column order C1-C5, establish a unique correspondence between "laser point identifier → emitted beam index", such as G1C2 → beam 2.

[0171] The laser point identified as G3C4 is mapped to the emitted beam R3C4, index 14. When calculating the 3D coordinates of this laser point, the direction vector d of beam 14 can be directly retrieved. 14。

[0172] A special case is the use of a 1D diffraction beam splitter, such as a 5x1 (5 rows, 1 column). In this case, the columns are arranged in order, and since each group contains only one ray, a complete correspondence can be established after the row regions are determined.

[0173] S311. If not, mark the laser points in this row area as invalid laser points.

[0174] All laser points in an "invalid row" are marked as "invalid". Invalid laser points do not participate in column order matching and 3D coordinate calculation, and can be marked only in the final output.

[0175] In addition, the distorted 2D coordinates of invalid laser points can be retained to facilitate subsequent troubleshooting, such as to confirm the location of obstruction.

[0176] Avoid incorrect matching and calculation due to missing light spots, and reduce unnecessary computing power consumption.

[0177] S312. If not perpendicular, directly match the laser point with the corresponding emitted beam.

[0178] When not perpendicular, the diffraction beam splitter is usually rotated during installation, resulting in the emitted beam array appearing as independent lines in the camera's field of view and the original image. The projections of all emitted beams onto the original image plane are independent straight lines with no overlap or intersection.

[0179] For example, the diffraction beam splitter is rotated to a specific angle, such as 15°, so that the 25 emitted beams of the 5x5 emitted beam array appear as independent straight lines in the original image.

[0180] During calibration, the projection line equations for each laser beam are recorded. : , , , To calibrate the parameters, calculate the distorted 2D coordinates of each laser point. The distance to the projection line is the distance to the laser beam that is closest to the projection line. This is the laser beam that is matched to the laser point.

[0181] In addition, if the emitted beam is not arranged in a grid, this method can also be applied as long as the projection lines of the laser lines on the original image plane are independent and do not overlap.

[0182] S313. Correct the distorted 2D coordinates of each laser point according to the camera parameters to obtain the undistorted 2D coordinates.

[0183] Call the OpenCV distortion removal routine and input... Camera matrix Camera distortion coefficient The distortion-free 2D coordinates were calculated. .

[0184] Example: Distorted 2D coordinates of a laser point =(840.5,659.8), after substituting the camera parameters, the distortion-free 2D coordinates are: =(840.0,659.5), eliminating a distortion error of 0.5 pixels, corresponding to a spatial error of approximately 0.1mm.

[0185] S314. Convert the distortion-free 2D coordinates into a normalized direction vector of the camera's line of sight based on the camera parameters.

[0186] Based on the camera pinhole imaging model, through the formula Calculate, Divide by its magnitude to ensure the vector length is 1.

[0187] Convert 2D pixel coordinates into 3D spatial direction vectors to establish the line-of-sight trajectory from the camera's optical center to the laser point.

[0188] Example, known =840.0, =659.5, =720, =540, =1200, =1195, calculated as follows = = The normalized vector is .

[0189] S315. Determine the intersection point based on the propagation trajectory of the emitted beam corresponding to the laser point and the normalized direction vector to obtain the 3D coordinates of the laser point.

[0190] By adjusting the origin of the matched emitted beam Unit direction vector Establish the equation for the propagation trajectory of the emitted beam. and camera line-of-sight trajectory equation Where s > 0, s is the line-of-sight length parameter, which is a scalar parameter representing the distance from the camera's optical center to the spatial point. distance, It is any 3D point in the camera's line of sight, and the units of s and the laser parameter t are the same;

[0191] Due to optical errors, the equation of the propagation trajectory of the emitted beam... and camera line-of-sight trajectory equation Since they are not strictly intersecting, the least squares method can be used to minimize the sum of squared errors and find the optimal solution. and ;

[0192] Finally, the optimal and Substituting the emitted beam equation yields the 3D coordinates. .

[0193] S316. The 3D coordinates of multiple laser points are arranged according to the corresponding distorted 2D coordinates. When the number of laser points in each row region is not a preset value, the output of the 3D coordinates of the laser points in the corresponding row region is invalid.

[0194] The 3D coordinates of multiple laser points are sorted according to the "row region + column order" of the distorted 2D coordinates of the laser points, such as G1C1→G1C5→G2C1→…→G5C5, which corresponds one-to-one with the emitted beam array; for the laser points in the invalid rows marked in step S311, their 3D coordinates are uniformly marked as "invalid" or NaN.

[0195] The output of multiple 3D coordinates can be encapsulated in industrially common data formats (such as CSV, JSON, etc.) and the dataset containing 3D coordinates can be synchronously output to external devices such as host computers and industrial control systems through preset communication interfaces (such as RS485, Ethernet, USB, etc.) to complete the output of the results of this multi-point displacement measurement. If continuous monitoring is required, steps S301-316 can be executed repeatedly.

[0196] The multi-point displacement measurement method based on a single optical source provided in this application firstly effectively filters out ambient stray light and false spot interference by employing adaptive brightness threshold segmentation and contour filtering technology. Combined with the centroid method or Gaussian fitting algorithm, sub-pixel-level positioning is achieved, significantly improving the extraction accuracy of laser point distortion 2D coordinates. Then, by determining the perpendicular relationship between the laser point array and the sensor plane, a correspondence between the laser points and the emitted beam is established. Subsequently, the distortion coordinates are corrected using camera parameters, the normalized direction vector is transformed, and the propagation trajectory of the emitted beam and the camera line of sight are combined. The optimal intersection point is solved using the least squares method, effectively eliminating errors such as lens distortion and installation deviation, and realizing the calculation of 3D coordinates. By synchronously projecting multiple beams from a single optical source, replacing the deployment of multiple sensors, multiple calibrations and images are eliminated. This improves measurement efficiency and accuracy while simplifying hardware deployment and data processing, adapting to the industrial application needs of discrete measurement point monitoring scenarios.

[0197] Figure 4 This is a schematic diagram of the structure of an electronic device provided in one embodiment of this application, such as... Figure 4 As shown, the electronic device 40 provided in this embodiment includes:

[0198] The device 40 includes at least one processor 401 and a memory 402. Optionally, the device 40 also includes a communication component 403. The processor 401, memory 402, and communication component 403 are connected via a bus 404.

[0199] In a specific implementation, at least one processor 401 executes computer execution instructions stored in memory 402, causing at least one processor 401 to perform the above-described method.

[0200] The specific implementation process of processor 401 can be found in the above method embodiments, and its implementation principle and technical effect are similar. It will not be repeated here.

[0201] In the above embodiments, it should be understood that the processor can be a Central Processing Unit (CPU), or other general-purpose processors, digital signal processors (DSPs), application-specific integrated circuits (ASICs), etc. The general-purpose processor can be a microprocessor or any conventional processor. The steps of the method disclosed in this invention can be directly implemented by a hardware processor, or implemented by a combination of hardware and software modules within the processor.

[0202] The memory may include random access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device.

[0203] The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. Buses can be categorized as address buses, data buses, control buses, etc. For ease of illustration, the buses shown in the accompanying drawings are not limited to a single bus or a single type of bus.

[0204] This application also provides a computer program product, including a computer program / instructions that, when executed by a processor, implement the above-described method.

[0205] This application also provides a computer-readable storage medium storing a computer program / instructions, which, when executed by a processor, implements the above-described method.

[0206] The aforementioned readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic storage, flash memory, magnetic disk, or optical disk. The readable storage medium can be any available medium accessible to a general-purpose or special-purpose computer.

[0207] An exemplary readable storage medium is coupled to a processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can reside in an Application Specific Integrated Circuit (ASIC). Alternatively, the processor and the readable storage medium can exist as discrete components in the device.

[0208] The division of units is merely a logical functional division; in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be indirect coupling or communication connection through some interfaces, devices, or units, and may be electrical, mechanical, or other forms.

[0209] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0210] In addition, the functional units in the various embodiments of the present invention can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0211] If a function is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods of the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0212] Those skilled in the art will understand that all or part of the steps of the above-described method embodiments can be implemented by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When executed, the program performs the steps of the above-described method embodiments; and the aforementioned storage medium includes various media capable of storing program code, such as ROM, RAM, magnetic disks, or optical disks.

[0213] Finally, it should be noted that other embodiments of the invention will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This invention is intended to cover any variations, uses, or adaptations of the invention that follow the general principles of the invention and include common knowledge or customary techniques in the art not disclosed herein, and is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of the invention is limited only by the appended claims.

Claims

1. A multi-point displacement measurement method based on a single optical source, applied to a controller, characterized in that, The method includes: Acquire an original image, wherein the original image includes the object under test and multiple laser points; Extract the laser points from the original image to obtain the distorted 2D coordinates of each laser point; Establish the correspondence between each laser point and the emitted beam. in, When the column of multiple laser points is perpendicular to the sensor plane formed by the camera optical axis and the laser axis of the emitted beam array, the process includes: determining the number of laser points in each row region according to the preset row region; determining whether the number of laser points in each row region is the preset value, and if so, determining the column order of the laser points according to the distorted 2D coordinates of the multiple laser points in each row region; and determining the correspondence between the laser points and the emitted beam according to the row region and column order of the laser points. When the column of multiple laser points is not perpendicular to the sensor plane formed by the camera optical axis and the laser axis of the emitted beam array, it includes: directly matching the laser points and the corresponding emitted beams, wherein the emitted beams are independent lines in the original image; Based on pre-calibrated camera and laser parameters, the distorted 2D coordinates of each laser point, and the correspondence between laser points and emitted beams, the 3D coordinates of each laser point are calculated. Output the 3D coordinates of multiple laser points.

2. The method according to claim 1, characterized in that, Extract the laser points from the original image to obtain the distorted 2D coordinates of each laser point, including: Distinguish between light spot regions and background regions in the original image based on brightness thresholds; Mark multiple light spots in the light spot area; Calculate the centroid of each light spot to obtain the laser point; The distorted 2D coordinates of each laser point are obtained based on its position in the original image.

3. The method according to claim 1, characterized in that, Based on pre-calibrated camera and laser parameters, the distorted 2D coordinates of each laser point, and the correspondence between laser points and emitted beams, the 3D coordinates of each laser point are calculated, including: The distorted 2D coordinates of each laser point are corrected according to the camera parameters to obtain the undistorted 2D coordinates; Based on the camera parameters, the distortion-free 2D coordinates are converted into a normalized direction vector of the camera's line of sight; The intersection point is determined based on the propagation trajectory of the emitted beam corresponding to the laser point and the normalized direction vector to obtain the 3D coordinates of the laser point.

4. The method according to claim 1, characterized in that, When the column of multiple laser points is perpendicular to the sensor plane formed by the camera's optical axis and the laser axis of the emitted beam array, the 3D coordinates of the multiple laser points are output, including: The 3D coordinates of multiple laser points are arranged according to the corresponding distorted 2D coordinates, and when the number of laser points in each row region is not a preset value, the output of the 3D coordinates of the laser points in the corresponding row region is invalid.

5. A multi-point displacement measurement system based on a single optical source, performing the method as described in any one of claims 1-4, characterized in that, The system includes: A laser source is a source beam used to emit a single beam of light. A diffraction beam splitter is used to split a single source beam into an array of emitted beams. A camera is used to acquire the original image of the laser point on the object being measured, wherein the optical axis of the camera is set at a preset angle with the laser axis of the emitted beam array; The controller, coupled to the laser source and the camera respectively, is used to control the laser source to emit a source beam, receive the raw image output by the camera, and calculate and output the 3D coordinates of multiple laser points.

6. The system according to claim 5, characterized in that, The system also includes: A narrowband filter is placed in the imaging optical path between the camera and the object being measured, and its center wavelength is the same as the laser beam wavelength. It is used to filter out ambient stray light.

7. An electronic device, characterized in that, include: Memory, processor; The memory is used to store computer programs / instructions; The processor is configured to execute the computer program / instructions stored in the memory, causing the processor to perform the method as described in any one of claims 1-4.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program / instruction, which, when executed by a processor, is used to implement the method as described in any one of claims 1-4.

Images