Double-color-based multi-line laser measurement system and double-color-based multi-line laser measurement method

Abstract

The invention discloses a double-color-based multi-line laser measurement system and method, which are characterized in that the color of projected line laser is coded to ensure that only line laser strips with specified colors are contained in a certain pixel interval, and the double-color separation of the same frame of RGB image is realized by combining Raw data and a Bayer filter array to realize the distinguishing between the line laser strips; the height information of the to-be-measured object at each pixel point is demodulated and reconstructed according to the relative position relation between the camera and the laser strip, single-line laser is replaced by multi-line laser, the efficiency of the measurement system is higher, and the measurement depth can be remarkably enlarged according to a two-color coding mode.

Description

Technical Field

[0001] The present application relates to the field of laser measurement, and in particular to a dual-color based multi-line laser measurement system and method. Background Art

[0002] At present, the three-dimensional measurement methods based on line lasers are mainly divided into single-line laser and multi-line laser. The single-line laser measurement method uses a line laser to scan the object to be measured to achieve three-dimensional measurement. During the measurement process, there is always only one line laser in the entire space. The line laser completely scans the object to be measured and demodulates the surface morphology information of the object according to the relative position relationship between the camera and the line laser. The measurement method based on the single-line laser has low cost and strong anti-interference ability, but low efficiency; the multi-line laser measurement method uses multiple line lasers to scan the object to be measured at the same time during measurement, which will cause ambiguity problems while improving the measurement efficiency of the system. Multi-line lasers will cause serious deformation when irradiated on the object, and the original complete laser strip will be cut off, making it easy for the pixel points on the multi-line laser strip to cause the corresponding error of the laser stripes during the three-dimensional reconstruction process. It is impossible to determine which light knife plane the center point of the laser strip is located on, so it is mistakenly substituted into the equation of other light knife planes, causing reconstruction errors. The ambiguity problem leads to a contradiction between the depth range of the object to be measured and the efficiency of the measurement system. Summary of the invention

[0003] In view of the above-mentioned deficiencies in the prior art, the present invention provides a dual-color based multi-line laser measurement system and method, which introduces a dual-color multi-line laser, distinguishes the color coding method, and solves the ambiguity problem in the multi-line laser measurement method according to the relationship between different line lasers in a certain pixel area.

[0004] In order to achieve the above-mentioned invention object, the technical solution adopted by the present invention is: a dual-color multi-line laser measurement system, comprising:

[0005] The translation stage, located on the stepper motor, is used to place the object to be measured;

[0006] A stepper motor is used to drive the translation stage to move;

[0007] CCD camera, used to collect Raw data of the object to be tested;

[0008] A red line laser, used for generating a red line laser and emitting it to the color combining prism;

[0009] The blue line laser is placed orthogonally to the red line laser and is used to generate a blue line laser and direct it toward the color combining prism;

[0010] The color combining prism is used to combine the red line laser and the blue line laser to obtain a two-color line laser, and to direct the two-color line laser toward the object to be measured.

[0011] Further: the color combining prism includes four equilateral right-angle prisms, and the right-angle sides of each equilateral right-angle prism are attached to each other;

[0012] The right-angle side of each equilateral right-angle prism is also coated with a blue light reflecting film.

[0013] Further: the red line laser bar generated by the red line laser is used to obtain three-dimensional information of the object to be measured according to the relative position relationship between its center and the CCD camera;

[0014] The blue laser line produced by the blue laser is used to mark the position of the red laser line that may cause ambiguity;

[0015] The color coding method of the red laser bar and the blue laser bar adopts red and red and blue dual colors to cycle periodically.

[0016] The present invention also provides a multi-line laser measurement method based on a dual-color multi-line laser measurement system, comprising: S1, starting the multi-line laser measurement system, and capturing Raw data of the object to be measured by a CCD camera;

[0017] S2, extracting the laser bars in the Raw data through the host computer to obtain the center of the red laser bar and the center of the blue laser bar;

[0018] S3, according to the center of the red laser bar and the center of the blue laser bar, combined with the internal and external parameters of the CCD camera and the light knife plane equation, the world coordinates of the object to be measured are obtained;

[0019] S4. Perform three-dimensional reconstruction based on the world coordinates of the object to be measured to obtain the surface morphology information of the object.

[0020] Further: S1 includes:

[0021] S11, start the multi-line laser measurement system and color-code the red and blue lines;

[0022] S12, using a color combining prism to combine the red laser line and the blue laser line into red and blue lines, and projecting them onto the surface of the object to be measured;

[0023] S13, obtaining the Raw data of the laser stripes modulated by the object morphology through the color CCD.

[0024] Further: S2 includes:

[0025] S21, read the Raw data through the host computer, and decompose the image of the same frame in the Raw data into a red image and a blue image through the Bayer filter template;

[0026] S22, using an interpolation algorithm to restore the red information and blue information filtered out by the Bayer filter template in the red image and the blue image, to obtain a full red image of the red component information of each pixel and a full blue image of the blue component information of each pixel;

[0027] S23, binarize the full red image of the red component information of each pixel and the full blue image of the blue component information of each pixel respectively, to obtain the center of the laser line in the full red image and the center of the laser line in the full blue image as the center of the red laser bar and the center of the blue laser bar.

[0028] The beneficial effects of the present invention are:

[0029] A two-color line laser is introduced into the measurement system. The relationship between different line lasers in a certain pixel area is distinguished by color coding, which solves the ambiguity problem in the multi-line laser measurement method, improves the measurement efficiency of the system and increases the depth range of the object to be measured. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 A schematic diagram of a measurement system and method according to an embodiment.

[0031] Figure 2 The figure is a flow chart of the measurement system method.

[0032] Figure 3 A Bayer color template diagram adopted by a color CCD camera used in an embodiment.

[0033] Figure 4 Schematic diagram of the object to be tested.

[0034] Figure 5 A schematic diagram of projected coded line structured light according to an embodiment.

[0035] Figure 6 FIG. 1 is a schematic diagram of a red laser bar after color separation according to an embodiment.

[0036] Figure 7 A schematic diagram of an extracted red laser stripe according to an embodiment.

[0037] Figure 8 A schematic diagram of a blue laser bar after color separation according to an embodiment.

[0038] Fig. 9 A schematic diagram of an extracted blue laser stripe according to an embodiment.

[0039] Fig.10 A schematic diagram of a measurement system and method according to another embodiment.

[0040] Fig.11This is a diagram showing the position relationship between the color CCD camera and the multi-line laser.

[0041] Fig.12 A schematic diagram of projected coded line structured light according to another embodiment.

[0042] Fig.13 FIG. 4 is a schematic diagram of a blue laser bar after color separation according to another embodiment.

[0043] Fig.14 A schematic diagram of an extracted blue laser bar according to another embodiment.

[0044] Fig.15 FIG. 4 is a schematic diagram of a red laser bar after color separation according to another embodiment.

[0045] Fig.16 FIG. 1 is a schematic diagram of an extracted red laser stripe according to another embodiment. DETAILED DESCRIPTION

[0046] The specific implementation modes of the present invention are described below so that those skilled in the art can understand the present invention. However, it should be clear that the present invention is not limited to the scope of the specific implementation modes. For those of ordinary skill in the art, as long as various changes are within the spirit and scope of the present invention as defined and determined by the attached claims, these changes are obvious, and all inventions and creations utilizing the concept of the present invention are protected.

[0047] Embodiment 1:

[0048] like Figure 1 As shown, in one embodiment of the present invention, a dual-color based multi-line laser measurement system is provided, comprising:

[0049] The translation stage is placed horizontally on the stepper motor and is used to place the object to be measured;

[0050] The stepper motor is used to drive the translation stage to move at a constant speed in a direction parallel to the CCD camera;

[0051] CCD camera, placed horizontally, used to collect Raw data of the object to be tested;

[0052] A red line laser, used for generating a red line laser and emitting it to the color combining prism;

[0053] The blue line laser is placed orthogonally to the red line laser and is used to generate a blue line laser and direct it toward the color combining prism;

[0054] The color combining prism is placed at an angle and is used to combine the red line laser and the blue line laser to obtain a two-color line laser, and then direct the two-color line laser toward the object to be measured.

[0055] Further, the color combining prism includes four equilateral right-angle prisms, and the right-angle sides of each equilateral right-angle prism are attached to each other;

[0056] The right-angled sides of each equilateral right-angle prism are also coated with a blue light reflection film; the blue light reflection film is a mirror for blue light, which will reflect blue light; for red light, it is equivalent to a piece of glass and can be transmitted. The red and blue two-color lasers are incident from two directions respectively, and after being reflected by the color-combining prism, they are emitted from the same direction, forming a red and blue two-color image.

[0057] Specifically, the red line laser bar generated by the red line laser is used to obtain three-dimensional information of the object to be measured based on the relative position relationship between its center and the CCD camera;

[0058] The blue laser line produced by the blue laser is used to mark the position of the red laser line that may cause ambiguity;

[0059] The color coding method of the red laser bar and the blue laser bar adopts red and red and blue dual colors to cycle periodically.

[0060] The encoding method ensures that the number of red light bars is twice that of blue light bars, and that red light bars must exist where blue light bars exist. The three-dimensional information of the object to be measured can be obtained based on the relative position relationship between the center of the red laser light bar and the camera. The purpose of the blue light bar is to mark the position of the red light bar that may cause ambiguity. The advantage of this design is that the calibration process of the system is simple, and the red and blue light bars can use the same set of calibration data. Other color encoding methods can also be designed according to the depth range of the object to be measured. The more complex the encoding method, the greater the depth range of measurement can be achieved, but the system design will also be more complex.

[0061] like Figure 2 As shown, this embodiment also provides a dual-color multi-line laser measurement method, which is used based on a multi-line laser measurement system, including:

[0062] S1. Start the multi-line laser measurement system and capture the Raw data of the object to be measured through the CCD camera;

[0063] S2, extracting the laser bars in the Raw data through the host computer to obtain the center of the red laser bar and the center of the blue laser bar;

[0064] S3, according to the center of the red laser bar and the center of the blue laser bar, combined with the internal and external parameters of the CCD camera and the light knife plane equation, the world coordinates of the object to be measured are obtained;

[0065] S4. Perform three-dimensional reconstruction based on the world coordinates of the object to be measured to obtain the surface morphology information of the object.

[0066] Specifically, S1 includes:

[0067] S11, start the multi-line laser measurement system and color-code the red and blue lines;

[0068] S12, using a color combining prism to combine the red laser line and the blue laser line into red and blue lines, and projecting them onto the surface of the object to be measured;

[0069] S13, obtaining the Raw data of the laser stripes modulated by the object morphology through the color CCD.

[0070] In this embodiment, the object to be measured is placed on a stepper motor, and the stepper motor drives the object to be measured to move at a constant speed toward the turned-on CCD camera until the red and blue laser lines scan the entire surface of the object to be measured, thereby obtaining Raw data of the object to be measured.

[0071] It should be emphasized that the stepper motor and translation method are only a feasible implementation method and are not mandatory. Other methods of scanning the red and blue laser lines across the entire surface of the object to be measured can be used, such as rotating or moving a color CCD camera.

[0072] Specifically, S2 includes:

[0073] S21, read the Raw data through the host computer, and decompose the image of the same frame in the Raw data into a red image and a blue image through the Bayer filter template;

[0074] S22, using an interpolation algorithm to restore the red information and blue information filtered out by the Bayer filter template in the red image and the blue image, to obtain a full red image of the red component information of each pixel and a full blue image of the blue component information of each pixel;

[0075] In single CCD cameras, digital cameras based on Bayer templates are the most widely used. The four commonly used Bayer filter array patterns are BGGR, GBRG, GRBG, and RGGB. The Bayer color template used by the color CCD camera used in this application is as follows: Figure 3 As shown;

[0076] When using the interpolation algorithm to restore the filtered color, when restoring red, the green of the first row (odd row) is obtained by the weighted average of the two adjacent reds in the same row; the green of the second row (even row) is obtained by the weighted average of the two adjacent reds in the same column; the blue of the second row (even row) is obtained by the weighted average of the reds of the four adjacent diagonals.

[0077] When restoring blue, the method is the same as red.

[0078] S23, binarize the full red image of the red component information of each pixel and the full blue image of the blue component information of each pixel respectively, to obtain the center of the laser line in the full red image and the center of the laser line in the full blue image as the center of the red laser bar and the center of the blue laser bar.

[0079] Specifically, S3 includes:

[0080] S31, determining the internal and external parameters of the camera through system calibration;

[0081] S32, determining the vertical distance from the camera coordinate origin to the light knife plane through the light knife plane equation of the line structured light in the camera coordinate system;

[0082] S33, according to the CCD camera perspective model, determine the point (u, v) in the image pixel coordinate system and the corresponding point (x) in the world coordinate system w ,y w ,z w ) to obtain the world coordinates (x w ,y w ,z w ).

[0083] The actual measurement steps are:

[0084] 1. Use a color-combining prism to combine two wavelengths of laser light into red and blue lines, project the coded colored light strips onto the object, and the CCD captures the fringe pattern that carries the surface information of the object;

[0085] 2. The stepper motor drives the object to be measured on the translation stage to move at a constant speed in the horizontal direction, so that the line laser light strip scans the entire surface of the object to be measured;

[0086] 3. Apply the Bayer filter template to the collected Raw data to achieve two-color separation, restore the full red and full blue images according to the interpolation algorithm, extract the center information of the red and blue light strips, and determine the number of the light strips according to the pixel interval division;

[0087] 4. Calculate the surface morphology information of the object at each pixel according to the positional relationship between the numbered light strips and the camera. The specific method includes:

[0088] By calibrating the camera, we can get the camera's intrinsic parameter matrix and distortion coefficient. By calibrating the laser plane, we can get the equation of the laser plane in the camera coordinate system. The laser plane equation in the camera coordinate system is:

[0089] aX c +bY c +cZ c +d=0 (1-1)

[0090] If the coordinates of the center point of a laser stripe in the pixel coordinate system are known to be (u, v), and its corresponding homogeneous coordinates are (u, v, 1), the coordinates of this point in the camera coordinate system (X c ,Y c ,Z c ) can be expressed by formula (1-2):

[0091]

[0092] But Z c is unknown, so the point (X c ,Y c ,Z c ) is transformed into the normalized plane, and we get (x c ,y c ,1), which can be expressed as:

[0093]

[0094] Among them, K is the camera intrinsic parameter matrix.

[0095] In the camera coordinate system, a line passes through the optical center O of the camera. c The straight line between point P and point P in space can be set as the two-point equation of the space straight line, expressed as:

[0096]

[0097] Among them, (X1, Y1, Z1) and (X2, Y2, Z2) represent the coordinates of two known three-dimensional points in the space. The camera optical center (0, 0, 0) and the point (x c ,y c ,1)Substitute into the above formula and obtain:

[0098]

[0099] The center point of the laser stripe satisfies both the laser plane equation and the straight line equation in the camera coordinate system. Therefore, the three-dimensional coordinates (X c ,Y c ,Z c ), as shown in formula (1-6):

[0100]

[0101] Then, we can use the camera external parameters to continue to transform point P into the world coordinate system:

[0102]

[0103] At this point, the three-dimensional coordinates of the center point p of the laser stripe are calculated. Since the center points of other laser stripes must also satisfy the laser plane equation (1-1) in the camera coordinate system, by repeating this process, the three-dimensional coordinates of all the center points of the laser stripes in the image taken by the camera in the camera coordinate system or the world coordinate system can be calculated to complete the three-dimensional reconstruction of this section of laser stripes.

[0104] Figures 4 to 9 The results of applying the method described in this application to separate two-color laser strips are shown in FIG. Figure 4 is a schematic diagram of the object to be tested. Figure 5 This is a schematic diagram of the line structured light after projection coding. Figure 6 This is a schematic diagram of the red laser strip after color separation. Figure 7 Schematic diagram of the extracted red laser strip. Figure 8 This is a schematic diagram of the blue laser bar after color separation. Fig. 9 Schematic diagram of the extracted blue laser bar.

[0105] according to Figure 4-Figure 9 It can be seen that the measurement method proposed in this application is accurate in measurement and solves the ambiguity problem in the multi-line laser measurement method while maintaining a large measurement depth range.

[0106] Embodiment 2:

[0107] like Fig.10 As shown, this embodiment, as another implementation of the present application, provides another dual-color based multi-line laser measurement system;

[0108] The system includes:

[0109] The translation stage is placed horizontally on the stepper motor and is used to place the object to be measured;

[0110] The stepper motor is used to drive the translation stage to move at a constant speed in a direction parallel to the CCD camera;

[0111] CCD camera, placed horizontally, used to collect Raw data of the object to be tested;

[0112] Red line laser, used to generate red multi-line laser and project it onto the surface of the object to be measured;

[0113] The blue line laser is used to generate a blue line laser and project a line laser orthogonal to the red line laser onto the surface of the object to be measured to form a laser grid.

[0114] The position relationship between the CCD camera and the multi-line laser, such as Fig.11As shown, when arranging the system, the camera is placed in the middle perpendicular to the object to be measured, a laser that projects horizontal line laser is placed below the camera, and a laser that projects vertical line laser is placed to the right of the camera; the optical axes of the camera and the laser form a plane, and the direction of the marking light strip should be perpendicular to the plane; the purpose of this arrangement is to ensure that the triangulation method can cause the stripes to move significantly.

[0115] The dual-color based multi-line laser measurement method includes:

[0116] S1. Start the multi-line laser measurement system and capture the Raw data of the object to be measured through the CCD camera;

[0117] S2, extracting the laser bars in the Raw data through the host computer to obtain the center of the red laser bar and the center of the blue laser bar;

[0118] S3, according to the center of the red laser bar and the center of the blue laser bar, combined with the internal and external parameters of the CCD camera and the light knife plane equation, the world coordinates of the object to be measured are obtained;

[0119] S4. Perform three-dimensional reconstruction based on the world coordinates of the object to be measured to obtain the surface morphology information of the object.

[0120] In this embodiment, sparse 3D data of the object to be measured is obtained through the intersection of red and blue laser lines, but such sparse 3D data cannot provide sufficient surface information. The subsequent step of the method is to detect the common 3D points (laser intersection points) in the subsequent 3D views.

[0121] In this embodiment, the schematic diagram of the line structured light after projection coding is as follows: Fig.12 As shown; Schematic diagram of the blue laser bar after color separation, as shown Fig.13 As shown; Schematic diagram of the extracted blue laser bar, as shown Fig.14 As shown; Schematic diagram of the red laser bar after color separation, as shown Fig.15 As shown; the schematic diagram of the extracted red laser bar is as follows Fig.16 shown.

[0122] Each view only provides three-dimensional data along the projection line. When the object to be measured moves during the measurement process, continuous views usually have no common 3D points. These points at different times are pulled together in the form of point clouds to align the sparse 3D views to obtain a dense 3D model of the captured object.

[0123] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A dual-color multi-line laser measurement system, characterized in that: include: The translation stage, located on the stepper motor, is used to place the object to be measured; A stepper motor is used to drive the translation stage to move; CCD camera, used to collect Raw data of the object to be tested; A red line laser, used for generating a red line laser and emitting it to the color combining prism; The blue line laser is placed orthogonally to the red line laser and is used to generate a blue line laser and direct it toward the color combining prism; The color combining prism is used to combine the red line laser and the blue line laser to obtain a two-color line laser, and to direct the two-color line laser toward the object to be measured.

2. The dual-color multi-line laser measurement system according to claim 1, characterized in that: The color-combining prism includes four equilateral right-angle prisms, and the right-angle sides of each equilateral right-angle prism are attached to each other; The right-angle side of each equilateral right-angle prism is also coated with a blue light reflecting film.

3. The dual-color multi-line laser measurement system according to claim 1, characterized in that: The red line laser bar generated by the red line laser is used to obtain the three-dimensional information of the object to be measured based on the relative position relationship between its center and the CCD camera; The blue laser line produced by the blue laser is used to mark the position of the red laser line that may cause ambiguity; The color coding method of the red laser bar and the blue laser bar adopts red and red and blue dual colors to cycle periodically.

4. A dual-color multi-line laser measurement method, based on the multi-line laser measurement system according to any one of claims 1 to 3, characterized in that: include: S1. Start the multi-line laser measurement system and capture the Raw data of the object to be measured through the CCD camera; S2, extracting the laser bars in the Raw data through the host computer to obtain the center of the red laser bar and the center of the blue laser bar; S3, according to the center of the red laser bar and the center of the blue laser bar, combined with the internal and external parameters of the CCD camera and the light knife plane equation, the world coordinates of the object to be measured are obtained; S4. Perform three-dimensional reconstruction based on the world coordinates of the object to be measured to obtain the surface morphology information of the object.

5. The dual-color multi-line laser measurement method according to claim 4, characterized in that: S1 includes: S11, start the multi-line laser measurement system and color-code the red and blue lines; S12, using a color combining prism to combine the red laser line and the blue laser line into red and blue lines, and projecting them onto the surface of the object to be measured; S13, obtaining the Raw data of the laser stripes modulated by the object morphology through the color CCD.

6. The dual-color multi-line laser measurement method according to claim 4, characterized in that S2 include: S21, read the Raw data through the host computer, and decompose the image of the same frame in the Raw data into a red image and a blue image through the Bayer filter template; S22, using an interpolation algorithm to restore the red information and blue information filtered out by the Bayer filter template in the red image and the blue image, to obtain a full red image of the red component information of each pixel and a full blue image of the blue component information of each pixel; S23, binarize the full red image of the red component information of each pixel and the full blue image of the blue component information of each pixel respectively, to obtain the center of the laser line in the full red image and the center of the laser line in the full blue image as the center of the red laser bar and the center of the blue laser bar.

Images