A method, device, equipment and storage medium for locating line structure light

Abstract

The present application discloses a line structured light calibration method, including: calibrating the internal parameters in the tilt camera; projecting line structured light to the first plane and the second plane with a height difference on the same side of the preset target through a line structured light laser, and controlling the light stripes formed on the first plane and the second plane to be located in the middle of the plane, and then performing image acquisition to obtain a corresponding light stripe image; determining the coordinate values ​​of the feature points of the light stripe image on the light stripe image and determining the coordinate values ​​of the target feature points corresponding to the feature points of the light stripe image from the target based on the internal parameters in the tilt camera; constructing the constraint equation of the light plane parameters based on the height difference between the first plane and the second plane, and solving the light plane parameters based on the coordinate values ​​of the feature points of the light stripe image, the coordinate values ​​of the target feature points and the constraint equation to obtain the corresponding light plane parameters. The present invention is simple to operate, simple to calculate and improves the calibration accuracy.

Description

Technical Field

[0001] The present invention relates to the field of visual measurement technology, and in particular to a line structure light positioning method, device, equipment and storage medium. Background Art

[0002] In recent years, line structured light 3D scanning technology, as an active non-contact scanning technology, has been widely used in 3D reconstruction, product inspection and other fields due to its advantages such as large measurement range, fast measurement speed and high measurement accuracy.

[0003] The calibration process of line structured light vision sensor includes two aspects: camera internal parameter calibration and light plane calibration. There are many literatures on camera calibration, so the light plane calibration process is mainly discussed. Common methods for light plane calibration include wire drawing method, sawtooth calibration method, cross ratio invariant method and plane target based calibration method. The wire drawing method measures the three-dimensional coordinates of multiple light points on the filament, and then obtains the light plane parameters by fitting. However, the wire drawing method requires other instruments to measure the coordinate values ​​of the bright spots in space, which is complicated to operate and has low accuracy. The sawtooth calibration method calibrates the x-direction and z-direction values ​​of the contour surface by the tooth width and tooth height of a single sawtooth. During calibration, the sawtooth bevel is obtained by obtaining 80% of the data of the sawtooth bevel, and then a straight line is fitted through the bevel data to obtain the intersection of the two oblique lines. However, the sawtooth target in the sawtooth calibration method is complicated to process, and the method of obtaining the intersection by fitting the straight line through the bevel has a large error. The principle of the cross-ratio invariance method is to generate the calibration point coordinates required for calibrating the structured light parameters through at least three collinear feature points with known coordinates. The small number of calibration points obtained based on the cross-ratio invariance method leads to low accuracy. Most methods based on plane targets first determine the plane target equation based on the feature points on the plane target, then determine the light stripe equation on the target, move the target multiple times, obtain the equations of multiple straight lines located on the same light plane in the camera coordinate system, and then fit the light plane. However, the method based on plane targets requires multiple calibrations of target planes with different postures, which is complex to calculate and easy to transmit errors.

[0004] In summary, how to achieve simple operation, simple calculation and simple target manufacturing in the optical plane calibration process, reduce calibration errors and improve calibration accuracy are problems to be solved in this field. Summary of the invention

[0005] In view of this, the purpose of the present invention is to provide a line structure light calibration method, device, equipment and storage medium, so as to realize the light plane calibration process with simple operation, simple calculation and simple target production, reduce the calibration error and improve the calibration accuracy. The specific scheme is as follows:

[0006] In a first aspect, the present application discloses a line structure light positioning method, comprising:

[0007] Calibrate the internal parameters of the tilt camera and establish the transformation matrix between the camera coordinate system and the pixel coordinate system;

[0008] Projecting line structured light onto a first plane and a second plane with a height difference on the same side of a preset target by a line structured light laser, and controlling the light stripes formed on the first plane and the second plane to be located at the middle position of the plane, and then performing image acquisition to obtain a corresponding light stripe image;

[0009] Determining coordinate values ​​of light streak image feature points in the light streak image and determining coordinate values ​​of target feature points corresponding to the light streak image feature points from a target based on the internal parameters in the tilt camera;

[0010] A constraint equation of a light plane parameter is constructed based on the height difference between the first plane and the second plane, and the light plane parameter is solved based on the coordinate values ​​of the feature points of the light streak image, the coordinate values ​​of the target feature points and the constraint equation to obtain the corresponding light plane parameter.

[0011] Optionally, projecting a line structured light onto a first plane and a second plane with a height difference on the same side of a preset target by a line structured light laser, and controlling the light stripes formed on the first plane and the second plane to be located at the middle position of the plane, comprises:

[0012] A line structured light laser is used to project line structured light onto a first plane and a second plane with a height difference on the same side of a preset concave target and / or a preset convex target, and the light stripes formed on the first plane and the second plane are controlled to be located in the middle of the plane.

[0013] Optionally, the performing image acquisition to obtain a corresponding light streak image includes:

[0014] The posture of the preset concave target and / or the preset convex target is adjusted, and images of the preset concave target and / or the preset convex target in different postures are captured to obtain light stripe images in corresponding different postures.

[0015] Optionally, before determining the coordinate values ​​of the light streak image feature points in the light streak image, the method further includes:

[0016] Performing image denoising on the light streak image to obtain a denoised light streak image.

[0017] Optionally, determining the coordinate values ​​of the light streak image feature points in the light streak image includes:

[0018] Extracting the sub-pixel light stripe center of the light stripe image located on the first plane, and averaging the image coordinate values ​​on the sub-pixel light stripe center of the first plane to obtain the coordinate value corresponding to the feature point of the light stripe image on the first plane; wherein the preset target includes two first planes and one second plane;

[0019] The sub-pixel light stripe center of the light stripe image located in the second plane is extracted, the sub-pixel light stripe center located in the second plane is divided into two segments of light stripe centers of equal length, and the image coordinate values ​​of each segment of the light stripe center are averaged to obtain coordinate values ​​corresponding to the light stripe image feature points on each segment of the light stripe center.

[0020] Optionally, before solving the light plane equation, the method further includes:

[0021] Determine a mapping equation between the image feature points and the corresponding target feature points based on the calibrated internal parameters in the tilt camera and the tilt camera imaging model;

[0022] Based on the characteristic that the target feature points corresponding to the feature points of the light fringe image are located in the light plane, constructing the light plane equation;

[0023] Based on the characteristic that the distance between the light streak image feature point on the second plane and the target line segment is consistent with the height difference, a constraint equation for the light plane parameter is constructed; the target line segment is a line connecting a first light streak image feature point on the first plane and a second light streak image feature point on the first plane.

[0024] Optionally, solving the light plane equation to obtain corresponding light plane calibration parameters includes:

[0025] constructing an objective function about the light plane parameters based on the mapping equation, the light plane equation, and the constraint equation;

[0026] The objective function is solved using a nonlinear optimization method to obtain an optimal solution for the light plane parameters under a maximum likelihood criterion.

[0027] In a second aspect, the present application discloses a linear structured optical positioning device, comprising:

[0028] The parameter calibration module is used to calibrate the internal parameters of the tilt camera and establish a transformation matrix between the camera coordinate system and the pixel coordinate system;

[0029] An image acquisition module, used to project a line structured light onto a first plane and a second plane with a height difference on the same side of a preset target through a line structured light laser, and control the light stripes formed on the first plane and the second plane to be located in the middle of the plane, and then perform image acquisition to obtain a corresponding light stripe image;

[0030] a coordinate determination module, used to determine the coordinate value of the light streak image feature point in the light streak image and determine the coordinate value of the target feature point corresponding to the light streak image feature point from the target based on the internal parameters in the tilt camera;

[0031] An equation solving module is used to construct a constraint equation of a light plane parameter based on the height difference between the first plane and the second plane, and solve the light plane parameter based on the coordinate values ​​of the feature points of the light streak image, the coordinate values ​​of the target feature points and the constraint equation to obtain the corresponding light plane parameter.

[0032] In a third aspect, the present application discloses an electronic device, including:

[0033] Memory for storing computer programs;

[0034] The processor is used to execute the computer program to implement the steps of the line structure light positioning method disclosed above.

[0035] In a fourth aspect, the present application discloses a computer-readable storage medium for storing a computer program; wherein, when the computer program is executed by a processor, the steps of the line structure light positioning method disclosed above are implemented.

[0036] It can be seen that the present application first calibrates the internal parameters in the tilt camera and establishes a conversion matrix between the camera coordinate system and the pixel coordinate system; projects a line structured light onto a first plane and a second plane with a height difference on the same side of a preset target through a line structured light laser, and controls the light stripes formed on the first plane and the second plane to be located in the middle of the plane, and then performs image acquisition to obtain a corresponding light stripe image; determines the coordinate values ​​of the feature points of the light stripe image in the light stripe image, and determines the coordinate values ​​of the target feature points corresponding to the feature points of the light stripe image from the target based on the internal parameters in the tilt camera; constructs a constraint equation for the light plane parameters based on the height difference between the first plane and the second plane, and solves the light plane parameters based on the coordinate values ​​of the feature points of the light stripe image, the coordinate values ​​of the target feature points and the constraint equation to obtain the corresponding light plane parameters. It can be seen that the target in the present application is simple to manufacture, and the operation process of controlling the projection of the line structured light onto the target plane is simple. Then, by extracting the center of the light streak in the light streak image, the feature points of the light streak image and the corresponding target feature points are determined, so that the selection of the feature points of the light streak image is easier and more representative. In addition, according to the preset target, it can be ensured that the light plane and the target side plane maintain an approximately parallel relationship, and the height difference between the same side planes with a smaller error can be obtained. Moreover, according to the coordinate values ​​of the feature points of the light streak image, the corresponding target feature points and the height difference, a simple constraint condition can be constructed, so that the solution of the light plane equation is simpler and more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0038] Figure 1 A flow chart of a line structure light positioning method disclosed in this application;

[0039] Figure 2 A schematic diagram of a light plane calibration target for an inclined camera disclosed in this application;

[0040] Figure 3 Schematic diagram of a light streak image taken by a tilted camera disclosed in this application

[0041] Figure 4 A flowchart of a specific line structure light determination method disclosed in this application;

[0042] Figure 5This is a schematic diagram of the structure of a linear structured optical positioning device disclosed in this application;

[0043] Figure 6 This is a structural diagram of an electronic device disclosed in this application. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Currently, the light plane calibration process plays an important role in the online structured light vision sensor calibration process. However, the existing several commonly used calibration methods for the light plane calibration process have problems such as complex operation, low precision, large measurement error and complex target production. For this reason, the embodiment of the present application discloses a line structured light calibration scheme, which can achieve simple operation, simple calculation and simple target production in the light plane calibration process, reduce calibration errors and improve calibration accuracy.

[0046] Reference Figure 1 As shown, an embodiment of the present invention discloses a line structure light positioning method, including:

[0047] Step S11: calibrate the internal parameters of the tilt camera and establish a conversion matrix between the camera coordinate system and the pixel coordinate system.

[0048] In this embodiment, according to the conventional tilt lens calibration method, the calibration plate images in different postures are taken, and the internal parameters of the tilt camera are calibrated to obtain the internal parameters such as camera focal length, distortion coefficient, pixel spacing, lens tilt angle, etc., so as to use the internal parameters to establish the conversion matrix between the camera coordinate system and the pixel coordinate system corresponding to the line structured light vision sensor. Furthermore, the line structured light calibration method in this application is applicable not only to the tilt camera because the tilt lens of the tilt camera has a large depth of field and can clearly capture targets occupying a large range, but also to ordinary cameras.

[0049] Step S12: Projecting line structured light onto a first plane and a second plane with a height difference on the same side of a preset target through a line structured light laser, and controlling the light stripes formed on the first plane and the second plane to be located in the middle position of the plane, and then performing image acquisition to obtain a corresponding light stripe image.

[0050] In this embodiment, a line structured light laser is used to project line structured light onto a first plane and a second plane with a height difference on the same side of a preset concave target and / or a preset convex target, and the light stripes formed on the first plane and the second plane are controlled to be located in the middle of the plane. It can be understood that a concave target and / or a convex target is pre-designed, and two planes with a height difference on the same side of the target are set as the first plane and the second plane, and the target includes two first planes; then the line structured light laser projects line structured light to the first plane and the second plane with a height difference on the same side of the preset target, and controls the light stripes formed on the first plane and the second plane to be located in the middle of the plane. When the line structured light is projected onto the first plane and the second plane of the target, it will intersect with the plane to produce a bright light stripe. Since there is a certain height difference between the first plane and the second plane of the target, the formed light stripes are distorted. The distortion is related to the height difference. Therefore, the three-dimensional coordinates of the light stripes on the target can be obtained in reverse through the mapping relationship. The control of the formed light stripes to be located in the middle of the plane is an indispensable condition of the embodiment of the present invention. At this time, the light plane can be approximately considered to be parallel to the side plane of the preset target. Reference Figure 2 As shown in the figure, a preset concave target is shown, wherein, from the main view, top view and stereoscopic three-dimensional view of the preset concave target, it can be seen that the first plane 1 and the first plane 3 are the first planes of the preset concave target, and the second plane 2 is the second plane of the preset concave target, and the line structured light laser projects the line structured light from the first plane 1, the first plane 3 and the second plane 2 in the direction of the top view of the preset concave target, and controls the light stripes formed on the first plane 1, the first plane 3 and the second plane 2 to be located in the middle position of the plane.

[0051] In this embodiment, the position of the preset concave target and / or the preset convex target is adjusted, and images of the preset concave target and / or the preset convex target in different positions are collected to obtain light streak images in corresponding different positions. It can be understood that, referring to Figure 3 As shown, the position of the tilted camera is adjusted so that the light plane is within the depth of field of the camera; then the target is placed at a suitable position, the position of the target is changed, and N (N>2) images are taken to obtain N light streak images in corresponding different positions.

[0052] Step S13: determining the coordinate values ​​of the light streak image feature points in the light streak image and determining the coordinate values ​​of the target feature points corresponding to the light streak image feature points from the target based on the internal parameters in the tilt camera.

[0053] In this embodiment, after obtaining the light streak images of different positions corresponding to the target at different positions, it is necessary to perform image denoising on the light streak images to obtain denoised light streak images. It can be understood that the denoised light streak images obtained after denoising the light streak images are clearer, more adaptive, stable and accurate; the noise in the light streak images may specifically include but are not limited to: environmental noise, hardware noise, etc.; the image denoising processing means may specifically include but are not limited to: mean denoising, median denoising, denoising based on partial differential equations, etc.

[0054] Furthermore, after acquiring the denoised light streak image, the coordinate values ​​of the light streak image feature points are further determined from the denoised light streak image, and the coordinate values ​​of the target feature points corresponding to the light streak image feature points are determined from the target based on the internal parameters in the tilted camera. It can be understood that the coordinate values ​​of the light streak image feature points located in the pixel coordinate system are determined, and further, the coordinate values ​​of the target feature points corresponding to the light streak image feature points determined in the target in the camera coordinate system are obtained by calculation, and the coordinate values ​​of the target feature points obtained at this time are unknown coordinates. That is, the corresponding relationship between the light streak image feature points and the target feature points is established using the tilted camera model.

[0055] Step S14: constructing a constraint equation of the light plane parameters based on the height difference between the first plane and the second plane, and solving the light plane parameters based on the coordinate values ​​of the feature points of the light streak image, the coordinate values ​​of the target feature points and the constraint equation to obtain corresponding light plane parameters.

[0056] It can be seen that the present application first calibrates the internal parameters in the tilt camera and establishes a conversion matrix between the camera coordinate system and the pixel coordinate system; projects a line structured light onto a first plane and a second plane with a height difference on the same side of a preset target through a line structured light laser, and controls the light stripes formed on the first plane and the second plane to be located in the middle of the plane, and then performs image acquisition to obtain a corresponding light stripe image; determines the coordinate values ​​of the feature points of the light stripe image in the light stripe image, and determines the coordinate values ​​of the target feature points corresponding to the feature points of the light stripe image from the target based on the internal parameters in the tilt camera; constructs a constraint equation for the light plane parameters based on the height difference between the first plane and the second plane, and solves the light plane parameters based on the coordinate values ​​of the feature points of the light stripe image, the coordinate values ​​of the target feature points and the constraint equation to obtain the corresponding light plane parameters. It can be seen that the target in the present application is simple to manufacture, and the operation process of controlling the projection of the line structured light onto the target plane is simple. Then, by extracting the center of the light streak in the light streak image, the feature points of the light streak image and the corresponding target feature points are determined, so that the selection of the feature points of the light streak image is easier and more representative. In addition, according to the preset target, it can be ensured that the light plane and the target side plane maintain an approximately parallel relationship, and the height difference between the same side planes with a smaller error can be obtained. Moreover, according to the coordinate values ​​of the feature points of the light streak image, the corresponding target feature points and the height difference, a simple constraint condition can be constructed, so that the solution of the light plane equation is simpler and more accurate.

[0057] Reference Figure 4 As shown, the embodiment of the present invention discloses a specific line structure light positioning method. Compared with the previous embodiment, this embodiment further illustrates and optimizes the technical solution. Specifically:

[0058] Step S21: calibrate the internal parameters of the tilt camera and establish a conversion matrix between the camera coordinate system and the pixel coordinate system.

[0059] Step S22: projecting line structured light onto a first plane and a second plane with a height difference on the same side of a preset target through a line structured light laser, and controlling the light stripes formed on the first plane and the second plane to be located in the middle position of the plane, and then performing image acquisition to obtain a corresponding light stripe image.

[0060] For more specific processing procedures of steps S21 and S22, reference may be made to the corresponding contents disclosed in the aforementioned embodiments, which will not be described in detail here.

[0061] Step S23: extracting the sub-pixel light stripe center of the light stripe image located on the first plane, and averaging the image coordinate values ​​on the sub-pixel light stripe center of the first plane to obtain coordinate values ​​corresponding to feature points of the light stripe image on the first plane; wherein the preset target includes two first planes and one second plane.

[0062] In this embodiment, after image acquisition is performed on the structured light stripes projected on the first plane and the second plane to obtain the corresponding light stripe images, the sub-pixel light stripe centers located on the first plane 1 and the first plane 3 in the light stripe images are extracted, and the image coordinates of the light stripe centers located on the first plane 1 and the first plane 3 are averaged, which are (u j1 ,v j1 ) and (u j4 ,v j4 ). Wherein, each image coordinate at the center of the light stripes on the first plane 1 and the first plane 3 is located in the image coordinate in the pixel coordinate system, and the image coordinates are averaged to obtain the (u j1 ,v j1 ) and (u j4 ,v j4 ) coordinate values ​​are also located on the first plane 1 and the first plane 3.

[0063] Step S24: extracting the sub-pixel light stripe center of the light stripe image located in the second plane, dividing the sub-pixel light stripe center located in the second plane into two segments of light stripe centers of equal length, and averaging the image coordinate values ​​of each segment of the light stripe center to obtain coordinate values ​​corresponding to the light stripe image feature points on each segment of the light stripe center.

[0064] In this embodiment, the light stripes on the second plane 2 are roughly divided into two segments of equal length, and the image coordinate values ​​of the centers of the light stripes of the two segments are averaged, which are (u j2 ,v j2 ) and (u j3 ,v j3 ), then according to the above-mentioned first plane 1 and the first plane 3 collected (u j1 ,v j1 ) and (u j4 ,v j4 ) provides corresponding coordinate values ​​for establishing constraint conditions later.

[0065] Step S25: Based on the internal parameters in the tilt camera, the coordinate values ​​of the target feature points corresponding to the light streak image feature points, the coordinate values ​​of the light streak image feature points, and the height difference between the first plane and the second plane are determined from the target to construct a mapping equation, a light plane equation, and a constraint equation of the light plane parameters.

[0066] In this embodiment, the mapping equation between the image feature point and the corresponding target feature point is determined based on the calibrated internal parameters in the tilt camera and the tilt camera imaging model; first, the corresponding relationship between the coordinate value corresponding to the light streak image feature point and the coordinate value of the target feature point is established, that is, the mapping equation, as shown in Formula 1:

[0067]

[0068] Among them, (u ji ,v ji ) is the coordinate value corresponding to the i-th (i=1,2,3,4)-th light fringe feature point on the j-th (j=1,2...N)-th light fringe image, u ji is the horizontal coordinate corresponding to the characteristic point of the light stripe, v ji is the ordinate corresponding to the light stripe feature point, i is the point corresponding to the light stripe feature point, j is the current light stripe image series, (x cji ,y cji ,z cji ) is the coordinate value of the i-th (i=1, 2, 3, 4)th target feature point on the j-th (j=1, 2...N)th target corresponding to the coordinate corresponding to the feature point of the light stripe image, x cji is the horizontal coordinate of the target feature point, y cji is the ordinate of the target feature point, z cji is the remote coordinate of the target feature point, and H0 is the conversion matrix between the camera coordinate system and the pixel coordinate system corresponding to the optical vision sensor;

[0069] In this embodiment, based on the characteristic that the target feature point corresponding to the feature point of the light stripe image is located on the light plane, the light plane equation is constructed; since the target feature point is located both on the target and on the light plane, the (x cji ,y cji ,z cji ) satisfies the following formula 2:

[0070] ax cji +by cji +cz cji +d=0;(Formula 2)

[0071] Wherein, a, b, c, d are parameters of the light plane;

[0072] In this embodiment, based on the characteristic that the distance between the light streak image feature point on the second plane 2 and the target line segment is consistent with the height difference, the constraint equation of the light plane parameter is constructed; the target line segment is the line between the first light streak image feature point on the first plane and the second light streak image feature point on the first plane.

[0073] In this embodiment, compared with the height difference between the first plane and the second plane of the target, the target in this application is very thin. When the line structured light stripes are all located in the middle of the plane of the target, it can be approximately considered that the light plane and the side plane of the target are parallel. At this time, there is:

[0074] d j1 =|M j ×M j1 | / |M j |=h,d j2 =|M j ×M j2 | / |M j |=h; (Formula 3)

[0075] Among them, M j Represents the coordinate point (x cj1 ,y cj1 ,z cj1 ) to the coordinate point (x cj4 ,y cj4 ,z cj4 ), that is, M j =(x cj4 -x cj1 ,y cj4 -y cj1 ,z cj4 -z cj1 );M j1 Represents the coordinate point (x cj1 ,y cj1 ,z cj1 ) to the coordinate point (x cj2 ,y cj2 ,z cj2 ), that is, M j1 =(x cj2 -x cj1 ,y cj2 -y cj1 ,z cj2 -z cj1 );M j2 Represents the coordinate point (x cj1 ,y cj1 ,z cj1 ) to the coordinate point (x cj3 ,y cj3 ,z cj3), that is, M j2 =(x cj3 -x cj1 ,y cj3 -y cj1 ,z cj3 -z cj1 );d j1 and d j2 Respectively represent the coordinate points (x cj2 ,y cj2 ,z cj2 ) and coordinate point (x cj3 ,y cj3 ,z cj3 ) to vector M j , h is the height difference between the first plane and the second plane of the target.

[0076] It can be understood that the thickness of the target used in the present application is particularly pointed out to be very thin, so when the line structured light is projected to the middle position of the plane of the target, it can be approximately considered that the light plane and the side plane of the target are parallel. In this way, a height difference with a very small error can be obtained. In the subsequent process of solving the light plane parameters according to the corresponding coordinate values ​​of the determined feature points of the light plane image, the constraint conditions can be constructed by using the above-mentioned height difference to make the solution of the constructed set of equations simpler and easier to converge.

[0077] Then, by combining the above formula 1, formula 2 and formula 3, we can get N following equations, as shown in formula 4:

[0078]

[0079] According to the equations, the following objective function is established:

[0080]

[0081] Among them, the optimal solution corresponding to the objective function g(a, b, c, d) is the light plane parameters a, b, c, d that can minimize the function value of the objective function. That is, in this embodiment, after substituting the coordinate values ​​corresponding to the light image feature points from the first light streak image to the Nth light streak image into the above objective function, the light plane parameters a, b, c, d that can minimize the function value of the objective function are the optimal solution of the above objective function.

[0082] Step S26: constructing an objective function about the light plane calibration parameters based on the mapping equation, the light plane equation and the constraint equation; solving the objective function using a nonlinear optimization method to obtain an optimal solution for the light plane calibration parameters under a maximum likelihood criterion.

[0083] In this embodiment, the equation group can use a nonlinear optimization method, such as Newton's method, to solve the optimal solution of the light plane parameters under the maximum likelihood criterion, that is, to complete the calibration of the light plane. Further, the equation group is constructed using the above constraint equations, and the constraint condition of the height difference between the first plane and the second plane is used, so that the solution of the constructed equation group is simpler, the result is easy to converge, and the calculation process and amount are reduced, thereby improving the calculation speed and the accuracy of the result.

[0084] It can be seen that in the embodiment of the present application, the light plane that is approximately parallel to the target side plane can be obtained according to the inherent characteristics of the preset target, and the group of equations is combined by the mapping equation, the light plane equation and the constraint equation to establish the objective function, solve the light plane parameters, and use the nonlinear optimization method to solve the optimal solution of the light plane parameters, so that the calculation process is relatively clear and simple, and easy to understand.

[0085] Reference Figure 5 As shown, an embodiment of the present invention discloses a line structure cursor positioning device, the device comprising:

[0086] A parameter calibration module 11 is used to calibrate the internal parameters of the tilt camera and establish a conversion matrix between the camera coordinate system and the pixel coordinate system;

[0087] An image acquisition module 12 is used to project a line structured light onto a first plane and a second plane with a height difference on the same side of a preset target through a line structured light laser, and control the light stripes formed on the first plane and the second plane to be located at the middle position of the plane, and then perform image acquisition to obtain a corresponding light stripe image;

[0088] A coordinate determination module 13, used to determine the coordinate value of the light streak image feature point in the light streak image and determine the coordinate value of the target feature point corresponding to the light streak image feature point from the target based on the internal parameters in the tilt camera;

[0089] The equation solving module 14 is used to construct a constraint equation of the light plane parameters based on the height difference between the first plane and the second plane, and solve the light plane parameters based on the coordinate values ​​of the feature points of the light streak image, the coordinate values ​​of the target feature points and the constraint equation to obtain corresponding light plane parameters.

[0090] It can be seen that the present application first calibrates the internal parameters in the tilt camera and establishes a conversion matrix between the camera coordinate system and the pixel coordinate system; projects a line structured light onto a first plane and a second plane with a height difference on the same side of a preset target through a line structured light laser, and controls the light stripes formed on the first plane and the second plane to be located in the middle of the plane, and then performs image acquisition to obtain a corresponding light stripe image; determines the coordinate values ​​of the feature points of the light stripe image in the light stripe image, and determines the coordinate values ​​of the target feature points corresponding to the feature points of the light stripe image from the target based on the internal parameters in the tilt camera; constructs a constraint equation for the light plane parameters based on the height difference between the first plane and the second plane, and solves the light plane parameters based on the coordinate values ​​of the feature points of the light stripe image, the coordinate values ​​of the target feature points and the constraint equation to obtain the corresponding light plane parameters. It can be seen that the target in the present application is simple to manufacture, and the operation process of controlling the projection of the line structured light onto the target plane is simple. Then, by extracting the center of the light streak in the light streak image, the feature points of the light streak image and the corresponding target feature points are determined, so that the selection of the feature points of the light streak image is easier and more representative. In addition, according to the preset target, it can be ensured that the light plane and the target side plane maintain an approximately parallel relationship, and the height difference between the same side planes with a smaller error can be obtained. Moreover, according to the coordinate values ​​of the feature points of the light streak image, the corresponding target feature points and the height difference, a simple constraint condition can be constructed, so that the solution of the light plane equation is simpler and more accurate.

[0091] Furthermore, the present application also discloses an electronic device. Figure 6 This is a structural diagram of an electronic device 20 according to an exemplary embodiment. The content in the diagram cannot be considered as any limitation on the scope of use of the present application.

[0092] Figure 6 The present invention provides a schematic diagram of the structure of an electronic device 20 provided in an embodiment of the present application. The electronic device 20 may include: at least one processor 21, at least one memory 22, a power supply 23, a communication interface 24, an input / output interface 25, and a communication bus 26. The memory 22 is used to store a computer program, which is loaded and executed by the processor 21 to implement the relevant steps in the line structure light positioning method disclosed in any of the above embodiments. In addition, the electronic device 20 in this embodiment may be specifically an electronic computer.

[0093] In this embodiment, the power supply 23 is used to provide working voltage for each hardware device on the electronic device 20; the communication interface 24 can create a data transmission channel between the electronic device 20 and the external device, and the communication protocol it follows is any communication protocol that can be applied to the technical solution of the present application, and is not specifically limited here; the input and output interface 25 is used to obtain external input data or output data to the outside world, and its specific interface type can be selected according to specific application needs and is not specifically limited here.

[0094] In addition, the memory 22, as a carrier for storing resources, can be a read-only memory, a random access memory, a disk or an optical disk, etc. The resources stored thereon can include an operating system 221, a computer program 222, etc., and the storage method can be temporary storage or permanent storage.

[0095] The operating system 221 is used to manage and control the hardware devices and computer programs 222 on the electronic device 20, and can be Windows Server, Netware, Unix, Linux, etc. In addition to including a computer program that can be used to complete the line structure cursor positioning method performed by the electronic device 20 disclosed in any of the aforementioned embodiments, the computer program 222 can further include a computer program that can be used to complete other specific tasks.

[0096] Furthermore, the present application also discloses a computer-readable storage medium for storing a computer program; wherein the computer program, when executed by a processor, implements the aforementioned line structure light determination method. The specific steps of the method can refer to the corresponding contents disclosed in the aforementioned embodiments, and will not be repeated here.

[0097] In this specification, each embodiment is described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the embodiments can be referred to each other. For the device disclosed in the embodiment, since it corresponds to the method disclosed in the embodiment, the description is relatively simple, and the relevant parts can be referred to the method part.

[0098] Professionals may further appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented with electronic hardware, computer software or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to the function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application. The steps of the method or algorithm described in conjunction with the embodiments disclosed herein can be directly implemented with a software module executed by a hardware or processor, or a combination of the two. The software module can be placed in a random access memory (RAM), a memory, a read-only memory (ROM), an electrically programmable ROM, an electrically erasable programmable ROM, a register, a hard disk, a removable disk, a CD-ROM, or any other form of storage medium known in the technical field.

[0099] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the presence of other identical elements in the process, method, article or device including the elements.

[0100] The above is a detailed introduction to a line structure light positioning method, device, equipment and storage medium provided by the present invention. Specific examples are used in this article to illustrate the principles and implementation methods of the present invention. The description of the above embodiments is only used to help understand the method of the present invention and its core idea; at the same time, for general technical personnel in this field, according to the idea of ​​the present invention, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present invention.

Claims

1. A line structure light positioning method, characterized in that: include: Calibrate the internal parameters of the tilt camera and establish the transformation matrix between the camera coordinate system and the pixel coordinate system; Projecting line structured light onto a first plane and a second plane with a height difference on the same side of a preset target by a line structured light laser, and controlling the light stripes formed on the first plane and the second plane to be located at the middle position of the plane, and then performing image acquisition to obtain a corresponding light stripe image; Determining coordinate values ​​of light streak image feature points in the light streak image and determining coordinate values ​​of target feature points corresponding to the light streak image feature points from a target based on the internal parameters in the tilt camera; constructing a constraint equation of a light plane parameter based on a height difference between the first plane and the second plane, and solving the light plane parameter based on coordinate values ​​of feature points of the light streak image, coordinate values ​​of feature points of the target and the constraint equation to obtain corresponding light plane parameters; The method of projecting a line structured light onto a first plane and a second plane with a height difference on the same side of a preset target by a line structured light laser, and controlling the light stripes formed on the first plane and the second plane to be located in the middle of the plane, comprises: Projecting line structured light onto a first plane and a second plane with a height difference on the same side of a preset concave target and / or a preset convex target by a line structured light laser, and controlling the light stripes formed on the first plane and the second plane to be located in the middle of the plane; Wherein, determining the coordinate values ​​of the light streak image feature points in the light streak image includes: Extracting the sub-pixel light stripe center of the light stripe image located on the first plane, and averaging the image coordinate values ​​on the sub-pixel light stripe center of the first plane to obtain the coordinate value corresponding to the feature point of the light stripe image on the first plane; wherein the preset target includes two first planes and one second plane; The sub-pixel light stripe center of the light stripe image located in the second plane is extracted, the sub-pixel light stripe center located in the second plane is divided into two segments of light stripe centers of equal length, and the image coordinate values ​​of each segment of the light stripe center are averaged to obtain coordinate values ​​corresponding to the light stripe image feature points on each segment of the light stripe center.

2. The line structure optical positioning method according to claim 1, characterized in that: The performing of image acquisition to obtain a corresponding light streak image comprises: The posture of the preset concave target and / or the preset convex target is adjusted, and images of the preset concave target and / or the preset convex target in different postures are captured to obtain light stripe images in corresponding different postures.

3. The line structure optical positioning method according to claim 1, characterized in that: Before determining the coordinate values ​​of the light streak image feature points in the light streak image, the method further includes: Performing image denoising on the light streak image to obtain a denoised light streak image.

4. The line structure optical positioning method according to claim 1, characterized in that: Before solving the light plane parameters, the method further includes: Determine a mapping equation between the image feature points and the corresponding target feature points based on the calibrated internal parameters in the tilt camera and the tilt camera imaging model; Based on the characteristic that the target feature points corresponding to the feature points of the light fringe image are located in the light plane, constructing a light plane equation; Based on the characteristic that the distance between the light streak image feature point on the second plane and the target line segment is consistent with the height difference, a constraint equation for the light plane parameter is constructed; the target line segment is a line connecting a first light streak image feature point on the first plane and a second light streak image feature point on the first plane.

5. The line structure optical positioning method according to claim 4, characterized in that: Solving the light plane parameters to obtain corresponding light plane parameters includes: constructing an objective function about the light plane parameters based on the mapping equation, the light plane equation, and the constraint equation; The objective function is solved using a nonlinear optimization method to obtain an optimal solution for the light plane parameters under a maximum likelihood criterion.

6. A linear structured light positioning device, characterized in that: include: The parameter calibration module is used to calibrate the internal parameters of the tilt camera and establish a transformation matrix between the camera coordinate system and the pixel coordinate system; An image acquisition module, used to project a line structured light onto a first plane and a second plane with a height difference on the same side of a preset target through a line structured light laser, and control the light stripes formed on the first plane and the second plane to be located in the middle of the plane, and then perform image acquisition to obtain a corresponding light stripe image; a coordinate determination module, used to determine the coordinate value of the light streak image feature point in the light streak image and determine the coordinate value of the target feature point corresponding to the light streak image feature point from the target based on the internal parameters in the tilt camera; an equation solving module, configured to construct a constraint equation of a light plane parameter based on a height difference between the first plane and the second plane, and solve the light plane parameter based on coordinate values ​​of feature points of the light streak image, coordinate values ​​of feature points of the target and the constraint equation to obtain corresponding light plane parameters; The image acquisition module is specifically used to: project line structured light onto a first plane and a second plane with a height difference on the same side of a preset concave target and / or a preset convex target through a line structured light laser, and control the light stripes formed on the first plane and the second plane to be located in the middle of the plane; Wherein, the coordinate determination module is specifically used to: extract the sub-pixel light stripe center of the light stripe image located in the first plane, and average the image coordinate values ​​on the sub-pixel light stripe center of the first plane to obtain the coordinate value corresponding to the feature point of the light stripe image on the first plane; wherein the preset target includes two first planes and one second plane; extract the sub-pixel light stripe center of the light stripe image located in the second plane, divide the sub-pixel light stripe center located in the second plane into two segments of light stripe centers of equal length, and average the image coordinate values ​​of each segment of the light stripe center to obtain the coordinate values ​​corresponding to the feature points of the light stripe image on the center of each segment of the light stripe.

7. An electronic device, characterized in that: include: Memory for storing computer programs; A processor, configured to execute the computer program to implement the steps of the line structure optical positioning method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: Used to store computer programs; wherein, when the computer program is executed by a processor, the steps of the line structure light positioning method as described in any one of claims 1 to 5 are implemented.

Images