Calibration method, device and electronic equipment of visual camera

Abstract

The application provides a kind of visual camera calibration method, device and electronic equipment, comprising: obtaining the image of calibration ruler photographed by at least two visual cameras;The position of the identification point in the image of calibration ruler is solved, and the position of the identification point in the image of each calibration ruler is obtained;Based on the position of the identification point in the image of each calibration ruler, the camera coordinate system is established, and the conversion relationship between the visual cameras is determined according to the image of calibration ruler, and then the calibration of visual camera is completed.The calibration of visual camera in the calibration method of visual camera of the present application is realized by the position of the identification point in the image of calibration ruler, and the image of calibration ruler is photographed when calibration ruler moves according to the preset moving direction and the preset moving distance.The calibration of visual camera based on the position of the identification point is more accurate, and the calibration process is automatically completed, and only one calibration personnel is needed to move the calibration ruler.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of camera calibration, and in particular to a visual camera calibration method, device and electronic equipment. BACKGROUND

[0002] The MPS / M series multi-camera real-time industrial photogrammetry system takes two or more high-precision real-time photogrammetry visual cameras as main sensors, takes software as the core, collects two or more measurement images of a workpiece to be measured in real time through a visual positioning system, obtains three-dimensional coordinates of feature points of the workpiece through double-image forward intersection, and measures the position, posture, deformation and motion parameters of the workpiece to be measured in real time.

[0003] Before positioning the workpiece to be measured, the visual camera needs to be calibrated, but the traditional visual camera calibration method has low precision and requires many calibration personnel. SUMMARY

[0004] Therefore, the present application aims to provide a visual camera calibration method, device and electronic equipment to alleviate the technical problems of low precision and the need for many calibration personnel in the existing visual camera calibration method.

[0005] In a first aspect, the present application provides a visual camera calibration method, which applies a visual positioning system connected with the visual camera, and the method comprises the following steps:

[0006] Obtaining images of a calibration ruler photographed by at least two visual cameras, wherein the calibration ruler is located at the end of a moving rod below the visual camera, the calibration ruler moves according to a preset moving direction and a preset moving distance, an image of the calibration ruler is obtained when the calibration ruler moves once, and the calibration ruler is provided with a mark point;

[0007] Solving the position of the mark point in the image of the calibration ruler to obtain the position of the mark point in each image of the calibration ruler;

[0008] Establishing a camera coordinate system based on the position of the mark point in each image of the calibration ruler, and determining the conversion relationship between the visual cameras according to the image of the calibration ruler, thereby completing the calibration of the visual cameras.

[0009] Further, the solving of the position of the mark point in the image of the calibration ruler comprises:

[0010] Coarsely positioning the mark point in the image of the calibration ruler by using an edge detection operator to obtain a pixel-level edge point of the mark point;

[0011] performing sub-pixel edge detection on the pixel-level edge points of the identification point to obtain edge points of the identification point at sub-pixel accuracy;

[0012] performing ellipse least square fitting on the edge points of the identification point at sub-pixel accuracy to obtain the position of the identification point in the image of the calibration ruler.

[0013] Further, a camera coordinate system is established based on the positions of the identification point in the images of the calibration rulers, comprising:

[0014] obtaining a preset coordinate system origin of the calibration ruler;

[0015] establishing a coordinate system of movement of the calibration ruler according to the coordinate system origin, the preset movement direction and the preset movement distance;

[0016] establishing a camera coordinate system according to the positions of the identification point in the images of the calibration rulers and the coordinate system of movement of the calibration ruler.

[0017] Further, the conversion relationship between the visual cameras is determined according to the images of the calibration rulers, comprising:

[0018] performing position and posture solving on the images of the calibration rulers by using a camera self-calibration based on bundle adjustment to obtain internal and external parameters of at least two of the visual cameras;

[0019] determining the conversion relationship between the visual cameras according to the internal and external parameters of at least two of the visual cameras to realize relative orientation between different visual cameras.

[0020] Further, after the calibration of the visual cameras is completed, the method further comprises:

[0021] obtaining a to-be-measured object image captured by the visual cameras on a to-be-measured object, wherein the to-be-measured object is provided with an identification point;

[0022] identifying the position of the identification point in the to-be-measured object image to obtain the position of the identification point in the to-be-measured object image;

[0023] compensating the position of the identification point in the to-be-measured object image.

[0024] Further, the identification point comprises a glass bead and a directional reflective material located below the glass bead.

[0025] Further, the preset movement direction comprises a serpentine direction or a diagonal direction.

[0026] In a second aspect, the embodiments of the present application further provide a device for calibrating a vision camera, which is applied to a vision positioning system connected with the vision camera, and the device comprises:

[0027] an acquisition unit configured to acquire images of a calibration ruler photographed by at least two vision cameras, wherein the calibration ruler is located at the end of a moving rod below the vision cameras, the calibration ruler moves according to a preset moving direction and a preset moving distance, an image of the calibration ruler is obtained when the calibration ruler moves once, and the calibration ruler is provided with a mark point;

[0028] a calculation unit configured to calculate the position of the mark point in the images of the calibration ruler to obtain the position of the mark point in each image of the calibration ruler;

[0029] an establishment and conversion unit configured to establish a camera coordinate system based on the position of the mark point in the images of the calibration ruler, and determine the conversion relationship between the vision cameras according to the images of the calibration ruler, so as to complete the calibration of the vision cameras.

[0030] In a third aspect, the embodiments of the present application further provide an electronic device, which comprises a memory, a processor, and a computer program stored in the memory and capable of running on the processor, and the processor implements the steps of the method of any one of the first aspect when executing the computer program.

[0031] In a fourth aspect, the embodiments of the present application further provide a computer readable storage medium, which stores machine executable instructions, and the machine executable instructions make the processor run the method of any one of the first aspect when the machine executable instructions are called and run by the processor.

[0032] In the embodiment of the present application, a visual camera calibration method is provided, which is applied to a visual positioning system connected with the visual camera, and the method comprises the following steps: acquiring images of a calibration ruler photographed by at least two visual cameras, wherein the calibration ruler is located at the end of a moving rod below the visual camera, the calibration ruler moves according to a preset moving direction and a preset moving distance, an image of the calibration ruler is obtained when the calibration ruler moves once, and the calibration ruler is provided with a mark point; calculating the position of the mark point in the image of the calibration ruler to obtain the position of the mark point in each image of the calibration ruler; establishing a camera coordinate system based on the position of the mark point in each image of the calibration ruler, and determining the conversion relationship between the visual cameras according to the images of the calibration ruler, thereby completing the calibration of the visual cameras. As can be seen from the above description, the calibration of the visual cameras in the visual camera calibration method of the present application is realized by the position of the mark point in the image of the calibration ruler, and the image of the calibration ruler is photographed when the calibration ruler moves according to the preset moving direction and the preset moving distance. The calibration of the visual cameras based on the position of the mark point is more accurate, and the calibration process is automatically completed, so that only one calibration personnel is needed to move the calibration ruler, thereby solving the technical problems of the prior art that the calibration accuracy of the visual cameras is low and many calibration personnel are needed. BRIEF DESCRIPTION OF DRAWINGS

[0033] In order to more clearly illustrate the specific embodiments of the present application or the technical solutions in the prior art, the following will briefly introduce the drawings needed to be used in the specific embodiments or the prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and those skilled in the art can also obtain other drawings according to these drawings without any creative labor.

[0034] Figure 1 A flow chart of a visual camera calibration method provided in the embodiment of the present application is shown in the figure.

[0035] Figure 2 A schematic diagram of the positional relationship between the visual camera and the calibration ruler provided in the embodiment of the present application is shown in the figure.

[0036] Figure 3 A method flow chart for calculating the position of the mark point in the image of the calibration ruler provided in the embodiment of the present application is shown in the figure.

[0037] Figure 4 A schematic diagram of the coordinate system of the movement of the calibration ruler provided in the embodiment of the present application is shown in the figure.

[0038] Figure 5 A schematic diagram of a stereo image pair provided in the embodiment of the present application is shown in the figure.

[0039] Figure 6A schematic view of a visual camera calibration device provided by an embodiment of the present application is shown in the figure;

[0040] Figure 7 A schematic view of an electronic device provided by an embodiment of the present application is shown in the figure. DETAILED DESCRIPTION

[0041] The technical solutions of the present application will be described in detail below with reference to the embodiments. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments of the present application, all other embodiments obtained by those skilled in the art without creative work fall within the scope of the present application.

[0042] At present, the traditional visual camera calibration method has low precision and requires many calibration personnel.

[0043] Therefore, in the visual camera calibration method of the present application, the visual camera is calibrated based on the positions of the marking points in the image of the calibration ruler, and the image of the calibration ruler is taken when the calibration ruler moves according to the preset moving direction and the preset moving distance. The calibration of the visual camera based on the positions of the marking points is more accurate, and the calibration process is automatically completed. Only one calibration personnel is needed to move the calibration ruler, thereby solving the technical problems of the existing visual camera calibration method, such as low precision and the need for many calibration personnel.

[0044] To facilitate the understanding of the present embodiment, first, a visual camera calibration method disclosed by the present embodiment will be described in detail.

[0045] Embodiment one:

[0046] According to the present embodiment, an embodiment of a visual camera calibration method is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order from that shown here.

[0047] Figure 1 A flowchart of a visual camera calibration method according to an embodiment of the present application is shown in FIG. Figure 1 The method comprises the following steps:

[0048] In step S102, at least two visual cameras are used to capture images of a calibration ruler, wherein the calibration ruler is located at the end of a moving rod below the visual camera, the calibration ruler moves according to a preset moving direction and a preset moving distance, an image of the calibration ruler is captured each time the calibration ruler moves, and marking points are provided on the calibration ruler;

[0049] In the embodiment of the present application, the calibration method of the visual camera can be applied to a visual positioning system connected with the visual camera, Figure 2 The positional relationship between the visual camera and the calibration ruler is shown in FIG. 2. Specifically, the calibration ruler is provided with a mark point, and the mark point includes a glass bead and a directional reflective material below the glass bead. The material used for the mark point is a glass bead with a diameter of only tens of microns. Since the refractive index of glass and air is different, light is focused on a point behind the glass bead. If a reflective surface (i.e., the directional reflective material) is attached at the focal point, the incident light beam can be reflected along the incident direction. Due to the unique reflective characteristics of the directional reflective material, the artificial mark (i.e., the mark point) made of the directional reflective material can obtain a "quasi-binary image", which is beneficial to image processing and improves the measurement accuracy.

[0050] Before the images of the calibration ruler taken by the at least two visual cameras are acquired, the reference space range of the at least two visual cameras is determined. The reference space range is the visual range of the visual cameras to be calibrated.

[0051] Specifically, a zero position, an X direction and a Y direction are determined, and a moving direction of the calibration direction is formulated. The above two directions can be a serpentine direction or a diagonal direction. The above two directions are because the calibration points are not allowed to overlap and be too close. It is verified that the serpentine and diagonal ways are better. The calibration ruler is moved 200 mm (i.e., a preset moving distance, which can be adjusted) along the X direction of the gantry (i.e., the chassis of the transfer car) because the distance between two points cannot be too close. Therefore, 200 mm is an optimal setting. The visual positioning system records the starting and ending positions, and the X direction can be calibrated. The spreader is moved 200 mm along the Y direction of the gantry, and the visual positioning system records the starting and ending positions, and the Y direction can be calibrated.

[0052] The system adopts the epipolar constraint relationship of multiple images to automatically match corresponding image points (such as mark points), and experiments prove that the method can achieve 100% matching of mark points. Matching is to seek corresponding image points of object points in different images. Stereoscopic matching is the most important and difficult problem in stereovision and photogrammetry. Due to the influence of many factors such as light conditions, geometric shapes of objects, noise interference, camera characteristics and distortion, when a three-dimensional scene is projected as a two-dimensional image, the images of the same object under different viewpoints will be quite different. The commonly used algorithms in the photogrammetry field include least square matching, feature-based matching, and relaxed image matching based on cross method. The matching based on epipolar constraint condition has been applied in binocular stereovision measurement and traditional photogrammetry for a long time, but the matching ambiguity is too large when the epipolar lines of two images are matched, which has always affected its application. At present, the matching method based on multiple epipolar constraints is gradually valued by the computer vision and close-range photogrammetry fields. According to the characteristics of close-range photogrammetry, the epipolar constraint of multiple images is used to realize the matching of corresponding image points, and good results are achieved. If the number of images is small, the epipolar constraint condition will be weak, which will increase the probability of mismatching. On the other hand, if the orientation parameters are not accurate, the corresponding epipolar lines obtained will also be inaccurate, which will obviously affect the accuracy of matching. The window size for searching the candidate corresponding image points along the epipolar line (mainly depends on the value of e) is another factor affecting the success of matching. In order to reduce the probability of mismatching, e should be as small as possible.

[0053] The encoding and decoding principles and schemes of mark points are studied in the system, and a specific concentric ring type mark point encoding and decoding is realized. Through the research on mark point materials, a certain amount of mark points are produced, and the correctness and matching accuracy have been proved in the use of other products and projects.

[0054] Specifically, in industrial photogrammetry, the same measured target object is photographed by one or two photographic cameras at different positions (camera stations), and the two images of the measured target object obtained at different angles are called stereoscopic image pairs. This process is called binocular stereovision in the field of computer vision, and binocular vision geometry is also called epipolar geometry in photogrammetry, which mainly refers to the internal geometric projection relationship between two images. It is uniquely determined by the internal orientation elements of the image and the relative attitude relationship of the two images, and is independent of the scene structure.

[0055] The orientation of the visual positioning system refers to the process of calibrating the relative position and attitude of a group of visual cameras.

[0056] In step S104, the positions of the mark points in the images of the calibration scales are calculated to obtain the positions of the mark points in the images of the calibration scales.

[0057] Specifically, digital close-range photogrammetry is an image-based measurement technique, which generally obtains the two-dimensional image coordinates of the feature targets (such as mark points) (edge, corner point, etc. natural features, crosshairs, laser projection points or light strips, circular marks, etc.) in the image of the measured object by processing the image of the feature targets, that is, positioning the image of the feature targets, and then performing measurement. If the positioning accuracy of the feature targets on the image is improved by the software processing method, the measurement accuracy is directly improved. For example, when the target positioning accuracy is improved from 0.1 pixel to 0.02 pixel, the camera resolution of the measurement system is increased by five times.

[0058] In step S106, a camera coordinate system is established based on the positions of the mark points in the images of the calibration scales, and the conversion relationship between the vision cameras is determined according to the images of the calibration scales, and then the calibration of the vision cameras is completed.

[0059] The process will be described in detail below, and will not be described here.

[0060] In the embodiment of the present application, a vision camera calibration method is provided, which applies a vision positioning system, the vision positioning system is connected with the vision camera, and the method comprises the following steps: obtaining images of calibration scales photographed by at least two vision cameras, wherein the calibration scales are located at the end of a moving rod below the vision cameras, the calibration scales move according to a preset moving direction and a preset moving distance, an image of the calibration scales is obtained when the calibration scales move once, and mark points are arranged on the calibration scales; the positions of the mark points in the images of the calibration scales are calculated to obtain the positions of the mark points in the images of the calibration scales; a camera coordinate system is established based on the positions of the mark points in the images of the calibration scales, and the conversion relationship between the vision cameras is determined according to the images of the calibration scales, and then the calibration of the vision cameras is completed. As known from the above description, the calibration of the vision cameras in the vision camera calibration method of the present application is realized by the positions of the mark points in the images of the calibration scales, and the images of the calibration scales are photographed when the calibration scales move according to the preset moving direction and the preset moving distance. The calibration of the vision cameras based on the positions of the mark points is more accurate, and the calibration process is automatically completed, so that only one calibration personnel is needed to move the calibration scales, thereby solving the technical problems of the existing vision camera calibration methods, such as low precision and the need for many calibration personnel.

[0061] The above describes the vision camera calibration method of the present application briefly, and the specific contents involved therein will be described in detail below.

[0062] In an optional embodiment of the present application, the reference Figure 3 The positions of the mark points in the images of the calibration scales are calculated, which specifically comprises the following steps:

[0063] Step S301: Use edge detection operands to coarsely locate the marker points in the calibration ruler image to obtain the pixel-level edge points of the marker points;

[0064] Step S302: Perform sub-pixel edge detection on the pixel-level edge points of the marker points to obtain the sub-pixel precision edge points of the marker points;

[0065] Step S303: Perform elliptic least squares fitting on the edge points of the marker points with sub-pixel precision to obtain the position of the marker points in the image of the calibration ruler.

[0066] Specifically, the system uses circular RRT signs made of directional reflective material. After being imaged through a lens (the lens of a vision camera), the circular sign becomes an ellipse. To achieve sub-pixel-level precision positioning of the ellipse's center, the system first uses edge detection operands to coarsely locate the ellipse's edges with integer-pixel precision, obtaining pixel-level edge points. Then, sub-pixel edge detection is performed on these pixel-level edge points to obtain sub-pixel-precision edge points. Finally, the extracted sub-pixel-precision edge points are fitted with ellipse least-squares to determine the precise location of the sign's center. The difficulty of this technology lies in the sub-pixel edge detection of the reflective sign (RRT sign) and the sub-pixel-precision center positioning. This system has a considerable theoretical foundation in this area, and the software development has been implemented.

[0067] In an optional embodiment of the present invention, a camera coordinate system is established based on the position of the marker points in the images of each calibration scale, specifically including the following steps:

[0068] (1) Obtain the preset coordinate system origin of the calibration ruler;

[0069] (2) Establish the coordinate system for the movement of the calibration ruler based on the origin of the coordinate system, the preset movement direction, and the preset movement distance;

[0070] (3) Establish the camera coordinate system based on the position of the marker points in the images of each calibration ruler and the coordinate system of the calibration ruler movement.

[0071] like Figure 4 As shown ( Figure 4 The preset movement distance is shown as 180mm (for example), which is the coordinate system for the movement of the calibration ruler. The preset coordinate system origin is obtained by taking the negative extreme value based on the position of the electric cylinder after the vehicle (loading vehicle) is retracted and the camera's spatial range.

[0072] In an optional embodiment of the present invention, determining the conversion relationship between visual cameras based on the image of the calibration ruler specifically includes the following steps:

[0073] (1) Camera self-calibration using bundle adjustment to solve the position and posture of the image of the calibration ruler, and to obtain the internal and external parameters of at least two vision cameras;

[0074] (2) Determine the conversion relationship between the vision cameras according to the internal and external parameters of the at least two vision cameras, and realize the relative orientation between different vision cameras.

[0075] Specifically, during calibration, the bundle adjustment of the camera is improved as follows:

[0076] Self-calibration refers to the simultaneous adjustment of internal and external parameters, that is, overall adjustment with additional parameters. Because this method simultaneously solves internal and external parameters, it does not necessarily require control points, and therefore belongs to indirect compensation of systematic errors. In the early stage of the development of the system, the self-calibration technology of digital cameras was studied, and it was proposed to improve the geometric conditions of adjustment calculation and to assign weights to internal parameters to overcome parameter overparameterization. The feasibility of the above method was verified through experiments, and it was concluded that the calibration results and the additional parameters are not very sensitive to the weight values assigned according to the signal-to-noise ratio. The self-calibration method with different internal parameters of each image has higher precision than the self-calibration method with the same internal parameters, and is more in line with the actual situation (the image point residual is reduced by 1 times). The traditional experimental field method of calibration uses a large number of control points for spatial resection to directly compensate for the system error of the image points. Its advantages are that it can accurately correct the system error, and the calculation amount is small, but its disadvantage is that it requires a large number of known control points, and it is not easy to implement in the work site. The self-calibration method simultaneously solves internal and external parameters, and does not necessarily require many control points. Because the number of control points is very small or even zero in the self-calibration process, the approximate linear relationship between the parameters causes the column vectors in the coefficient matrix of the normal equation to have an approximate linear relationship, which is called multicollinearity. Multicollinearity causes the coefficient matrix of the normal equation to be ill-conditioned, thereby affecting the stability and accuracy of the solution. In order to improve the accuracy of the calibration calculation results, two measures can be taken: one is to break the multicollinearity, that is, to use the hypothesis testing method in mathematical statistics to eliminate those additional parameters that cannot be separated from each other, which requires various statistical tests and analyses of the results, including significance test, measurability test, sensitivity test, and correlation coefficient test. The disadvantages of this method are that the calculation amount is large, and that after eliminating some additional parameters, the compensation for the system error will not be complete. The other processing method is to improve the properties of the normal equation, such as improving the geometric conditions of adjustment, using parameter weighted adjustment or biased estimation method, which can ensure the completeness of the compensation for the system error and overcome the multicollinearity caused by overparameterization.

[0077] The mathematical model of the system error is corrected as follows:

[0078] According to the principle of perspective projection imaging, three points of object point, lens center and image point are collinear. In fact, due to various interference factors, the main factors interfering with the imaging of the digital camera-based photography are radial distortion and decentering distortion of the camera lens, non-flat distortion of the image plane, and in-plane proportion and orthogonal distortion of the image plane. However, if the internal orientation elements (x0, y0, f) are inaccurate, the collinearity equation will also be mathematically interfered. The image point coordinate error caused by these internal parameters is systematic, so it is called the systematic error of the image point, and the image point has a deviation (△x, △y) relative to its theoretical position on the focal plane.

[0079] The systematic error of any image point is the sum of the radial distortion, decentering distortion, in-plane distortion of the image plane, and the distortion caused by inaccurate internal orientation elements. The image point coordinate error caused by these internal parameters is called the systematic error of the image point, which is written as follows:

[0080]

[0081] Considering the influence of the systematic error of the image point, the following equation (collinearity condition equation model with internal parameters) can be formed:

[0082]

[0083] Putting the internal and external parameters together and performing overall adjustment calculation, that is, overall adjustment with additional parameters can effectively ensure the completeness of the compensation of the systematic error and complete the mathematical model correction of the system.

[0084] In the calibration process, bidirectional analytic geometry is applied, which is introduced as follows:

[0085] One or two cameras are used to take pictures of the same object from different positions (camera stations) to obtain two different angle images of the measured target, which is called a stereo image pair (model). In the field of computer vision, this process is called binocular stereo vision. Figure 5 A stereo image pair is shown, where the imaging of object point P on image 1 and image 2 is p1 and p2, respectively. p1 and p2 are called homologous image points (also called homonymous image points). Object point P, projection centers S1 and S2 are coplanar, and this plane is called the epipolar plane (also called the polar plane) of object point P. The intersection line (l1 and l2) of the epipolar plane and the image plane is called the epipolar line (also called the polar line). Obviously, corresponding image points p1 and p2 must be on the corresponding epipolar lines l1 and l2.

[0086] Relative orientation and absolute orientation

[0087] Six exterior orientation elements are needed to determine the orientation of one image. Therefore, twelve exterior orientation elements are needed to determine the orientation of two images (i.e. images) of a stereo pair, i.e.

[0088] Image 1: Xs1, Ys1, Zs1, ω1, κ1;

[0089] Image 2: Xs2, Ys2, Zs2, ω2, κ2.

[0090] With the twelve exterior orientation elements, the orientation of the two images in the object coordinate system is determined, and of course the relative orientation between the two images is determined. For the convenience of solving the problem, the relative orientation between the two images is often considered first, and then the absolute orientation of the entire stereo pair in the object space is considered. Therefore, the process of determining the relative orientation between the two images in a stereo pair is called relative orientation, and the parameters used to determine the relative orientation between the two images are called the relative orientation elements of the stereo pair. The process of determining the absolute orientation of the stereo pair in the object coordinate system is called absolute orientation, and the parameters required for absolute orientation are called the absolute orientation elements of the stereo pair.

[0091] Subtract the exterior orientation elements of image 1 from the exterior orientation elements of image 2, we get:

[0092] ΔX s =X s2 -X s1 ,

[0093] ΔY s =Y s2 -Y s1 ,

[0094] ΔZ s =Z s2 -Z s1 ,

[0095] Δω=ω2-ω1,

[0096]

[0097] Δκ=κ2-κ1.

[0098] Wherein, △Xs, △Ys, △Zs are the projections of the photographic base (the connecting line of the projection centers of the two stations) on the three coordinate axes of the object coordinate system, denoted as Bx, By, Bz. If we denote:

[0099]

[0100] tan(T)=B y / B x

[0101] sin(ν) = B z / B

[0102] Then, the three elements Bx, By, Bz can be replaced by the three elements B, T, v.

[0103] It can be seen that the length of the baseline B only affects the scale of the stereo pair, but not the relative orientation. Therefore, the relative orientation elements of the stereo pair only need five, i.e. T, v, Δω, and Δκ. The remaining seven parameters are the absolute orientation elements of the stereo pair, i.e. Xs1, Ys1, Zs1, ω1, κ1 and B. That is, the relative orientation can restore a stereo model similar to the actual object, and the absolute orientation makes the measured model completely consistent with the actual model.

[0104] In the calibration process, the generation and conversion of the coordinate system are carried out, in which the coplanar equation and the multi-station intersection geometry are used.

[0105] The coplanar equation is as follows:

[0106] Now, the image space coordinate system S1-xyz of image 1 is selected as the photogrammetric coordinate system, the coordinates of image point p1 in S1-xyz are (x1, y1, -f), the coordinates of image point p2 in the image space coordinate system S2-x'y'z' of image 2 are (x2, y2, -f); the coordinates of the projection center S2 in S1-xyz are (Bx, By, Bz) (the projection of the photographic baseline on the three coordinate axes of the object coordinate system), the coordinates of image point p2 in the coordinate system S2-xyz (S2-xyz is parallel to the three axes of S1-xyz, and is an auxiliary coordinate system) are (x2', y2', z2'), and the rotation matrix between S2-x'y'z' and S1-xyz (or S2-xyz) is M, because the vectors and are coplanar, and have

[0107]

[0108]

[0109] The above formula is written in the coordinate form, and has

[0110]

[0111] The above formula is another basic equation in conventional photogrammetry, which is the coplanarity condition equation.

[0112] The multi-station intersection geometry is as follows:

[0113] If the target is photographed from multiple camera stations, multiple stereoscopic pairs of the measured object can be obtained, thereby forming multi-view stereo vision. If the object point Pi is intersected by j camera stations (j light rays), there are j collinear equations:

[0114]

[0115] For the solving of the stereoscopic model formed by multiple camera stations, the process can also be divided into two processes of relative orientation and absolute orientation, i.e., determining the relative orientation among multiple images and the absolute orientation of the entire stereoscopic model.

[0116] In an optional embodiment of the present application, after the calibration of the vision camera is completed, the method further comprises:

[0117] (1) obtaining an image of a to-be-measured object photographed by a vision camera, wherein the to-be-measured object is provided with a mark point;

[0118] (2) identifying the position of the mark point in the image of the to-be-measured object to obtain the position of the mark point in the image of the to-be-measured object;

[0119] (3) compensating the position of the mark point in the image of the to-be-measured object.

[0120] Specifically, the to-be-measured object can be the chassis of a repositioning vehicle, and the chassis is provided with six mark points. The six mark points are used because the effect formed is the best, and the precision can reach the desired effect.

[0121] After the calibration is completed, compensation of the calibration is performed. The compensation means that the feedback value after the camera is photographed cannot be 0, which is a value of a camera internal generated space coordinate system, and needs to be compensated in the later period to make the current value 0, thereby becoming the effect of calibration zero position.

[0122] According to the above introduction of the principle and key technology of the system, the main factors affecting the measurement of the system are image extraction and image recognition of the measurement mark in the image, and the system has solved such problems. The influence of different weather on the system is also very small.

[0123] a) Influence of rain, snow and fog weather

[0124] The system measurement mainly completes precise measurement through the recognition of the camera to the measurement mark image. In the case of rain, snow and fog weather, the cleaning of the mark point needs to be handled to ensure that the measurement mark point is not blocked (after the mark point is blocked, the mark point cannot be imaged on the image, and the measurement cannot be completed). There cannot be a large amount of water and snow on the mark. If there is snow, water, debris, etc., the measurement needs to be performed after cleaning.

[0125] According to the principle of measurement, the visibility of fog weather greater than 10 meters does not affect the measurement of the system.

[0126] b) Influence of outdoor stray light

[0127] Since the system uses a man-made circular RRT mark made of precision measurement, the mark can obtain a "quasi-binary image" to facilitate image processing and improve measurement accuracy.

[0128] Since the RRT mark is only illuminated by a flash (little affected by ambient light, which can be ignored), its exposure is independent of ambient light. Therefore, under a certain flash intensity, adjusting the light intake of the camera can obtain images of different brightness backgrounds, even a "quasi-binary image" with a nearly black background and a bright RRT mark.

[0129] In this system, the aperture and shutter speed of the camera are fixed, and the exposure time needs to be adjusted. Ambient light affects the brightness of the image, and in certain cases it may affect the processing speed of the image. The influence of ambient light on the system can be changed by adjusting the exposure time.

[0130] In the present application, the calibration ruler is moved to multiple positions to provide multiple positions of the same image point for relative orientation, and the distance between the two photographic coding points is used as a distance constraint to establish a relative orientation model, so that the relative position and attitude of the two cameras can be obtained. After the visual camera completes the orientation, the origin and direction of the visual camera coordinate system are in the image coordinate system of one of the visual cameras, and need to be converted to the coordinate system of the spreader mobile gantry. This process is to establish the coordinate system of the visual orientation system. Similar to the visual camera orientation process, the calibration ruler is moved 200mm along the X direction of the gantry, and the visual positioning system records the starting and ending positions, which can calibrate the X direction; the calibration ruler is moved 200mm along the Y direction of the gantry, and the visual positioning system records the starting and ending positions, which can calibrate the Y direction. Thus, the coordinate system of the visual positioning system and the coordinate system of the gantry are unified in direction.

[0131] The origin of the coordinate system of the visual positioning system and the origin of the coordinate system of the gantry are unified when the first load carrier is retracted.

[0132] Note: The calibration ruler does not need to be stable during orientation, but it needs to be stable when the visual positioning camera measures during the establishment of the image coordinate system.

[0133] In this scheme, the mathematical model of the error can be corrected and the number of points can be increased, which improves the accuracy and simplifies the personnel, and one person can complete all the operation processes.

[0134] The calibration method of the visual camera of the present application has the following characteristics:

[0135] (1)High precision: the highest measurement accuracy can reach 3pm+3ppm·L, which meets the accuracy requirements of various large-size industrial product shape detection;

[0136] (2) Automation: the system has high automation degree, and the whole measurement process can be completed by one measurement personnel;

[0137] (3) Fast speed: thousands of three-dimensional coordinates can be obtained in one measurement;

[0138] (4) Large range: the measurement range can reach hundreds of meters, and it is suitable for product detection of various sizes;

[0139] (5) Non-contact: the optical non-contact measurement method is adopted, and for the flexible structure of the measured workpiece, the system has incomparable advantages;

[0140] (6) Super-portable: the measurement system can be placed in a travel box, and only one measurement personnel is needed to carry the equipment to any measurement site;

[0141] (7) Strong environmental adaptability: the measurement system can measure in vacuum, toxic, high and low temperature environments; and high-precision measurement can be realized in very small space.

[0142] Example two:

[0143] The embodiment of the application also provides a visual camera calibration device, which is mainly used for executing the visual camera calibration method provided in the embodiment one of the application, and the visual camera calibration device provided in the embodiment of the application is specifically introduced as follows.

[0144] Figure 6 It is a schematic diagram of a visual camera calibration device according to the embodiment of the application, as shown in Figure 6 The visual positioning system is connected with the visual camera, and the device mainly comprises an acquisition unit 10, a calculation unit 20 and an establishment and conversion unit 30.

[0145] The acquisition unit is used for acquiring images of a calibration ruler photographed by at least two visual cameras, wherein the calibration ruler is located at the end of a moving rod below the visual camera, the calibration ruler moves according to a preset moving direction and a preset moving distance, an image of the calibration ruler is obtained when the calibration ruler moves once, and the calibration ruler is provided with a mark point;

[0146] The calculation unit is used for calculating the position of the mark point in the image of the calibration ruler to obtain the position of the mark point in each image of the calibration ruler.

[0147] The establishing and converting unit is configured to establish a camera coordinate system based on the positions of the identification points in the images of the calibration scales, and determine a conversion relationship between the visual cameras according to the images of the calibration scales, so as to complete the calibration of the visual cameras.

[0148] In the embodiment of the present application, a calibration device for visual cameras is provided. The calibration device is applied to a visual positioning system, and the visual positioning system is connected with the visual cameras. The calibration device comprises: an image acquisition unit configured to acquire images of calibration scales captured by at least two visual cameras, wherein the calibration scales are located at the ends of a moving rod below the visual cameras, the calibration scales are moved according to a preset moving direction and a preset moving distance, an image of the calibration scales is acquired when the calibration scales are moved once, and identification points are arranged on the calibration scales; a position calculation unit configured to calculate the positions of the identification points in the images of the calibration scales; and an establishing and converting unit configured to establish a camera coordinate system based on the positions of the identification points in the images of the calibration scales, and determine a conversion relationship between the visual cameras according to the images of the calibration scales, so as to complete the calibration of the visual cameras. As described above, the calibration of the visual cameras is realized by the positions of the identification points in the images of the calibration scales in the calibration device for visual cameras of the present application, and the images of the calibration scales are captured when the calibration scales are moved according to the preset moving direction and the preset moving distance. The calibration of the visual cameras based on the positions of the identification points is more accurate, and the calibration process is automatically completed. Only one calibration personnel is needed to move the calibration scales, so that the technical problem that the existing calibration device for visual cameras is low in precision and needs many calibration personnel is solved.

[0149] Optionally, the position calculation unit is further configured to: coarsely position the identification points in the images of the calibration scales by using an edge detection operator, to obtain pixel-level edge points of the identification points; perform sub-pixel edge detection on the pixel-level edge points of the identification points, to obtain edge points of the identification points at sub-pixel accuracy; and perform ellipse least square fitting on the edge points of the identification points at sub-pixel accuracy, to obtain the positions of the identification points in the images of the calibration scales.

[0150] Optionally, the establishing and converting unit is further configured to: acquire a preset coordinate system origin of the calibration scales; establish a coordinate system of the movement of the calibration scales according to the coordinate system origin, the preset moving direction and the preset moving distance; and establish the camera coordinate system according to the positions of the identification points in the images of the calibration scales and the coordinate system of the movement of the calibration scales.

[0151] Optionally, the establishing and converting unit is further configured to: perform position and posture solving on the images of the calibration scales by using a camera self-calibration of bundle adjustment, to obtain internal and external parameters of the at least two visual cameras; and determine the conversion relationship between the visual cameras according to the internal and external parameters of the at least two visual cameras, to realize the relative orientation between different visual cameras.

[0152] Optionally, the apparatus is further configured to: acquire a to-be-measured object image of the to-be-measured object captured by the visual camera, wherein the to-be-measured object is provided with a mark point; identify a position of the mark point in the to-be-measured object image to obtain a position of the mark point in the to-be-measured object image; and compensate the position of the mark point in the to-be-measured object image.

[0153] Optionally, the mark point comprises a glass bead and a directional reflective material located below the glass bead.

[0154] Optionally, the preset moving direction comprises a serpentine direction or a diagonal direction.

[0155] The apparatus provided by the embodiments of the present application has the same implementation principle and technical effects as the above-mentioned method embodiments, and for brevity of description, the part not mentioned in the apparatus embodiment can be referred to the corresponding content in the above-mentioned method embodiments.

[0156] As shown in Figure 7 The electronic device 600 provided by the embodiments of the present application comprises a processor 601, a memory 602 and a bus, the memory 602 stores machine readable instructions executable by the processor 601, when the electronic device is running, the processor 601 and the memory 602 communicate through the bus, and the processor 601 executes the machine readable instructions to perform the steps of the above-mentioned visual camera calibration method.

[0157] Specifically, the above-mentioned memory 602 and processor 601 can be general memory and processor, which are not specifically limited here, when the processor 601 runs the computer program stored in the memory 602, the above-mentioned visual camera calibration method can be executed.

[0158] The processor 601 can be an integrated circuit chip having a processing capability of signals. In the implementation process, each step of the above method can be completed by the integrated logic circuit of hardware or the instruction in the form of software in the processor 601. The processor 601 described above can be a general processor, including a central processing unit (CPU), a network processor (NP), etc.; can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. Each method, step and logic block diagram disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in combination with the embodiments of the present application can be directly embodied as a hardware code processor for execution, or a combination of hardware and software modules in the code processor for execution. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register or other mature storage medium in the art. The storage medium is located in the memory 602, and the processor 601 reads the information in the memory 602 and combines the hardware to complete the steps of the above method.

[0159] Corresponding to the above-mentioned calibration method of the visual camera, the embodiments of the present application also provide a computer readable storage medium, the computer readable storage medium stores machine executable instructions, when the processor calls and runs the computer executable instructions, the computer executable instructions make the processor run the steps of the above-mentioned calibration method of the visual camera.

[0160] The calibration device of the visual camera provided by the embodiments of the present application can be specific hardware on the device or software or firmware installed on the device. The device provided by the embodiments of the present application has the same implementation principle and generated technical effects as the above-mentioned method embodiments. For the sake of brevity, the part of the device embodiment not mentioned in the description can refer to the corresponding content in the above-mentioned method embodiments. Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the system, device and unit described above can refer to the corresponding process in the above-mentioned method embodiments, which will not be repeated here.

[0161] In the embodiments of the present application, it should be understood that the disclosed apparatus and method can be implemented in other manners. The embodiments described above are merely specific implementation manners of the present application, and for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation; for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed mutual couplings or direct couplings or communication connections between different units, or the among different units, can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electric, mechanical or other forms.

[0162] For another example, the flowcharts and block diagrams in the drawings show the possible implementation architectures, functions and operations of the apparatus, method and computer program product according to the embodiments of the present application. In this regard, each block in the flowcharts or block diagrams can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in different orders from that shown in the drawings. For example, two consecutive blocks can actually be executed substantially in parallel, and sometimes they can be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and the combination of blocks in the block diagrams and / or flowcharts, can be implemented by a dedicated hardware-based system for performing the specified functions or actions, or can be implemented by a combination of dedicated hardware and computer instructions.

[0163] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units, i.e. they can be located in one place, or can be distributed on a plurality of network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the present embodiment.

[0164] In addition, each functional unit in the embodiments of the present application can be integrated into one processing unit, or each unit can exist physically as a separate unit, or two or more units can be integrated into one unit.

[0165] If the functions are implemented in the form of software function units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application essentially or the parts that contribute to the prior art or parts of the technical solutions can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for making an electronic device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the vehicle marking method described in the embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0166] It should be noted that similar reference numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, the terms "first", "second", "third" and the like are only used to distinguish the description, and cannot be understood as indicating or implying relative importance.

[0167] Finally, it should be noted that the above-described embodiments are only specific embodiments of the present application, which are used to illustrate the technical solutions of the present application, but not to limit them. The protection scope of the present application is not limited thereto. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that any person skilled in the art can modify or easily think of changes to the technical solutions described in the foregoing embodiments within the technical scope disclosed by the present application, or make equivalent replacements to some technical features thereof. These modifications, changes or replacements do not cause the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present application. They should be covered within the protection scope of the present application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims

1. A calibration method for a visual camera, characterized in that, The method includes using a visual positioning system connected to the visual camera, the method comprising: Acquire images of a calibration ruler captured by at least two vision cameras, wherein the calibration ruler is located at the end of a moving rod below the vision cameras, the calibration ruler moves according to a preset moving direction and a preset moving distance, and an image of the calibration ruler is captured each time the calibration ruler moves, and the calibration ruler is provided with marking points; The positions of the marker points in the images of the calibration rulers are calculated to obtain the positions of the marker points in the images of each calibration ruler. A camera coordinate system is established based on the position of the marker points in the images of each calibration ruler, and the transformation relationship between the vision cameras is determined according to the images of the calibration rulers, thereby completing the calibration of the vision cameras; The marker points include: glass beads and directional reflective material located below the glass beads, in order to obtain quasi-binary images; The preset movement direction includes a serpentine direction or a diagonal direction, to ensure that the calibration points do not overlap and are optimally distributed. The calculation of the positions of the marker points in the image of the calibration ruler includes: The edge detection operands are used to coarsely locate the marker points in the image of the calibration ruler to obtain the pixel-level edge points of the marker points; Sub-pixel edge detection is performed on the pixel-level edge points of the marker points to obtain the sub-pixel precision edge points of the marker points; Elliptical least-squares fitting is performed on the edge points of the marker points with sub-pixel precision to obtain the positions of the marker points in the image of the calibration ruler. The establishment of a camera coordinate system based on the position of the marker points in the images of each calibration ruler includes: Obtain the preset coordinate system origin of the calibration ruler; Establish a coordinate system for the movement of the calibration ruler based on the origin of the coordinate system, the preset movement direction, and the preset movement distance; A camera coordinate system is established based on the position of the marker points in the images of each of the calibration rulers and the coordinate system of the movement of the calibration rulers; Determining the conversion relationship between the visual cameras based on the image of the calibration ruler includes: The position and orientation of the image of the calibration ruler are solved by using the bundle adjustment method for camera self-calibration to obtain the intrinsic and extrinsic parameters of at least two vision cameras; The conversion relationship between the visual cameras is determined based on the intrinsic and extrinsic parameters of at least two of the visual cameras, thereby achieving relative orientation between different visual cameras.

2. The method according to claim 1, characterized in that, After completing the calibration of the visual camera, the method further includes: The visual camera captures an image of the object under test, wherein the object under test is provided with marker points. The positions of the marker points in the image of the object under test are identified to obtain the positions of the marker points in the image of the object under test; The position of the marker point in the image of the object under test is compensated.

3. A calibration device for a visual camera, characterized in that, The device includes a visual positioning system connected to the visual camera, and comprises: The acquisition unit is used to acquire images of a calibration ruler captured by at least two vision cameras. The calibration ruler is located at the end of a moving rod below the vision cameras. The calibration ruler moves according to a preset moving direction and a preset moving distance. Each time the calibration ruler moves, an image of the calibration ruler is captured. The calibration ruler is provided with marking points. The calculation unit is used to calculate the position of the marker point in the image of the calibration ruler, and obtain the position of the marker point in the image of each calibration ruler; The establishment and transformation unit is used to establish a camera coordinate system based on the position of the marker point in the image of each calibration ruler, and to determine the transformation relationship between the vision cameras according to the image of the calibration ruler, thereby completing the calibration of the vision cameras; The marker points include: glass beads and directional reflective material located below the glass beads, in order to obtain quasi-binary images; The preset movement direction includes a serpentine direction or a diagonal direction, to ensure that the calibration points do not overlap and are optimally distributed. The calculation unit is further configured to: use edge detection operands to coarsely locate the marker points in the image of the calibration ruler to obtain pixel-level edge points of the marker points; perform sub-pixel edge detection on the pixel-level edge points of the marker points to obtain sub-pixel precision edge points of the marker points; and perform ellipse least squares fitting on the sub-pixel precision edge points of the marker points to obtain the position of the marker points in the image of the calibration ruler. The establishment and conversion unit is further configured to: obtain the preset coordinate system origin of the calibration ruler; establish the coordinate system for the movement of the calibration ruler based on the coordinate system origin, the preset movement direction, and the preset movement distance; and establish a camera coordinate system based on the position of the marker point in the image of each calibration ruler and the coordinate system for the movement of the calibration ruler. The establishment and conversion unit is further configured to: solve the position and orientation of the image of the calibration ruler by using the camera self-calibration with bundle adjustment to obtain the intrinsic and extrinsic parameters of at least two vision cameras; determine the conversion relationship between the vision cameras based on the intrinsic and extrinsic parameters of at least two vision cameras to achieve relative orientation between different vision cameras.

4. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 2.

5. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores machine-executable instructions that, when invoked and executed by a processor, cause the processor to perform the method according to any one of claims 1 to 2.

Images