A docking correction method, a docking correction device, a docking device and a computer device

Abstract

The application discloses a docking correction method, a docking correction device, a docking device and computer equipment, and relates to the technical field of positioning correction. The docking correction method comprises the following steps: acquiring an image, wherein the image comprises imaging of a laser emitter on an inner wall; obtaining a three-dimensional laser point set in a coordinate system of an image collector based on the image; acquiring an external parameter of the image collector in a coordinate system of a first docking device; then, a three-dimensional reference line in the coordinate system of the first docking device is calculated; the relative pose relationship between the first docking device and a second docking device is calculated based on the three-dimensional reference line in the coordinate system of the first docking device; and the pose of the first docking device or the second docking device is adjusted according to the relative pose relationship between the first docking device and the second docking device. The docking correction device comprises an acquisition module, an image processing module, a first calculation module, a second calculation module and a control module. The application guarantees the precision and safety of docking.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of positioning correction, in particular to a docking correction method, a docking correction device, a docking device and a computer device. BACKGROUND

[0002] When docking with a docking device having a hollow inner wall, if there is a large offset angle or a relative position offset, collision is likely to occur, resulting in damage to the docking device. For example, when a material taking arm docks with a material shaft, because the cross section of the material shaft is usually not very large, the material taking arm needs to be more accurately aligned with the material shaft. Moreover, because the hollow inner wall is dark and it is not convenient to set up a recognition mark, the existing docking method through mark detection has poor implementation effect. Therefore, a method for quickly and accurately docking with a docking device having a hollow inner wall is in great need. SUMMARY

[0003] The present application provides a docking correction method, a docking correction device, a docking device and a computer device to solve the problems of the prior art.

[0004] In a first aspect, the present application relates to a docking correction method for docking a first docking device with a second docking device, the first docking device being provided with a laser emitter and an image collector; the second docking device having a hollow inner wall; the method comprising:

[0005] obtaining an image collected by the image collector, the image including an image of the laser emitter on the inner wall;

[0006] based on the image, obtaining a three-dimensional laser point set in the coordinate system of the image collector;

[0007] obtaining an extrinsic parameter of the image collector in the coordinate system of the first docking device;

[0008] based on the extrinsic parameter of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the coordinate system of the image collector, calculating a three-dimensional reference line in the coordinate system of the first docking device; wherein the three-dimensional reference line is obtained based on the three-dimensional laser point set and is used to represent the pose of the second docking device;

[0009] based on the three-dimensional reference line in the coordinate system of the first docking device, calculating the relative pose relationship between the first docking device and the second docking device;

[0010] adjusting the pose of the first docking device or the second docking device according to the relative pose relationship between the first docking device and the second docking device.

[0011] When the first docking device and the second docking device are docked, a three-dimensional reference line is obtained through the collection and calculation of the laser image, and then the relative pose relationship between the first docking device and the second docking device is calculated according to the three-dimensional reference line, so that the pose of the first docking device or the second docking device can be adjusted to make the first docking device and the second docking device docked in parallel, effectively preventing the first docking device from colliding with the inner wall of the second docking device and causing damage, and preventing the second docking device from deviating and falling in the collision, thereby ensuring the accuracy and safety of the docking. The application of the laser collector is low in cost, good in imaging effect, and convenient for image collector collection.

[0012] As preferred, based on the image, a three-dimensional laser point set in the coordinate system of the image collector is obtained, specifically including the following steps:

[0013] At least two correction laser lines located in the inner wall of the second docking device in the image are extracted;

[0014] A plurality of three-dimensional correction coordinates of the correction laser lines in the coordinate system of the image collector are respectively obtained;

[0015] All the three-dimensional correction coordinates form a three-dimensional correction coordinate point set.

[0016] As preferred, the extraction of the at least two correction laser lines located in the inner wall of the second docking device in the image specifically includes the following steps: all the laser lines in the image are identified, if the offset angle of any one of the laser lines reaches a preset threshold, the laser line is the correction laser line, otherwise the laser line is discarded.

[0017] As preferred, the obtaining of the plurality of three-dimensional correction coordinates of the correction laser lines in the coordinate system of the image collector specifically includes the following steps:

[0018] The following steps are repeated until all the correction laser lines are converted into a plurality of three-dimensional correction coordinates:

[0019] A plurality of correction laser points on the correction laser line are extracted;

[0020] Based on the light plane equation of the laser emitter in the coordinate system of the image collector, the three-dimensional correction coordinates of each correction laser point in the coordinate system of the image collector are calculated.

[0021] As preferred, the obtaining of the light plane equation of the laser emitter in the coordinate system of the image collector specifically includes the following steps:

[0022] acquiring the calibration board images and the calibration laser line images of the calibration board in different positions captured by the image collector;

[0023] acquiring a three-dimensional calibration coordinate point set in the coordinate system of the calibration board according to the calibration laser line images;

[0024] acquiring the calibration poses of the calibration board in different positions in the coordinate system of the image collector according to the calibration board images;

[0025] acquiring a three-dimensional synchronous coordinate point set in the coordinate system of the image collector according to the three-dimensional calibration coordinate point set and the calibration poses;

[0026] fitting to acquire a light plane equation in the coordinate system of the image collector according to the three-dimensional synchronous coordinate point set.

[0027] Preferably, the acquiring of the three-dimensional calibration coordinate point set in the coordinate system of the calibration board according to the calibration laser line images specifically comprises the following steps:

[0028] repeating the following steps until all the calibration laser line images are converted into three-dimensional calibration coordinate point sets:

[0029] extracting a plurality of calibration laser points on one of the calibration laser line images;

[0030] acquiring three-dimensional calibration coordinates of the plurality of calibration laser points in the coordinate system of the calibration board, and collecting a three-dimensional calibration coordinate point set from all the three-dimensional calibration coordinates.

[0031] Preferably, based on the extrinsic parameters of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the coordinate system of the image collector, a three-dimensional reference line in the coordinate system of the first docking device is calculated, specifically comprising the following steps:

[0032] based on the extrinsic parameters of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the coordinate system of the image collector, at least two three-dimensional correction laser lines in the coordinate system of the first docking device are fitted;

[0033] a three-dimensional reference line located in the middle of the three-dimensional correction laser lines is calculated according to the three-dimensional correction laser lines.

[0034] Preferably, based on the extrinsic parameters of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the coordinate system of the image collector, at least two three-dimensional correction laser lines in the coordinate system of the first docking device are fitted, specifically comprising the following steps:

[0035] Based on the extrinsic parameters of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the coordinate system of the image collector, the three-dimensional laser point set in the coordinate system of the first docking device is calculated.

[0036] Based on the three-dimensional laser point set in the coordinate system of the first docking device, at least two three-dimensional correction laser lines in the coordinate system of the first docking device are fitted.

[0037] As a preferred, based on the extrinsic parameters of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the coordinate system of the image collector, at least two three-dimensional correction laser lines in the coordinate system of the first docking device are fitted, which specifically includes the following steps:

[0038] Based on the three-dimensional laser point set in the coordinate system of the image collector, at least two three-dimensional correction laser lines in the coordinate system of the image collector are fitted.

[0039] Based on the extrinsic parameters of the image collector in the coordinate system of the first docking device and the three-dimensional correction laser lines in the coordinate system of the image collector, the three-dimensional correction laser lines in the coordinate system of the first docking device are calculated.

[0040] As a preferred, the relative pose relationship between the first docking device and the second docking device is calculated based on the three-dimensional reference line in the coordinate system of the first docking device, which specifically includes the following steps:

[0041] The three-dimensional reference line includes a direction vector and any passing point coordinate;

[0042] The relative angle between the first docking device and the second docking device is calculated according to the direction vector;

[0043] The relative distance between the first docking device and the second docking device is calculated according to the any passing point coordinate.

[0044] As a preferred, the pose of the first docking device or the second docking device is adjusted according to the relative pose relationship between the first docking device and the second docking device, which further includes the following steps: comparing the relative pose relationship with a preset relative pose threshold value, if the relative pose relationship is within the preset relative pose threshold value range, the pose of the first docking device or the second docking device does not need to be adjusted, otherwise the pose of the first docking device or the second docking device is adjusted.

[0045] In a second aspect, the present application relates to a docking correction device for a first docking device internally provided with an image collector and a laser emitter, which comprises:

[0046] an acquisition module, configured to acquire an image captured by the image collector and an extrinsic parameter of the image collector in a coordinate system of the first docking device;

[0047] an image processing module, configured to obtain a three-dimensional laser point set in a coordinate system of the image collector based on the image;

[0048] a first calculation module, configured to calculate a three-dimensional reference line in the coordinate system of the first docking device based on the extrinsic parameter of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the coordinate system of the image collector;

[0049] a second calculation module, configured to calculate a relative pose relationship between the first docking device and the second docking device based on the three-dimensional reference line in the coordinate system of the first docking device;

[0050] a control module, configured to adjust a pose of the first docking device according to the relative pose relationship between the first docking device and the second docking device.

[0051] Preferably, the image processing module comprises:

[0052] an identification and extraction module, configured to extract a laser line in the image and a laser point on the laser line;

[0053] a third calculation module, configured to calculate a three-dimensional coordinate of each laser point in the coordinate system of the image collector based on a light plane equation of the laser emitter in the coordinate system of the image collector.

[0054] In a third aspect, the present application relates to a docking device, which comprises an image collector, a laser emitter, a moving device, a moving control device, a docking correction device; the image collector and the laser emitter are arranged inside the docking device; the image collector and the laser emitter are in communication connection with the docking correction device, the moving control device is in communication connection with the moving device, and the docking correction device is in communication connection with the moving control device.

[0055] In a fourth aspect, the present application relates to a computer device, which comprises a memory and a processor; the memory stores a computer program; when the processor executes the computer program, the steps of the docking correction method are realized.

[0056] In a fifth aspect, the present application relates to a computer readable storage medium, which stores a computer program; when the processor executes the computer program, the steps of the docking correction method are realized.

[0057] Compared with the prior art, the application has the following advantages:

[0058] The first docking device and the second docking device are docked in parallel, effectively preventing the first docking device from colliding with the inner wall of the second docking device and causing damage, and preventing the second docking device from deviating and falling in the collision, thereby ensuring the accuracy and safety of the docking. BRIEF DESCRIPTION OF DRAWINGS

[0059] Figure 1 is a flow chart in one embodiment of the docking correction method of the application.

[0060] Figure 2 is a docking schematic diagram in one embodiment of the docking correction method of the application Figure 1 .

[0061] Figure 3 is a docking schematic diagram in one embodiment of the docking correction method of the application Figure 1 A partial schematic diagram.

[0062] Figure 4 is a docking schematic diagram in one embodiment of the docking correction method of the application Figure 2 .

[0063] Figure 5 is a first docking device schematic diagram in one embodiment of the docking correction method of the application.

[0064] Figure 6 is a second docking device coordinate system schematic diagram in one embodiment of the docking correction method of the application.

[0065] Figure 7 is a module block diagram in one embodiment of the docking correction device of the application.

[0066] Figure 8 is an internal structure diagram in one embodiment of the computer device of the application.

[0067] In the figure: 1-first docking device, 11-laser emitter, 12-image collector, 13-moving device, 2-second docking device, 21-inner wall, 3-imaging of the laser emitter on the inner wall, 4-three-dimensional correction laser line, 5-three-dimensional reference line, 110-acquisition module, 120-image processing module, 130-first calculation module, 140-second calculation module, 150-control module. DETAILED DESCRIPTION

[0068] The application will be further described below in conjunction with the drawings and specific embodiments.

[0069] The technical solutions in the embodiments of the present application will be clearly described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments of the present application. Unless otherwise defined, all the technical and scientific terms used herein have the same meanings as those generally understood by the person skilled in the art to which the present application belongs. The terms used in the specification of the present application are only for the purpose of describing the specific embodiments and are not intended to limit the present application.

[0070] All the directional indications in the embodiments of the present application are only used for explaining the relative positional relationship between the components in a certain posture, and if the certain posture changes, the directional indications also change accordingly. In addition, the descriptions such as "first", "second" and the like in the present application are only for the purpose of description, and cannot be understood as indicating or implying the relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first", "second" can explicitly or implicitly include at least one of the features.

[0071] As shown in Figures 1-6 A docking correction method for docking of a first docking device and a second docking device, the first docking device is provided with a laser emitter and an image collector; the second docking device has a hollow inner wall; the method comprises:

[0072] Step S01, acquiring an image collected by the image collector, the image comprising imaging of the laser emitter on the inner wall.

[0073] Generally, the docking device is parallel to the ground, so the adjustment is mainly in the horizontal direction. When the first docking device and the second docking device do not need to be adjusted in height, only the laser plane parallel to the horizontal plane needs to be emitted, and if higher precision is required, the number of laser planes parallel to the horizontal plane can be increased. When adjustment in height is required, a cross laser plane is emitted, which is composed of two mutually perpendicular laser planes, one of which is parallel to the horizontal plane, and the other is perpendicular to the horizontal plane, and if higher precision is required, the number of cross laser planes can be increased.

[0074] The image collector can adopt a color camera, which is convenient for subsequent extraction of the laser image.

[0075] Step S02, obtaining a three-dimensional laser point set in the coordinate system of the image collector based on the image.

[0076] The coordinate system of the image collector takes the lens of the image collector as the zero point.

[0077] Step S03, obtaining the extrinsic parameter of the image collector in the coordinate system of the first docking device.

[0078] Step S04, based on the extrinsic parameter of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the image collector coordinate system, calculating the three-dimensional reference line in the coordinate system of the first docking device. Wherein, the three-dimensional reference line is established based on the three-dimensional laser point set, and is used to represent the pose of the second docking device.

[0079] Step S05, based on the three-dimensional reference line in the first docking device coordinate system, calculating the relative pose relationship between the first docking device and the second docking device.

[0080] Pose, namely position and attitude, is the general term of the position and direction of the coordinate system.

[0081] Step S06, adjusting the pose of the first docking device or the second docking device according to the relative pose relationship between the first docking device and the second docking device.

[0082] When the first docking device docks with the second docking device, the three-dimensional reference line is obtained through the collection and calculation of the laser image, and then the relative pose relationship between the first docking device and the second docking device is calculated according to the three-dimensional reference line, so as to adjust the pose of the first docking device or the second docking device, so that the first docking device docks with the second docking device in parallel, effectively preventing the first docking device from colliding with the inner wall of the second docking device and causing damage, and also preventing the second docking device from deviating and falling in the collision, thereby ensuring the accuracy and safety of the docking. The application of laser collector has low cost, good imaging effect and is convenient for image collector collection.

[0083] In specific implementation, taking the material taking arm as the first docking device and the material shaft as the second docking device as an example: the material taking arm has a space inside, provided with a laser emitter and an image collector. The material shaft is generally in the shape of a circular tube. When docking, because the height of the material shaft placed in most factories is fixed, the height of the material taking arm does not need to be adjusted, so the laser emitter only needs to emit a laser plane parallel to the horizontal plane. On a large material shaft, there is generally a certain gap between the docking shaft, and the accuracy requirement is not too strict, so the laser plane only needs one to complete the correction. Moreover, because the material shaft is often wound with materials, it is difficult to adjust the direction of the material shaft, so in step S06, the pose of the material taking arm is adjusted according to the relative pose relationship between the material taking arm and the material shaft.

[0084] In some embodiments, in step S02, based on the image, a three-dimensional laser point set in the coordinate system of the image collector is obtained, specifically comprising the following steps:

[0085] In step S21, at least two correction laser lines in the image located on the inner wall of the second docking device are extracted. Because the line connecting the center point of the first docking device and the center point of the second docking device needs to be parallel to the inner wall of the second docking device during alignment, at least two correction laser lines are needed for subsequent calculation. When only two or fewer correction laser lines are identified, it can be known that the offset distance is too large or that each device has a problem, and re-adjustment or repair is performed.

[0086] In step S22, a plurality of three-dimensional correction coordinates of the correction laser line in the coordinate system of the image collector are obtained.

[0087] In step S23, all the three-dimensional correction coordinates are collected as a three-dimensional correction coordinate point set.

[0088] In some embodiments, in step S21, at least two correction laser lines in the image located on the inner wall of the second docking device are extracted, specifically comprising the following steps: all laser lines in the image are identified, if the offset angle of any laser line reaches a preset threshold, the laser line is the correction laser line, otherwise the laser line is discarded.

[0089] Because the correction laser line of the laser irradiation on the inner wall of the second docking device will be deformed, there will be a certain offset angle, and the offset angle will fall within a specific threshold, the specific threshold is set as the preset threshold, so that the correction laser line is extracted simply and quickly. The preset threshold is 5°-35°.

[0090] In some embodiments, in step S22, a plurality of three-dimensional correction coordinates of the correction laser line in the coordinate system of the image collector are obtained, specifically comprising the following steps:

[0091] The following steps are repeated until all the correction laser lines are converted into a plurality of three-dimensional correction coordinates:

[0092] A plurality of correction laser points on the correction laser line are extracted.

[0093] Based on the light plane equation of the laser emitter in the coordinate system of the image collector, the three-dimensional correction coordinates of each correction laser point in the coordinate system of the image collector are calculated. The coordinates of the correction laser point are represented by formula (4), in order to transform to three-dimensional space, formula (1), formula (2) and formula (3) are used to obtain ​​A three-dimensional correction coordinate in a coordinate system of the image collector :

[0094]

[0095]

[0096]

[0097] wherein the represents a Z-axis coordinate in a coordinate system of the image collector, the represents a horizontal coordinate in a coordinate system of the image collector, the represents a vertical coordinate in a coordinate system of the image collector, the K represents an image collector intrinsic parameter, the represents a light plane extrinsic parameter, the represents a coordinate on a light plane, the formula (2) represents that the Z-axis coordinate of the is 0.

[0098] Such setting ensures the accuracy and fast calculation, and the calculation through the light plane equation can be quickly associated with the image collector, thereby improving the overall calculation speed and facilitating the subsequent fitting of the point set.

[0099] In some embodiments, the obtaining of the light plane equation of the laser emitter based on the coordinate system of the image collector specifically comprises the following steps:

[0100] Step a, obtaining the images of the calibration board and the calibration laser line image collected by the image collector at different positions of the calibration board. For example, obtaining the first calibration board image and the first calibration laser line image of the calibration board collected by the image collector at a first position of the calibration board. Obtaining the second calibration board image and the second calibration laser line image of the calibration board collected by the image collector at a second position of the calibration board. The first position and the second position are positions of the calibration board relative to the image collector, and the first position and the second position are different.

[0101] When collecting the calibration board image, the focal length of the image collector is adjusted to make the texture of the collected calibration board image clear. When collecting the calibration laser line image, the focal length of the image collector is adjusted to make the laser line in the collected calibration laser line image clear and visible, and the calibration board is blurred. The calibration laser line image needs to be processed, and the center line of the laser stripe on the calibration laser line image is extracted by the centroid method.

[0102] Step b, obtaining a three-dimensional calibration coordinate point set in the coordinate system of the calibration board according to the calibration laser line image. That is, obtaining a three-dimensional calibration coordinate point set of the center line of the laser stripe in the coordinate system of the calibration board. A first three-dimensional calibration coordinate point set of the first calibration laser line can be obtained according to the first calibration laser line image, and a second three-dimensional calibration coordinate point set of the second calibration laser line can be obtained according to the second calibration laser line image.

[0103] Specifically, the following steps are included:

[0104] The following steps are repeated until all the calibration laser line images are converted into three-dimensional calibration coordinate point sets:

[0105] Extracting a plurality of calibration laser points on one of the calibration laser line images.

[0106] Obtaining three-dimensional calibration coordinates of the plurality of calibration laser points in the coordinate system of the calibration board, and all the three-dimensional calibration coordinates collectively form a three-dimensional calibration coordinate point set. The three-dimensional calibration coordinates of the calibration laser points in the coordinate system of the calibration board all have a Z-axis coordinate of 0, and the horizontal and vertical coordinates are pixel coordinates, and the horizontal and vertical coordinates are the row and column of the calibration laser point image, respectively.

[0107] Step c, obtaining a calibration pose of the calibration board at different positions in the coordinate system of the image collector according to the calibration board images. A first calibration pose of the calibration board at a first position in the coordinate system of the image collector can be obtained according to the first calibration board image, and a second calibration pose of the calibration board at a second position in the coordinate system of the image collector can be obtained according to the second calibration board image.

[0108] Step d, obtaining a three-dimensional synchronization coordinate point set in the coordinate system of the image collector according to the three-dimensional calibration coordinate point set and the calibration pose. A first three-dimensional synchronization coordinate point set can be obtained according to the first three-dimensional calibration coordinate point set and the first calibration pose, and a second three-dimensional synchronization coordinate point set can be obtained according to the second three-dimensional calibration coordinate point set and the second calibration pose.

[0109] Specifically, the calibration pose is set as , the first calibration pose is , and the second calibration pose is . The coordinates of the calibration laser points are represented by . represents the three-dimensional calibration coordinate point coordinates. Substitute and into formula (4), and calculate the three-dimensional synchronization coordinate point coordinates in the coordinate system of the image collector by formula (5) and formula (6) :

[0110]

[0111]

[0112]

[0113] wherein the represents the Z-axis coordinate in the coordinate system of the image collector, and the represents the horizontal coordinate in the coordinate system of the image collector, and the represents the vertical coordinate in the coordinate system of the image collector, and the K represents the image collector intrinsic parameter, and the formula (5) represents that the Z-axis coordinate of the image collector is 0.

[0114] Step e, fitting the light plane equation in the coordinate system of the image collector according to the three-dimensional synchronous coordinate point set. That is, fitting all the three-dimensional synchronous coordinate points to obtain the light plane equation in the coordinate system of the image collector, and completing the light plane calibration. The RANSAC algorithm is used for fitting.

[0115] The RANSAC (Random Sample Consensus) algorithm, that is, the random sample consensus algorithm, is an algorithm for calculating the mathematical model parameters of data from a sample data set containing abnormal data, and obtaining effective sample data.

[0116] The calibrated light plane equation can quickly obtain three-dimensional calibration coordinates of the corrected laser points, and the calibration is rigorous, which ensures the accuracy of the overall alignment and correction.

[0117] In some embodiments, in step S04, based on the extrinsic parameter of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the coordinate system of the image collector, a three-dimensional reference line in the coordinate system of the first docking device is calculated, which specifically includes the following steps:

[0118] Step S41, based on the extrinsic parameter of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the coordinate system of the image collector, at least two three-dimensional correction lasers in the coordinate system of the first docking device are fitted.

[0119] Step S42, according to the three-dimensional correction laser line, a three-dimensional reference line located in the middle of the three-dimensional correction laser line is calculated. The three-dimensional reference line is the angle bisector of the three-dimensional correction laser line. ​

[0120] In some examples, the step S41 specifically comprises the following steps:

[0121] Based on the extrinsic parameters of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the coordinate system of the image collector, the three-dimensional laser point set in the coordinate system of the first docking device is calculated. That is, the three-dimensional correction coordinates are transformed into the coordinate system of the first docking device, and are represented by , which are calculated by formula (7):

[0122]

[0123] wherein, represents the extrinsic parameters of the image collector in the coordinate system of the first docking device.

[0124] Based on the three-dimensional laser point set in the coordinate system of the first docking device, at least two three-dimensional correction laser lines in the coordinate system of the first docking device are fitted. The three-dimensional correction laser lines in the coordinate system of the first docking device are fitted by the RANSAC algorithm.

[0125] In some examples, the step S41 specifically comprises the following steps:

[0126] Based on the three-dimensional laser point set in the coordinate system of the image collector, at least two three-dimensional correction laser lines in the coordinate system of the image collector are fitted. Specifically, the three-dimensional correction laser lines in the coordinate system of the image collector are fitted by the RANSAC algorithm. The equation of a straight line in a two-dimensional plane is , which is usually represented by a straight line, so a three-dimensional correction laser line can be written as .

[0127] Based on the extrinsic parameters of the image collector in the coordinate system of the first docking device and the three-dimensional correction laser lines in the coordinate system of the image collector, the three-dimensional correction laser lines in the coordinate system of the first docking device are calculated.

[0128] Specifically, two points and on the three-dimensional correction laser line are taken, and and are obtained according to formula (8), formula (9) and formula (10) in the coordinate system of the image collector and :

[0129]

[0130]

[0131]

[0132] wherein the represents the Z-axis coordinate in the coordinate system of the image collector, and the represents the horizontal coordinate in the coordinate system of the image collector, and the represents the vertical coordinate in the coordinate system of the image collector, and the K represents the image collector intrinsic parameter, and the represents the light plane extrinsic parameter, and the represents the coordinate on the light plane, and the formula (9) represents that the Z-axis coordinate of the is 0.

[0133] Then, the and are transformed to the coordinate system of the first docking device through the formula (11), and are represented by and

[0134]

[0135] wherein, the extrinsic parameter of the image collector in the coordinate system of the first docking device is represented.

[0136] Finally, the and two points are used to calculate the in the coordinate system of the first docking device. Similarly, the in the coordinate system of the first docking device can be obtained. and are the three-dimensional correction laser lines in the coordinate system of the first docking device.

[0137] As shown in Figure 6 , in step S05, the relative pose relationship between the first docking device and the second docking device is calculated based on the three-dimensional reference line in the coordinate system of the first docking device, and specifically includes the following steps:

[0138] In step S51, the three-dimensional reference line is taken as the x-axis of the second docking device coordinate system, the z-axis is perpendicular to the x-axis upward, the y-axis is determined by the x-axis and the z-axis, and the center coordinate is selected at the starting point of the three-dimensional reference line.

[0139] ​Step S52, the relative pose relationship between the first docking device and the second docking device is calculated, i.e., the relative pose relationship between the first docking device and the second docking device. The relative angle and the relative distance between the origins of the two coordinate systems can be calculated.

[0140] The calculation can quickly obtain the alignment line coordinates, and ensure that the first docking device and the second docking device can be docked in parallel.

[0141] In step S06, the adjusting the pose of the first docking device or the second docking device according to the relative pose relationship between the first docking device and the second docking device further includes the following steps: comparing the relative pose relationship with a preset relative pose threshold value, if the relative pose relationship is within the preset relative pose threshold value range, the pose of the first docking device or the second docking device does not need to be adjusted, otherwise the pose of the first docking device or the second docking device is adjusted.

[0142] When the relative pose relationship is small, the docking can be stable without adjustment, so that the docking time can be saved.

[0143] In one embodiment, as shown in Figure 7 A docking correction device is provided for a first docking device internally provided with an image collector and a laser emitter, which includes:

[0144] An acquisition module is configured to acquire an image collected by the image collector and an external parameter of the image collector in a coordinate system of the first docking device.

[0145] An image processing module is configured to obtain a three-dimensional laser point set in a coordinate system of the image collector based on the image.

[0146] A first calculation module is configured to calculate a three-dimensional reference line in the coordinate system of the first docking device based on the external parameter of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the coordinate system of the image collector.

[0147] A second calculation module is configured to calculate the relative pose relationship between the first docking device and the second docking device based on the three-dimensional reference line in the coordinate system of the first docking device.

[0148] A control module is configured to adjust the pose of the first docking device according to the relative pose relationship between the first docking device and the second docking device.

[0149] In some embodiments, the image processing module includes:

[0150] An identification extraction module is configured to extract the laser line in the image and extract the laser point on the laser line.

[0151] A third calculation module is configured to calculate the three-dimensional coordinates of each laser point in the coordinate system of the image collector based on the light plane equation of the laser emitter.

[0152] The alignment correction method of the docking correction device is the same as the above-described alignment correction method, and will not be described here.

[0153] In one embodiment, a docking device is provided, which includes an image collector, a laser emitter, a moving device, a moving control device, and a docking correction device. The image collector and the laser emitter are arranged inside the docking device. The image collector and the laser emitter are in communication connection with the docking correction device, the moving control device is in communication connection with the moving device, and the docking correction device is in communication connection with the moving control device.

[0154] The alignment correction method of the docking device has been described in detail in the above method embodiments, and will not be described here.

[0155] In one embodiment, a computer device is provided, which can be a terminal, and the internal structure diagram thereof can be as shown in Figure 8 The computer device includes a processor, a memory, a communication interface, a display screen, and an input device connected through a system bus. The processor of the computer device is configured to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The internal memory provides an environment for the operating system and the computer program in the non-volatile storage medium to run. The communication interface of the computer device is configured to perform wired or wireless communication with an external terminal. The wireless communication can be achieved through WIFI, a carrier network, NFC (Near Field Communication), or other technologies. The computer program is executed by the processor to implement a mapping method based on fusion of two types of sensor data or a positioning method based on fusion of two types of sensor data. The display screen of the computer device can be a liquid crystal display screen or an electronic ink display screen. The input device of the computer device can be a touch layer overlaid on the display screen, or can be a key, trackball, or touchpad arranged on the shell of the computer device. In addition, the input device can be an external keyboard, touchpad, or mouse, etc.

[0156] Those skilled in the art can understand, Figure 5The structure shown in the figure is only a block diagram of part of the structure related to the scheme of the present application, and does not constitute a limitation on the computer device to which the scheme of the present application is applied. The specific computer device can include more or fewer components than those shown in the figure, or combine certain components, or have a different arrangement of components.

[0157] In one embodiment, a computer device is also provided, including a memory and a processor, the memory storing a computer program, and the processor executing the computer program to implement the steps in the above method embodiments. The specific alignment correction steps have been described in detail in the above method embodiments, and will not be described here.

[0158] In one embodiment, a computer readable storage medium is provided, storing a computer program, the computer program being executed by a processor to implement the steps in the above method embodiments. The specific steps have been described in detail in the above method embodiments, and will not be described here.

[0159] In the description of the present specification, the description of the terms "one embodiment", "some embodiments", "an example", "a specific example", or "some examples" means that the specific features, structures, materials or characteristics described in connection with the embodiment or example are included in at least one embodiment or example of the present application. In the present specification, the illustrative description of the above terms does not necessarily mean the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner.

[0160] Although the embodiments of the present application have been shown and described above, it should not be understood as a limitation on the claims. The present application is not limited to the above embodiments, and the specific structure allows changes. Any changes made within the scope of the independent claims of the present application are within the scope of the present application.

Claims

1. A docking calibration method for docking a first docking device with a second docking device, characterized in that: The first docking device is provided with a laser emitter and an image collector; The second docking device has a hollow inner wall; the method includes: Acquire an image acquired by the image collector, wherein the image includes an image of the laser emitter on the inner wall; Based on the image, a three-dimensional laser point set is obtained in the coordinate system of the image collector, including: extracting at least two correction laser lines located on the inner wall of the second docking device in the image; respectively obtaining a plurality of three-dimensional correction coordinates of the correction laser lines in the coordinate system of the image collector; the set of all the three-dimensional correction coordinates is the three-dimensional laser point set; the extraction of at least two correction laser lines located on the inner wall of the second docking device in the image includes: identifying all laser lines in the image, and if the offset angle of any one of the laser lines reaches a preset threshold range, then the laser line is the correction laser line, otherwise the laser line is discarded; Acquire the external parameters of the image collector in the coordinate system of the first docking device; Based on the external parameters of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the image collector coordinate system, a three-dimensional reference line in the coordinate system of the first docking device is calculated; wherein, the three-dimensional reference line is established based on the three-dimensional laser point set and is used to characterize the position and posture of the second docking device, including: based on the external parameters of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the image collector coordinate system, fitting at least two three-dimensional correction laser lines in the coordinate system of the first docking device; and calculating a three-dimensional reference line located at the center of the three-dimensional correction laser line based on the three-dimensional correction laser line; Calculating a relative position relationship between the first docking device and the second docking device based on a three-dimensional reference line in the first docking device coordinate system; The posture of the first docking device or the second docking device is adjusted according to the relative posture relationship between the first docking device and the second docking device.

2. A docking correction method according to claim 1, characterized in that: The step of respectively obtaining a plurality of three-dimensional correction coordinates of the correction laser line in the coordinate system of the image collector specifically includes the following steps: Repeat the following steps until all the calibration laser lines are converted into a number of three-dimensional calibration coordinates: extracting a plurality of correction laser points on the correction laser line; Based on the light plane equation emitted by the laser emitter in the coordinate system of the image collector, the three-dimensional correction coordinates of each correction laser point in the coordinate system of the image collector are calculated.

3. A docking correction method according to claim 2, characterized in that: The acquisition of the light plane equation emitted by the laser emitter based on the coordinate system of the image collector specifically includes the following steps: Acquire the calibration plate image and the calibration laser line image captured by the image collector when the calibration plate is at different positions; Acquiring a three-dimensional calibration coordinate point set in the coordinate system of the calibration plate according to the calibration laser line image; Acquire the calibration poses of the calibration plate at different positions in the coordinate system of the image collector according to the calibration plate image; Acquire a three-dimensional synchronous coordinate point set in the coordinate system of the image collector according to the three-dimensional calibration coordinate point set and the calibration pose; The light plane equation in the coordinate system of the image collector is acquired by fitting the three-dimensional synchronous coordinate point set.

4. A docking correction method according to claim 3, characterized in that: The step of obtaining a three-dimensional calibration coordinate point set in the coordinate system of the calibration plate according to the calibration laser line image specifically includes the following steps: Repeat the following steps until all the calibration laser line images are converted into three-dimensional calibration coordinate point sets: extracting a plurality of calibration laser points on the calibration laser line image; The three-dimensional calibration coordinates of a plurality of the calibration laser points in the coordinate system of the calibration plate are obtained, and all the three-dimensional calibration coordinates are collected to obtain a three-dimensional calibration coordinate point set.

5. The docking correction method according to claim 1, characterized in that: Based on the external parameters of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the coordinate system of the image collector, fitting at least two three-dimensional correction laser lines in the coordinate system of the first docking device specifically includes the following steps: Calculating a three-dimensional laser point set in the coordinate system of the first docking device based on the external parameters of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the coordinate system of the image collector; Based on the three-dimensional laser point set in the coordinate system of the first docking device, at least two three-dimensional correction laser lines in the coordinate system of the first docking device are fitted.

6. A docking correction method according to claim 1, characterized in that: Based on the external parameters of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the coordinate system of the image collector, fitting at least two three-dimensional correction laser lines in the coordinate system of the first docking device specifically includes the following steps: Fitting at least two three-dimensional correction laser lines in the image collector coordinate system based on the three-dimensional laser point set in the image collector coordinate system; Based on the external parameters of the image collector in the coordinate system of the first docking device and the three-dimensional correction laser line in the image collector coordinate system, the three-dimensional correction laser line in the coordinate system of the first docking device is calculated.

7. The docking correction method according to claim 1, characterized in that: The step of calculating the relative position relationship between the first docking device and the second docking device based on the three-dimensional reference line in the coordinate system of the first docking device specifically includes the following steps: Obtaining the three-dimensional reference line including the direction vector and the coordinates of any passing point; Calculating a relative angle between the first docking device and the second docking device according to the direction vector; The relative distance between the first docking device and the second docking device is calculated based on the coordinates of any passing point.

8. The docking correction method according to claim 1, characterized in that: The adjusting of the posture of the first docking device or the second docking device according to the relative posture relationship between the first docking device and the second docking device also includes the following steps: comparing the relative posture relationship with a preset relative posture threshold; if the relative posture relationship is within the preset relative posture threshold, there is no need to adjust the posture of the first docking device or the second docking device; otherwise, the posture of the first docking device or the second docking device is adjusted.

9. A docking correction device, for a first docking device having an image collector and a laser emitter inside, the first docking device docking with a second docking device, the second docking device having a hollow inner wall, characterized in that: include: an acquisition module, configured to acquire an image acquired by the image collector and an external parameter of the image collector in the coordinate system of the first docking device, wherein the image includes an image of the laser emitter on the inner wall; An image processing module is configured to obtain, based on the image, a three-dimensional laser point set in the coordinate system of the image collector, including: extracting at least two correction laser lines located on the inner wall of the second docking device in the image; respectively obtaining a plurality of three-dimensional correction coordinates of the correction laser lines in the coordinate system of the image collector; a set of all three-dimensional correction coordinates forming the three-dimensional laser point set; and extracting at least two correction laser lines located on the inner wall of the second docking device in the image including: identifying all laser lines in the image, and if the offset angle of any one of the laser lines reaches a preset threshold range, determining that the laser line is the correction laser line; otherwise, discarding the laser line; A first calculation module is configured to calculate a three-dimensional reference line in the coordinate system of the first docking device based on the external parameters of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the coordinate system of the image collector; wherein the three-dimensional reference line is established based on the three-dimensional laser point set and is used to characterize the position and posture of the second docking device, including: fitting at least two three-dimensional correction laser lines in the coordinate system of the first docking device based on the external parameters of the image collector in the coordinate system of the first docking device and the three-dimensional laser point set in the coordinate system of the image collector; and calculating a three-dimensional reference line located at the center of the three-dimensional correction laser line based on the three-dimensional correction laser line. a second calculation module, configured to calculate a relative position relationship between the first docking device and the second docking device based on a three-dimensional reference line in the first docking device coordinate system; The control module is used to adjust the posture of the first docking device according to the relative posture relationship between the first docking device and the second docking device.

10. The docking correction device according to claim 9, characterized in that: The image processing module includes: an identification and extraction module, configured to extract the laser line in the image and the laser point on the laser line; The third calculation module is used to calculate the three-dimensional coordinates of each laser point in the coordinate system of the image collector based on the plane equation of the light emitted by the laser emitter in the coordinate system of the image collector.

11. A docking device, characterized in that: The docking correction device comprises an image collector, a laser emitter, a mobile device, a mobile control device, and the docking correction device according to any one of claims 9 to 10; the image collector and the laser emitter are arranged inside the docking device; the image collector and the laser emitter are communicatively connected to the docking correction device, the mobile control device is communicatively connected to the mobile device, and the docking correction device is communicatively connected to the mobile control device.

12. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the docking correction method according to any one of claims 1 to 8 are implemented.

13. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the docking correction method according to any one of claims 1 to 8 are implemented.

Images