A calibration method of a monitoring device, a terminal device and a computer storage medium

Abstract

The application discloses a kind of monitoring equipment calibration method, terminal equipment and computer storage medium, the calibration method includes: acquisition continuous multiple frames contain robot image, and from image, the image pose estimation information corresponding to robot is obtained;The odometer corresponding to the odometer pose information of acquisition is obtained for robot;Form closed loop in response to the position corresponding to image pose estimation information, calibrate monitoring equipment using image pose estimation information and odometer pose estimation information, wherein the parameters that monitoring equipment needs to calibrate include at least one parameter in installation orientation angle, installation height, pixel corresponding actual distance proportionality coefficient.The calibration method of the application is when meeting the position corresponding to image pose estimation information of robot forms loop, at least one of installation orientation angle, installation height, pixel corresponding actual distance proportionality coefficient and other parameters to be calibrated of monitoring equipment is calibrated using two kinds of pose information, improve the accuracy and convenience of calibration.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of computer vision, in particular to a calibration method of a monitoring device, a terminal device and a computer storage medium. BACKGROUND

[0002] In image measurement or machine vision applications, the calibration of monitoring device parameters is a very critical link, and the accuracy of the calibration results and the stability of the algorithm directly affect the accuracy of the results produced by the working of the monitoring device. Therefore, good calibration of the monitoring device is the premise of good follow-up work, and improving the calibration accuracy is the focus of scientific research.

[0003] At present, the process of calibrating the external parameters (external parameters) of the monitoring device is complicated, and if the monitoring scene of the monitoring device changes, repeated calibration is required. SUMMARY

[0004] The present application provides a calibration method of a monitoring device, a terminal device and a computer storage medium.

[0005] One of the technical solutions adopted by the present application is to provide a calibration method of a monitoring device, the calibration method comprising:

[0006] Collecting continuous multiple frames of images containing a robot, and obtaining image pose estimation information corresponding to the robot from the images;

[0007] Obtaining odometer pose estimation information corresponding to the collected odometer of the robot;

[0008] In response to the formation of a closed loop at the position corresponding to the image pose estimation information, calibrating the monitoring device using the image pose estimation information and the odometer pose estimation information, wherein the parameters of the monitoring device that need to be calibrated include at least one of the installation orientation angle, the installation height, and the pixel-to-actual distance proportionality coefficient.

[0009] Calibrating the monitoring device using the image pose estimation information and the odometer pose estimation information, comprising:

[0010] Obtaining a target pose of the monitoring device using the camera intrinsic parameters of the monitoring device, the odometer pose estimation information and the image pose estimation information;

[0011] Performing pose calibration using the target pose.

[0012] Obtaining a target pose of the monitoring device using the camera intrinsic parameters of the monitoring device, the odometer pose estimation information and the image pose estimation information, comprising:

[0013] Obtaining a three-dimensional global coordinate of the robot according to the odometer pose estimation information;

[0014] obtaining a robot image center coordinate according to the image pose estimation information;

[0015] obtaining a target pose of the monitoring device by using the camera intrinsic parameter of the monitoring device, the robot three-dimensional global coordinate and the robot image center coordinate.

[0016] performing pose calibration by using the target pose, including:

[0017] determining a first rotation angle, a second rotation angle and a third rotation angle by using the target pose; wherein the first rotation angle is used to represent a rotation angle between the monitoring device and a robot moving surface in an X-axis direction of a world coordinate system, the second rotation angle is used to represent a rotation angle between the monitoring device and the robot moving surface in a Y-axis direction of the world coordinate system, and the third rotation angle is used to represent a rotation angle between the monitoring device and the robot moving surface in a Z-axis direction of the world coordinate system.

[0018] performing pose calibration by using the first rotation angle, the second rotation angle and the third rotation angle.

[0019] after determining the first rotation angle, the second rotation angle and the third rotation angle by using the target pose, including:

[0020] accumulating a mileage distance and a pixel distance between adjacent two points;

[0021] determining a pixel-to-actual distance proportionality coefficient by using the mileage distance, the pixel distance, the first rotation angle, the second rotation angle and the third rotation angle.

[0022] determining the pixel-to-actual distance proportionality coefficient by using the mileage distance, the pixel distance, the first rotation angle, the second rotation angle and the third rotation angle, including:

[0023] determining a plane rotation alignment matrix by using the first rotation angle, the second rotation angle and the third rotation angle.

[0024] determining the pixel-to-actual distance proportionality coefficient by using the plane rotation alignment matrix, the mileage distance and the pixel distance.

[0025] before calibrating the monitoring device by using the image pose estimation information and the mileage pose estimation information, in response to a position corresponding to the image pose estimation information forming a closed loop, including:

[0026] correcting the mileage pose estimation information by using the pixel-to-actual distance proportionality coefficient;

[0027] calibrating the monitoring device by using the image pose estimation information and the mileage pose estimation information, in response to a position corresponding to the image pose estimation information forming a closed loop, including:

[0028] In response to the position corresponding to the image pose estimation information forming a closed loop, the monitoring device is calibrated by using the image pose estimation information and the corrected odometer pose estimation information.

[0029] After the target pose of the monitoring device is obtained by using the camera intrinsic parameter of the monitoring device, the odometer pose estimation information and the image pose estimation information, the method further comprises:

[0030] The position of the camera optical center of the monitoring device in the world coordinate system is determined by using the target pose.

[0031] The installation height is determined according to the position.

[0032] The installation height calibration is performed by using the installation height.

[0033] After the odometer pose estimation information corresponding to the odometer is obtained, the method comprises:

[0034] The pose optimization is performed by using the odometer pose estimation information and the relative pose.

[0035] The odometer is recalibrated by using the odometer pose estimation information after the pose optimization.

[0036] Before the monitoring device is calibrated by using the image pose estimation information and the odometer pose estimation information in response to the position corresponding to the image pose estimation information forming a closed loop, the method comprises:

[0037] When the collection time of the image pose estimation information and the odometer pose estimation information is different, the image pose estimation information and the odometer pose information of the same collection time are determined again by using an interpolation algorithm.

[0038] Another technical solution adopted by the present application is to provide a terminal device, which comprises a memory and a processor connected with the memory.

[0039] The memory is used to store program data, and the processor is used to execute the program data to realize the calibration method of the monitoring device as described above.

[0040] Another technical solution adopted by the present application is to provide a computer storage medium, which is used to store program data, and the program data is used to realize the method of the monitoring device as described above when executed by a computer.

[0041] The beneficial effects of this application are: It obtains the robot's image pose estimation information from multiple consecutive frames of images containing the robot, and combines this with the odometry pose estimation information collected by the robot's odometry. When the positions corresponding to the image pose estimation information form a loop, the two pose information are used to calibrate at least one of the monitoring equipment's installation orientation angle, installation height, and extrinsic parameters. This scheme, combining the robot's own odometry pose estimation information with the robot's corresponding image pose estimation information, improves the accuracy and convenience of monitoring equipment calibration. Attached Figure Description

[0042] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0043] Figure 1 This is a flowchart illustrating an embodiment of the calibration method for monitoring equipment provided in this application;

[0044] Figure 2 This is a schematic diagram of a monitoring device acquiring multiple frames containing image pose estimation information corresponding to the robot.

[0045] Figure 3 This is a flowchart illustrating another embodiment of the calibration method for monitoring equipment provided in this application;

[0046] Figure 4 This is a schematic diagram of the structure of an embodiment of the terminal device provided in this application;

[0047] Figure 5 This is a schematic diagram of the structure of an embodiment of the computer storage medium provided in this application. Detailed Implementation

[0048] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of this application, and not all of the embodiments. Based on the embodiments of this application, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of this application.

[0049] Robot technology gradually spreads from standardized industrial scenarios to daily life, such as restaurant service robots, floor cleaning robots, pet companion robots and the like. The robot performs a specific task in the room and cannot be perceived without the surrounding environment and the motion posture of itself. The robot uses various sensors to complete the perception, and because each implementation principle is different, each has its own advantages. Combining multiple sensors can improve the overall operation effect of the robot.

[0050] With the popular application of indoor monitoring devices, they are increasingly applied to robot perception, and the combined application scheme gradually increases. In order to ensure the positioning accuracy of the robot, the robot and the monitoring device need to be calibrated before use. By calibrating the proportional relationship between the pixels of the monitoring device and the moving pose of the robot, the motion pose of the robot can be provided with reference, and the robot can use multiple pose sources for fusion positioning and pose deviation correction; The calibration of the monitoring device includes the installation position, the pitch, roll and yaw angles, etc.

[0051] In actual application scenarios, the monitoring device is deployed at a certain height in the indoor flat ground. Exemplarily, the monitoring device can be installed at the ceiling. There can be pitch, roll and rotation angles {roll, pitch, yaw} between the camera plane and the ground. Because the scale information of the monocular monitoring camera is uncertain, and the installation can have a certain inclination, the image taken will have a certain deformation, such as a circle can be projected as an ellipse; At the same time, its installation height h is different, resulting in real environment scale and pixel scale coefficient k p are different, and need to be calibrated for use according to different scenarios.

[0052] For details, please refer to Figure 1 , Figure 1 is a flowchart of an embodiment of the calibration method of the monitoring device provided by the present application.

[0053] As Figure 1 shown, the calibration method of the monitoring device of the present application can specifically include the following steps:

[0054] S1, acquire a plurality of continuous frames of images containing the robot, and obtain the image pose estimation information corresponding to the robot from the images.

[0055] The calibration method of the monitoring device provided in the present application is executed by a monitoring device calibration apparatus. In some application scenarios, the monitoring device calibration apparatus can be the monitoring device itself, wherein the robot is in communication connection with the monitoring device. In some application scenarios, the monitoring device calibration apparatus can be a device that is in communication connection with the monitoring device, wherein the monitoring device calibration apparatus is also in communication connection with the robot. In the calibration method of the monitoring device provided in the present application, the robot is in communication connection with the monitoring device, and the robot is a mobile robot. Exemplarily, the monitoring device can be a camera.

[0056] Specifically, the monitoring device calibration apparatus acquires a plurality of continuous frames of images containing the robot through the monitoring device, and obtains image pose estimation information corresponding to the robot from the images. The monitoring device is erected at a certain height in a room with a flat ground. Optionally, the monitoring field of view of the monitoring device can completely cover the ground or partially cover the ground, which is not limited here.

[0057] The image pose estimation information corresponding to the robot obtained by the monitoring device calibration apparatus includes robot image position estimation information and robot image attitude estimation information. Exemplarily, the robot image position estimation information can be coordinate estimation information of the center of the robot in the image. In some embodiments, the monitoring device calibration apparatus can determine the image attitude estimation information of the robot through a visual beacon installed on the robot. Optionally, the visual beacon can be an AprilTag. In some embodiments, the monitoring device calibration apparatus can estimate the image attitude estimation information of the robot through a deep learning algorithm.

[0058] As shown in Figure 2 , Figure 2 is a schematic diagram of the monitoring device acquiring a plurality of frames of image pose estimation information corresponding to the robot. The numbers represent the numbers of the image pose estimation information acquired by the monitoring device in sequence. The arrows represent the orientation of the current robot, and the circles represent the position of the center of the robot.

[0059] Further, the image pose estimation information corresponding to the robot can be represented in the form of a picture coordinate package. In some embodiments, the picture coordinate package can be represented as {time_pixel_stamp, pixel_x, pixel_y, pixel_th}, wherein time_pixel_stamp represents the time stamp of the picture, pixel_x and pixel_y represent the horizontal coordinate and vertical coordinate estimation information of the center of the robot estimated using the image respectively, and pixel_th represents the image attitude estimation information of the robot. Exemplarily, the attitude estimation information can be represented by the orientation of the robot.

[0060] Please refer to Figure 1 .

[0061] S2, acquire odometer pose estimation information of the robot corresponding to the odometer.

[0062] Specifically, the monitoring device calibration apparatus acquires odometer pose estimation information of the robot corresponding to the odometer.

[0063] It can be understood that the robot is provided with a wheeled odometer, and by determining the left and right wheel radii, the left and right wheel track, and the number of revolutions of the left and right wheel odometers, the corresponding movement distance and rotation angle of the robot can be obtained, i.e., the odometer pose estimation information of the robot is acquired by the robot odometer. Illustratively, the pose of the robot at the start of movement is taken as the origin of the world coordinate system, and the coordinates estimated by the wheeled odometer of the robot can be calculated.

[0064] Further, the odometer pose estimation information corresponding to the robot can be represented in the form of an odometer coordinate package. In some embodiments, the odometer coordinate package can be represented as {time_odom_stamp, odom_x, odom_y, odom_th}, wherein time_odom_stamp represents the timestamp of the odometer, odom_x and odom_y represent the horizontal and vertical coordinates of the robot center in the world coordinate system estimated using the odometer, and odom_th represents the odometer pose estimation information of the robot. Illustratively, the pose estimation information can be represented by the orientation of the robot.

[0065] S3, in response to the position corresponding to the image pose estimation information forming a closed loop, calibrating the monitoring device using the image pose estimation information and the odometer pose estimation information.

[0066] When it is detected that the position corresponding to the image pose estimation information of the robot forms a closed polygon, the monitoring device calibration apparatus calibrates the monitoring device using the image pose estimation information and the odometer pose estimation information. The parameters of the monitoring device that need to be calibrated include at least one of the installation orientation angle, the installation height, and the pixel-to-actual distance scale factor.

[0067] In some embodiments, the monitoring device calibration apparatus constructs a robot pose unified structure using the picture coordinate package representing the image pose estimation information and the odometer coordinate package representing the odometer pose estimation information at the same time. It can be understood that before the step of constructing the robot pose unified structure, the time synchronization operation between the robot odometer system and the monitoring device needs to be completed. Alternatively, the robot pose unified structure can be represented as {time_stamp, pixel_x, pixel_y, pixel_th, odom_x, odom_y, odom_th}, wherein time_stamp is the timestamp after time synchronization.

[0068] In some embodiments, the monitoring device calibration apparatus guides the robot to sequentially reach the designated pixel positions in the image in a visual servo manner to complete a closed polygonal trajectory, and continuously acquires images during the movement of the robot. When the deviation of the robot coordinate center in the current frame from the center of the robot in the last frame is greater than one pixel, the pixel coordinate bag is added, and the robot recorded odometer data is combined to obtain a robot pose uniform structure. Optionally, the monitoring device calibration apparatus uses a polygon fitting function (such as the approxPolyDP function in OpenCV) to perform polygon approximation on the robot center contour point set recorded in the pixel coordinate bag, approximately obtains the vertex coordinates of the polygon, and removes the redundant coordinate data generated by the robot during straight line movement. The first and last vertexes of the polygon are made to coincide in the above manner, so that the position difference between the starting point and the last point in the obtained polygon is zero.

[0069] In some embodiments, the monitoring device calibration apparatus controls the robot to construct a closed polygon according to the action mode of rotating a fixed angle in a clockwise or counterclockwise single direction and then advancing a fixed distance, and simultaneously uses the monitoring device to acquire images containing the robot. When the robot rotates, the robot center coordinates in the image are detected as the pixel coordinate vertexes of the polygon. The pixel coordinate bag and the odometer coordinate bag corresponding to these pixel coordinate vertexes are used to form a pose uniform structure, which is used to determine whether the polygon constitutes a closed condition. Further, the monitoring device calibration apparatus sequentially connects two adjacent vertexes after obtaining a vertex coordinate of each polygon, and accumulatively calculates the angle difference between the connecting lines. When the angle difference exceeds π radian and the pixel distance between the first and last vertexes is less than 10 pixels, it is determined whether the number of vertexes at this time is greater than or equal to 3 points. If so, the monitoring device calibration apparatus guides the robot to move from the pixel coordinates of the current last vertex in the image to the pixel coordinates of the initial vertex, and ensures that the final robot orientation angle is consistent with the initial one and the final first and last nodes coincide. The monitoring device calibration apparatus can obtain a closed loop observation result through the odometer for the last path. Optionally, the last path can also be roughly calculated according to the proportional relationship between the pixel distance of the first n-1 connecting line segments and the odometer, through the pixel deviation between the nth point in the image and the origin, to obtain the odometer result as an observation result.

[0070] Further, the monitoring device calibration apparatus calibrates the monitoring device according to the image pose estimation information and the odometer pose estimation information corresponding to the obtained closed polygon vertexes, and determines at least one parameter of the installation orientation angle, the installation height, and the pixel corresponding actual distance proportional coefficient of the monitoring device.

[0071] The above scheme obtains image pose estimation information corresponding to the robot from continuous multiple frames of images containing the robot, combines the odometer pose estimation information collected by the robot odometer, and calibrates at least one of the installation orientation angle, installation height, and external parameter of the monitoring device when the position corresponding to the image pose estimation information forms a loop. The online calibration of the monitoring device is realized by combining the odometer pose estimation information collected by the robot itself and the image pose estimation information corresponding to the robot, and the accuracy and convenience of the calibration of the monitoring device are improved.

[0072] The calibration method of the monitoring device of another embodiment of the application can further include the following steps:

[0073] S11, collecting continuous multiple frames of images containing the robot, and obtaining image pose estimation information corresponding to the robot from the images.

[0074] S12, obtaining odometer pose estimation information collected by the odometer of the robot.

[0075] S13, obtaining three-dimensional global coordinates according to the odometer pose estimation information.

[0076] Specifically, the monitoring device calibration apparatus obtains the three-dimensional global coordinates (x, y, z) of the robot according to the odometer pose estimation information.

[0077] In some embodiments, the pose of the robot at the boot time is taken as the origin of the three-dimensional world coordinate system. In some application scenarios, the environment in which the robot is located is a completely horizontal plane, i.e., the z value in the three-dimensional global coordinates (x, y, z) of the robot is 0 in any case. At the same time, the x and y in the three-dimensional global coordinates correspond to odom_x and odom_y in the odometer coordinate package representing the odometer pose estimation information, respectively.

[0078] S14, obtaining image robot image center coordinates according to the image pose estimation information.

[0079] Specifically, the monitoring device calibration apparatus obtains the center coordinates (u, v) of the robot in the image according to the image pose estimation information.

[0080] The image center coordinates are used to represent the pixel position of the robot center in the image.

[0081] In some embodiments, the monitoring device calibration apparatus obtains the robot center coordinates in the image by using the picture coordinate package representing the image pose estimation information. Specifically, u and v in the image center coordinates (u, v) correspond to pixel_x and pixel_y in the picture coordinate package, respectively.

[0082] S15, obtaining the target pose of the monitoring device by using the camera intrinsic parameter of the monitoring device, the three-dimensional global coordinate of the robot, and the robot image center coordinate.

[0083] Specifically, the monitoring device calibration apparatus determines the target pose of the monitoring device according to the obtained camera intrinsic parameter, three-dimensional global coordinate, and robot image center coordinate by using the relationship between the camera intrinsic parameter, three-dimensional global coordinate, robot image center coordinate, and target pose of the monitoring device.

[0084] The relationship between the camera intrinsic parameter, three-dimensional global coordinate, robot image center coordinate, and target pose of the monitoring device satisfies:

[0085]

[0086] wherein u and v correspond to the robot center coordinate in the image; f x , f y , c x , c y is the camera intrinsic parameter of the monitoring device; x, y, and z correspond to the three-dimensional global coordinate; [R|t] corresponds to the target pose of the monitoring device, wherein R is the rotation matrix of the monitoring device, and t is the translation vector of the monitoring device. The three-dimensional global coordinate and the robot center coordinate in the image are respectively the three-dimensional global coordinate value and the robot center coordinate value in the image corresponding to the vertices of the closed polygon.

[0087] It can be understood that the camera intrinsic parameter is obtained in advance.

[0088] Further, the matrix containing the camera intrinsic parameter is denoted as K, that is:

[0089]

[0090] The 4x3 matrix F composed of K, R, and t is taken as the variable to be solved, the equation AF=0 composed of the known and unknown parameters is expanded and recombined, and the three-dimensional global coordinate and the robot center coordinate in the image corresponding to the different vertices are substituted into the equation and arranged to obtain:

[0091]

[0092] The variable to be solved f ij is obtained by solving by the least square method, that is:

[0093] F=(A T A) -1 A T *0

[0094] The equation result is finally obtained. Understandably, the pose R, t of the camera in the monitoring device contains 6 degrees of freedom, and at least 3 sets of vertices are required to construct the equation set. After F is solved, the pose of the camera in the monitoring device in the world coordinate system is obtained.

[0095] The relationship between R and F satisfies:

[0096] R = K -1 F[1:3]

[0097] The relationship between t and F satisfies:

[0098] t = K -1 F[:4]

[0099] S16, calibrating the pose by using the target pose.

[0100] In some embodiments, the monitoring device calibration apparatus calibrates the pose by using the pose of the camera in the monitoring device in the world coordinate system, and determines the corresponding relationship between the monitoring device and the ground environment. Optionally, the corresponding relationship between the monitoring device and the ground environment can be one or more of the height between the monitoring device and the ground environment, and the rotation angle between the monitoring device and the ground environment.

[0101] The above scheme obtains image pose estimation information corresponding to the robot from continuous multiple frames of images containing the robot, combines the odometer pose estimation information collected by the robot odometer, and calibrates at least one of the installation orientation angle, the installation height, and the external parameter of the monitoring device when the position corresponding to the image pose estimation information forms a loop. At the same time, the odometer pose estimation information collected by the robot itself and the image pose estimation information corresponding to the robot are combined, and after the image pose estimation information corresponding to the robot is collected to form a loop, automatic calibration can be performed, which improves the accuracy and convenience of the calibration of the monitoring device.

[0102] The calibration method of the monitoring device of another embodiment of the application can further include the following steps:

[0103] S21, collecting continuous multiple frames of images containing the robot, and obtaining image pose estimation information corresponding to the robot from the images.

[0104] S22, obtaining odometer pose estimation information collected by the odometer of the robot.

[0105] S23, obtaining a target pose of the monitoring device by using the camera intrinsic parameter of the monitoring device, the odometer pose estimation information, and the image pose estimation information.

[0106] S24, determining a first rotation angle, a second rotation angle, and a third rotation angle by using the target pose.

[0107] wherein the first rotation angle is used to represent the rotation angle between the monitoring device and the robot moving surface in the X-axis direction of the world coordinate system, the second rotation angle is used to represent the rotation angle between the monitoring device and the robot moving surface in the Y-axis direction of the world coordinate system, and the third rotation angle is used to represent the rotation angle between the monitoring device and the robot moving surface in the Z-axis direction of the world coordinate system.

[0108] Specifically, the monitoring device calibration apparatus determines the first rotation angle, the second rotation angle and the third rotation angle by using the obtained target pose.

[0109] wherein the relationship between the rotation matrix R in the target pose and the first rotation angle, the second rotation angle and the third rotation angle respectively satisfies:

[0110] θ x = atan2(-R(2,0),sqrt(R(2,1)*R(2,1)+R(2,2)*R(2,2)))

[0111] θ y = atan2(2,1),(2,2))

[0112] θ z = atan2(R(1,0),R(0,0))

[0113] wherein θ x is the first rotation angle, θ y is the second rotation angle, and θ z is the third rotation angle.

[0114] S25, calibrating the pose by using the first rotation angle, the second rotation angle and the third rotation angle.

[0115] In some embodiments, the monitoring device calibration apparatus calibrates the rotation angle of the camera by using the first rotation angle θ x , the second rotation angle θ y and the third rotation angle θ z .

[0116] The above scheme obtains image pose estimation information corresponding to the robot from continuous multiple frames of images containing the robot, combines the odometer pose estimation information collected by the robot odometer, and calibrates at least one of the installation orientation angle, the installation height and the external parameter of the monitoring device when the position corresponding to the image pose estimation information forms a loop. The combination of the odometer pose estimation information collected by the robot itself and the image pose estimation information corresponding to the robot realizes online calibration of the monitoring device, and improves the accuracy and convenience of the calibration of the monitoring device.

[0117] The calibration method of the monitoring device of another embodiment of the application can further include the following steps:

[0118] S31, acquire a plurality of continuous frames of images containing the robot, and obtain image pose estimation information corresponding to the robot from the images.

[0119] S32, obtain odometer pose estimation information corresponding to the acquired odometer from the robot.

[0120] S33, obtain the target pose of the monitoring device using the camera intrinsic parameters of the monitoring device, the odometer pose estimation information, and the image pose estimation information.

[0121] S34, determine the first rotation angle, the second rotation angle, and the third rotation angle using the target pose.

[0122] S35, accumulate the odometer distance and the pixel distance between adjacent two points.

[0123] Specifically, the monitoring device calibration apparatus counts the odometer distance L odom and the pixel distance between adjacent two vertices in the closed polygon, wherein the pixel distance can be represented as the pixel distance D x_pixel in the x-axis direction of the image and the pixel distance D y_pixel in the y-axis direction of the image.

[0124] S36, determine the pixel-to-actual distance proportionality coefficient using the odometer distance, the pixel distance, the first rotation angle, the second rotation angle, and the third rotation angle.

[0125] Specifically, the monitoring device calibration apparatus determines the pixel-to-actual distance proportionality coefficient using the odometer distance and the pixel distance of a plurality of adjacent two points, and the first rotation angle, the second rotation angle, and the third rotation angle.

[0126] Specifically, the pixel-to-actual distance proportionality coefficient can be represented as kp. Specifically, it can also be represented as the image horizontal coordinate-to-actual distance proportionality coefficient kp x and the image vertical coordinate-to-actual distance proportionality coefficient kp y.

[0127] S37, perform pose calibration using the first rotation angle, the second rotation angle, and the third rotation angle.

[0128] In some embodiments, the monitoring device calibration apparatus calibrates the rotation angle of the camera using the first rotation angle θ x , the second rotation angle θ y , and the third rotation angle θ z . In some embodiments, the monitoring device calibration apparatus can also calibrate the rotation angle of the camera using the first rotation angle θ x , the second rotation angle θ y , and the third rotation angle θ zThe pixel corresponding actual distance proportional coefficient, that is, the camera external parameter of the monitoring device, is derived, and the monitoring device calibration device can also calibrate the camera external parameter of the monitoring device by using the first rotation angle, the second rotation angle and the third rotation angle.

[0129] The above scheme obtains image pose estimation information corresponding to the robot from continuous multiple frames of images containing the robot, combines the odometer pose estimation information collected by the robot odometer, and calibrates at least one of the installation orientation angle, the installation height and other external parameters of the monitoring device when the position corresponding to the image pose estimation information forms a loop. The online calibration of the monitoring device is realized by combining the odometer pose estimation information collected by the robot itself and the image pose estimation information corresponding to the robot, and the accuracy and convenience of the calibration of the monitoring device are improved.

[0130] The calibration method of the monitoring device of another embodiment of the present application can further include the following steps:

[0131] S41, continuous multiple frames of images containing the robot are collected, and image pose estimation information corresponding to the robot is obtained from the images.

[0132] S42, odometer pose estimation information collected by the odometer of the robot is obtained.

[0133] S43, the target pose of the monitoring device is obtained by using the camera internal parameter of the monitoring device, the odometer pose estimation information and the image pose estimation information.

[0134] S44, the first rotation angle, the second rotation angle and the third rotation angle are determined by using the target pose.

[0135] S45, the odometer distance and the pixel distance between adjacent two points are accumulated.

[0136] S46, the planar rotation alignment matrix is determined by using the first rotation angle, the second rotation angle and the third rotation angle.

[0137] Specifically, the monitoring device calibration device determines the planar rotation alignment matrix, that is, the upper left 2x2 matrix of the inverse rotation matrix, by using the first rotation angle θ x , the second rotation angle θ y and the third rotation angle θ z .

[0138] Further, the planar rotation alignment matrix can be expressed as:

[0139] M(θ x ,θ y ,θ z )

[0140] By determining the plane rotation alignment matrix, a parallel mapping relationship between the image coordinate system xy plane and the global coordinate system xy plane is constructed.

[0141] S47, the pixel corresponding actual distance scale factor is determined by using the plane rotation alignment matrix, the odometer distance and the pixel distance.

[0142] Specifically, the monitoring device calibration apparatus determines the pixel corresponding actual distance scale factor kp_x and kp_y by using the relationship between the plane alignment matrix, the odometer distance, the pixel distance and the pixel corresponding actual distance scale factor.

[0143] The relationship between the plane rotation alignment matrix, the odometer distance, the pixel distance and the pixel corresponding actual distance scale factor satisfies:

[0144] L_odom' = M(θ x ,θ y ,θ z )*[kp_x*D_x_pixel+pose_init_x,kp_y*D_y_pixel+pose_init_y]

[0145] Wherein, M is a rotation alignment matrix of the image plane and the global coordinate according to the rotation angle. pose_init_x and pose_init_y are respectively the x-axis and y-axis coordinates of the image coordinate system origin corresponding in the world coordinate system.

[0146] S48, the pose calibration is performed by using the first rotation angle, the second rotation angle and the third rotation angle.

[0147] In some embodiments, the monitoring device calibration apparatus calibrates the rotation angle of the camera by using the first rotation angle θ x , the second rotation angle θ y and the third rotation angle θ z . In some embodiments, the monitoring device calibration apparatus can also derive the pixel corresponding actual distance scale factor by using the first rotation angle θ x , the second rotation angle θ y and the third rotation angle θ z .

[0148] The above scheme obtains image pose estimation information corresponding to the robot from continuous multiple frames of images containing the robot, combines the odometer pose estimation information collected by the robot odometer, and calibrates at least one of the installation orientation angle, installation height, and pixel scale of the monitoring device when the position corresponding to the image pose estimation information forms a loop.

[0149] The calibration method of the monitoring device of another embodiment of the present application can further include the following steps:

[0150] S51, collect continuous multiple frames of images containing the robot, and obtain image pose estimation information corresponding to the robot from the images.

[0151] S52, obtain odometer pose estimation information collected by the odometer of the robot.

[0152] S53, correct the odometer pose estimation information using the pixel-to-actual distance scale factor.

[0153] Specifically, after obtaining the pixel-to-actual distance scale factor once, the monitoring device calibration device corrects the odometer pose estimation information using the obtained pixel-to-actual distance scale factor.

[0154] Illustratively, after obtaining the camera extrinsic parameter and pixel scale factor in the monitoring device once, the monitoring device calibration device corrects the odometer pose estimation information using the camera extrinsic parameter and image pose parameter in the monitoring device.

[0155] The corrected odometer pose estimation information can be represented as pose_camera_x, pose_camera_y, and pose_camera_th.

[0156] Wherein, pose_camera_x satisfies the following relationship:

[0157] pose_camera_x = M(θ x ,θ y ,θ z )(0,0)*(kp_x*pixel_x+pose_init_x)+M(θ x ,θ y ,θ z )(0,1)*(kp_y*pixel_y+pose_init_y)

[0158] Wherein, pose_camera_y satisfies the following relationship:

[0159] pose_camera_y=M(θ x ,θ y ,θ z )(1,0)*(kp_x*pixel_x+pose_init_x)+M(θ x ,θ y ,θ z )(1,1)*(kp_y*pixel_y+pose_init_y)

[0160] Where pose_camera_th satisfies the following relationship:

[0161] pose_camera_th=pixel_th+pose_init_th

[0162] Furthermore, based on the corrected odometer pose estimation information, the monitoring equipment calibration device can also estimate the odometer's proportional coefficients ko_x = pose_camera_x / odom_x and ko_y = pose_camera_y / odom_y. Then, during use, the information collected by the odometer can be directly corrected using the proportional coefficients.

[0163] pose_odom_x can satisfy the following relationship:

[0164] pose_odom_x=ko_x*odom_x

[0165] pose_odom_y can satisfy the following relationship:

[0166] pose_odom_y=ko_y*odom_y

[0167] Where odom_x and odom_y are the horizontal and vertical coordinates in the world coordinate system of the odometry estimation information before correction, respectively.

[0168] Understandably, pose_odom_th satisfies:

[0169] pose_odom_th=odom_th

[0170] S54, responding to the position corresponding to the image pose estimation information to form a closed loop, uses the image pose estimation information and the odometer pose estimation information to calibrate the monitoring equipment.

[0171] The above scheme obtains image pose estimation information corresponding to the robot from continuous multiple frames of images containing the robot, combines the odometer pose estimation information collected by the robot odometer, and calibrates at least one of the installation orientation angle, installation height, and pixel scale of the monitoring device when the position corresponding to the image pose estimation information forms a loop. At the same time, the robot odometer pose estimation information and the image pose estimation information corresponding to the robot are combined to realize online calibration of the monitoring device, improving the convenience of calibration. Further, after completing the calibration of the monitoring device once, the parameters of the last calibration are updated when the robot forms a closed loop at the position containing the robot image, improving the accuracy and convenience of the monitoring device calibration.

[0172] The calibration method of the monitoring device of another embodiment of the present application can further include the following steps:

[0173] S61, collecting continuous multiple frames of images containing the robot, and obtaining image pose estimation information corresponding to the robot from the images.

[0174] S62, obtaining odometer pose estimation information collected by the odometer of the robot.

[0175] S63, correcting the odometer pose estimation information using the pixel-to-actual distance scale factor.

[0176] S64, in response to the position corresponding to the image pose estimation information forming a closed loop, calibrating the monitoring device using the image pose estimation information and the corrected odometer pose estimation information.

[0177] Specifically, when the position corresponding to the image pose estimation information forms a closed polygon, the monitoring device calibration device calibrates the monitoring device using the image pose estimation information and the corrected odometer estimation information.

[0178] Further, after the calibration of the monitoring device is completed, the image pose estimation information and the corrected odometer pose estimation information of the calibrated monitoring device can be fused, and the fused result can be used as the pose of the robot:

[0179] pose_fuse_x=f_x(pose_camera_x,pose_odom_x)

[0180] pose_fuse_y=f_y(pose_camera_y,pose_odom_y)

[0181] pose_fuse_th=f_th(pose_camera_th,pose_odom_th)

[0182] Wherein, pose_fuse_x, pose_fuse_y and pose_fuse_th represent the fused robot pose, and f_x, f_y and f_th represent the mapping relationship between the image pose estimation information and the odometer pose estimation information and the fused robot pose respectively.

[0183] The above scheme obtains image pose estimation information corresponding to the robot from continuous multiple frames of images containing the robot, combines odometer pose estimation information collected by the robot odometer, and calibrates at least one of the installation orientation angle, the installation height and the external parameter of the monitoring device when the position corresponding to the image pose estimation information forms a loop. Meanwhile, the online calibration of the monitoring device is realized by combining the odometer pose estimation information collected by the robot itself and the image pose estimation information corresponding to the robot, and the convenience of calibration is improved. Further, after completing the calibration of the monitoring device once, when the robot forms a closed loop at the position of the image containing the robot, the parameters of the last calibration can be updated, and the accuracy and convenience of the calibration of the monitoring device are improved.

[0184] The calibration method of the monitoring device of another embodiment of the present application can further include the following steps:

[0185] S71, continuous multiple frames of images containing the robot are collected, and image pose estimation information corresponding to the robot is obtained from the images.

[0186] S72, odometer pose estimation information collected by the odometer of the robot is obtained.

[0187] S73, the target pose of the monitoring device is obtained by using the camera intrinsic parameter of the monitoring device, the odometer pose estimation information and the image pose estimation information.

[0188] S74, the position of the camera optical center of the monitoring device in the world coordinate system is determined by using the target pose.

[0189] Specifically, the monitoring device calibration apparatus determines the position of the camera optical center of the monitoring device in the world coordinate system by using the rotation matrix R and the translation vector t in the target pose of the monitoring device.

[0190] Exemplarily, t is obtained as the position of the camera optical center in the world coordinate system in front of the monitoring device calibration apparatus.

[0191] S75, the installation height is determined according to the position.

[0192] Specifically, the monitoring device calibration apparatus determines the installation height of the monitoring device according to the position of the camera optical center in the world coordinate system, and the first rotation angle, the second rotation angle and the third rotation angle.

[0193] S76, calibrate the installation height by using the installation height.

[0194] Specifically, the monitoring device calibration apparatus calibrates the installation height by using the installation height.

[0195] S77, calibrate the pose by using the target pose.

[0196] In some embodiments, the monitoring device calibration apparatus calibrates the rotation angles of the camera by using the first rotation angle θ x , the second rotation angle θ y and the third rotation angle θ z determined by the target pose. In some embodiments, the monitoring device calibration apparatus can further derive the pixel-to-actual distance scale factor by using the first rotation angle θ x , the second rotation angle θ y and the third rotation angle θ z determined by the target pose. In some embodiments, the monitoring device calibration apparatus can further calibrate the height by using the installation height of the camera determined by the target pose.

[0197] The above scheme obtains image pose estimation information corresponding to the robot from a plurality of continuous images containing the robot, combines the odometer pose estimation information collected by the robot odometer, and calibrates at least one of the installation orientation angle, the installation height and the pixel scale factor of the monitoring device when the position corresponding to the image pose estimation information forms a loop. Meanwhile, the odometer pose estimation information collected by the robot itself and the image pose estimation information corresponding to the robot are combined to realize online calibration of the monitoring device, and the accuracy and convenience of the calibration of the monitoring device are improved.

[0198] The calibration method of the monitoring device according to another embodiment of the present application can further include the following steps:

[0199] S81, collect a plurality of continuous images containing the robot, and obtain image pose estimation information corresponding to the robot from the images.

[0200] S82, obtain odometer pose estimation information collected by the odometer of the robot.

[0201] S83, perform pose optimization by using the odometer pose estimation information and the relative pose.

[0202] Specifically, the monitoring device calibration apparatus performs iterative optimization by using the odometer pose estimation information collected by the robot and the relative pose.

[0203] wherein the odometer pose is the odometer pose estimation information corresponding to each vertex, and the relative pose η ij satisfies the following relationship:

[0204]

[0205] Among them, R i ≡R i ( i φ is a planar rotation matrix. j The rotation angle estimated by the robot's odometry at the current vertex, φ i The rotation angle estimated by the robot's odometry at the previous vertex, x j and x i These correspond to the robot's position at the current vertex and the previous vertex, respectively.

[0206] Relative pose η ij It will also be affected by zero-mean Gaussian noise, which satisfies the following relationship:

[0207] ε ij ~N(0,p ij )

[0208] In other words, the relationship between the measured relative pose and the actual pose satisfies:

[0209]

[0210] in, For the original measurement results, η ij This represents the actual result.

[0211] Understandably, the relative pose relationship is not calculated using adjacent nodes, but is obtained by directly guiding the robot to move based on the pixels of the first and last nodes, or by calculating the odometry difference through pixel ratio.

[0212] Specifically, the optimization objective of the odometry pose estimation information is to minimize Gaussian noise, that is:

[0213] min∑|r ij (p)| 2

[0214] Where, r ij The square of (p) is expressed as:

[0215]

[0216] Optionally, Q is divided into two parts: one part is the odometer's generated value q_odom_ij, and the other part is the closed-loop generated value q_lc_ij.

[0217] The generated value q_odom_ij of the odometer satisfies:

[0218] q_odom ij =iag([cos(φ)i )(a3*AT + a4*AR); sin(φ i )(a3*AT + a4

[0219] *AR); 1*AR + a2*AT])

[0220] Wherein, AR is the single direction rotation angle estimated by the odometer, AA is the single direction displacement value estimated by the odometer, a1, a2, a3, a4 are weight parameters respectively.

[0221] The q_lc_ij generated by the closed loop satisfies:

[0222] q_lc_ij = diag([1, 1, 1])C ij

[0223] Wherein, C ij Satisfies:

[0224]

[0225] Wherein, m is the total number of vertices, θ ik And θ jk Are the angles between the remaining vertices and the previous vertex and the current vertex respectively.

[0226] Optionally, Q can also be set as a diagonal matrix with the same coefficient, and the value of the coefficient is obtained through actual test.

[0227] Further, the optimal solution p j p i needs to be solved. In some embodiments, the monitoring device calibration device uses the Gauss Newton method to solve.

[0228] First, the Jacobian matrix J of the optimization objective function is calculated, and the numerical method is to calculate the following formula when a small amount of p is randomly used to move from node i to node j:

[0229]

[0230] Then calculate the inverse Jacobian from node j to node i:

[0231]

[0232] Then add the results of the two to get the average:

[0233] F(x+dx) = (x) + J()

[0234] Take the square of the above formula to get:

[0235] dx T J(x) T J(x)dx + 2J(x)f(x)dx + f(x)2

[0236] Differentiate dx again, get:

[0237] J(x) T J(x)dx=-J(x)f(x)

[0238] Expand the objective function to get the derivative:

[0239] J T JΔp j p i =-Jr ij

[0240] Each time get the iteration deviation Δp j p i , gradually solve the update p j p i , wherein p j p in+1 =p j p in +Δp j p i . Adjust the weight through Q. In some embodiments, the iteration is completed when the number of iterations is greater than or equal to a set iteration threshold value, and the iteration threshold value can be 100, for example. In other embodiments, the iteration is completed when the iteration deviation is less than a set value, and the iteration set value can be 0.01, for example.

[0241] S84, recalibrate the odometer using the optimized odometer pose estimation information.

[0242] Specifically, the monitoring device calibration apparatus recalibrates the odometer using the optimized odometer pose estimation information.

[0243] By recalibrating the odometer, the cumulative error of the robot odometer pose estimation over time is reduced.

[0244] S85, in response to the position corresponding to the image pose estimation information forming a closed loop, calibrate the monitoring device using the image pose estimation information and the odometer pose estimation information.

[0245] The above scheme obtains image pose estimation information corresponding to the robot from continuous multiple frames of images containing the robot, combines the odometer pose estimation information collected by the robot odometer, and calibrates at least one of the installation orientation angle, installation height, and pixel proportion coefficient of the monitoring device when the position corresponding to the image pose estimation information forms a loop. In the process of calibrating the monitoring device, the odometer information of the robot is also calibrated, which improves the accuracy of the odometer pose estimation information. The above scheme does not require manual intervention and can be calibrated once after the image pose estimation information corresponding to the robot forms a loop, thereby improving the accuracy and convenience of the calibration of the monitoring device.

[0246] The calibration method of the monitoring device of another embodiment of the present application can further include the following steps:

[0247] S91, continuous multiple frames of images containing the robot are collected, and image pose estimation information corresponding to the robot is obtained from the images.

[0248] S92, odometer pose estimation information corresponding to the odometer collected is obtained.

[0249] S93, when the collection time of the image pose estimation information and the odometer pose estimation information is different, an interpolation algorithm is used to determine the image pose estimation information and the odometer pose estimation information of the same collection time.

[0250] In some application scenarios, when the sampling frequency of the odometer is higher than the sampling frequency of the monitoring device, the collection time of the image pose estimation information is different from the collection time of the odometer pose estimation information, and the monitoring device calibration device uses an interpolation algorithm to determine the image pose estimation information and the odometer pose estimation information of the same sampling time.

[0251] In some embodiments, the sampling time odom0 of the odometer is earlier than the sampling time of the image, and the next sampling time odom1 of the odometer is later than the sampling time of the image, and the odometer prediction value in the image is represented as:

[0252] K1=(pic_timestamp-odom0_timestamp) / (odom1_timestamp-odom0_timestamp)

[0253] Wherein, pic_time_stamp represents the timestamp of the image, odom0_timestamp represents the timestamp corresponding to the current sampling time of the odometer, odom1_timestamp represents the timestamp corresponding to the next sampling time of the odometer, and K1 represents the interpolation proportion.

[0254] After obtaining the interpolation ratio, the odometer pose estimation information with the same image sampling time can be expressed as:

[0255] pic_odom_x = odom_x + K1 * (odom1_x - odom0_x)

[0256] pic_odom_y = odom_y + K1 * (odom1_y - odom0_y)

[0257] pic_odom_th = odom_th + K1 * (odom1_th - odom0_th)

[0258] Further, the obtained robot pose uniform structure can be a structure with an image timestamp as a timestamp, and the obtained robot pose uniform structure can be represented as {time_stamp, pixel_x, pixel_y, pixel_th, odom_x, odom_y, odom_th}.

[0259] S94, in response to the position corresponding to the image pose estimation information forming a loop, calibrating the monitoring device by using the image pose estimation information and the odometer pose estimation information.

[0260] The above scheme obtains the image pose estimation information corresponding to the robot from the continuous multiple frames of images containing the robot, combines the odometer pose estimation information collected by the robot odometer, and calibrates at least one of the installation orientation angle, the installation height, and the pixel scale factor of the monitoring device and other calibration parameters when the position corresponding to the image pose estimation information forms a loop. Further, when the timestamps of the image pose estimation information and the odometer information pose information are inconsistent, the time information of the two kinds of pose information is unified by using an interpolation algorithm. At the same time, the above scheme does not require manual intervention, and can calibrate once after the image pose estimation information corresponding to the robot forms a loop, thereby improving the accuracy and convenience of the calibration of the monitoring device.

[0261] Please refer to Figure 3 , Figure 3 which is a step flowchart of a calibration method of a monitoring device according to another embodiment of the present application.

[0262] As shown in Figure 3 , the calibration method of the monitoring device according to another embodiment of the present application can specifically include the following steps:

[0263] S101, odometer information collection.

[0264] In which, S101 corresponds to the content described in S2 above, which will not be repeated here.

[0265] S102, picture information collection.

[0266] S102, determining whether the robot is in the image.

[0267] S103, collecting information linear interpolation fusion.

[0268] S103 corresponds to the content described in S93 above, and will not be repeated here.

[0269] S104, determining whether the path of the robot is closed.

[0270] S104 corresponds to the content described in S3 above, and will not be repeated here.

[0271] If the result is yes, jump to S105. If the result is no, jump to S101.

[0272] S105, pose optimization updates odometer results.

[0273] S105 corresponds to the content of S83 above, and will not be repeated here.

[0274] S106, solving the external parameters of the monitoring device and the pixel scale coefficient.

[0275] S106 corresponds to the content of S15 and S46 above, and will not be repeated here.

[0276] The above scheme obtains the image pose estimation information corresponding to the robot from the continuous multiple frames of images containing the robot, combines the odometer pose estimation information collected by the robot odometer, and when the position corresponding to the image pose estimation information forms a loop, at least one of the installation orientation angle, installation height and pixel scale coefficient of the monitoring device is calibrated. At the same time, the above scheme does not need human intervention, and can be calibrated once after the image pose estimation information corresponding to the robot is collected to form a loop, thereby improving the accuracy and convenience of the monitoring device calibration.

[0277] Please continue to see Figure 4 , Figure 4 Figure 1 is a structural schematic diagram of an embodiment of a terminal device provided by the present application. The terminal device 500 of the embodiment of the present application comprises a processor 51 and a memory 52.

[0278] The processor 51, the memory 52 and the bus are connected, the memory 52 stores program data, and the processor 51 is used to execute the program data to realize the calibration method of the monitoring device described in the above embodiment. In some embodiments, the terminal device 500 is the monitoring device calibration device itself.

[0279] In the embodiments of the present application, the processor 51 can also be referred to as a CPU (Central Processing Unit). The processor 51 can be an integrated circuit chip with processing capability. The processor 51 can also be a general-purpose processor, a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a FPGA (Field Programmable Gate Array) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The general-purpose processor can be a microprocessor, or the processor 51 can also be any conventional processor.

[0280] The present application also provides a computer storage medium, please continue to refer to Figure 5 , Figure 5 is a structural schematic diagram of an embodiment of the computer storage medium provided by the present application. The computer storage medium 600 stores program data 61, which, when executed by a processor, implements the calibration method of the monitoring device of the above-mentioned embodiments.

[0281] When the embodiments of the present application are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application or the whole or part of the technical solutions that make essential contributions to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a number of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes a U disk, a mobile hard disk, a ROM (Read-Only Memory), a RAM (Random Access Memory), a magnetic disk or an optical disk, and various media that can store program codes.

[0282] The above only describes the embodiments of the present application, and does not limit the patent scope of the present application. The equivalent structure or equivalent flow transformation made by the content of the present application specification and drawings, or direct or indirect application in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A calibration method for monitoring equipment, characterized in that, The method includes: Acquire multiple consecutive frames of images containing the robot, and obtain image pose estimation information corresponding to the robot from the images; Obtain the odometry pose estimation information collected by the robot's odometry. In response to the formation of a closed loop at the position corresponding to the image pose estimation information, the monitoring device is calibrated using the image pose estimation information and the odometer pose estimation information. The parameters requiring calibration of the monitoring device include at least one of the following: installation orientation angle, installation height, and pixel-to-actual-distance ratio coefficient. The calibration of the monitoring device using the image pose estimation information and the odometer pose estimation information includes: The target pose of the monitoring device is obtained by using the camera intrinsic parameters of the monitoring device, the odometer pose estimation information, and the image pose estimation information. The target pose is used for pose calibration.

2. The method according to claim 1, characterized in that, The step of obtaining the target pose of the monitoring device using the camera intrinsic parameters of the monitoring device, the odometer pose estimation information, and the image pose estimation information includes: The robot's three-dimensional global coordinates are obtained based on the odometry pose estimation information; The robot image center coordinates are obtained based on the image pose estimation information; The target pose of the monitoring device is obtained by using the camera intrinsic parameters of the monitoring device, the three-dimensional global coordinates of the robot, and the image center coordinates of the robot.

3. The method according to claim 1, characterized in that, The pose calibration using the target pose includes: The first rotation angle, the second rotation angle, and the third rotation angle are determined using the target pose; wherein, the first rotation angle is used to represent the rotation angle between the monitoring device and the robot moving surface in the X-axis direction of the world coordinate system, the second rotation angle is used to represent the rotation angle between the monitoring device and the robot moving surface in the Y-axis direction of the world coordinate system, and the third rotation angle is used to represent the rotation angle between the monitoring device and the robot moving surface in the Z-axis of the world coordinate system; Pose calibration is performed using the first rotation angle, the second rotation angle, and the third rotation angle.

4. The method according to claim 3, characterized in that, After determining the first rotation angle, the second rotation angle, and the third rotation angle using the target pose, the process includes: Accumulate the odometer distance and pixel distance between two adjacent points; Using the odometer distance, the pixel distance, the first rotation angle, the second rotation angle, and the third rotation angle, the pixel-to-actual-distance ratio coefficient is determined.

5. The method according to claim 4, characterized in that, The step of determining the pixel-to-actual-distance ratio coefficient using the odometer distance, the pixel distance, the first rotation angle, the second rotation angle, and the third rotation angle includes: The planar rotation alignment matrix is ​​determined using the first rotation angle, the second rotation angle, and the third rotation angle; The actual distance ratio coefficient corresponding to the pixel is determined by using the planar rotation alignment matrix, the odometer distance, and the pixel distance.

6. The calibration method according to claim 5, characterized in that, Before calibrating the monitoring device using the image pose estimation information and the odometry pose estimation information, the following steps are included in response to the formation of a closed loop at the position corresponding to the image pose estimation information: The odometry pose estimation information is corrected using the pixel-to-actual-distance scaling factor. In response to the formation of a closed loop at the position corresponding to the image pose estimation information, the monitoring device is calibrated using the image pose estimation information and the odometry pose estimation information, including: In response to the formation of a closed loop at the position corresponding to the image pose estimation information, the monitoring device is calibrated using the image pose estimation information and the corrected odometer pose estimation information.

7. The method according to claim 1, characterized in that, After obtaining the target pose of the monitoring device using the camera intrinsic parameters of the monitoring device, the odometer pose estimation information, and the image pose estimation information, the method further includes: The position of the camera optical center of the monitoring device in the world coordinate system is determined using the target pose. The installation height is determined based on the location; The installation height is calibrated using the aforementioned installation height.

8. The calibration method according to claim 1, characterized in that, After obtaining the odometry pose estimation information corresponding to the odometry of the robot, the process includes: Pose optimization is performed using the odometry pose estimation information and relative pose. The odometer is recalibrated using the odometer pose estimation information after pose optimization.

9. The calibration method according to claim 1, characterized in that, Before calibrating the monitoring device using the image pose estimation information and the odometry pose estimation information, the following steps are included in response to the formation of a closed loop at the position corresponding to the image pose estimation information: When the acquisition times of the image pose estimation information and the odometer pose estimation information are different, an interpolation algorithm is used to re-determine the image pose estimation information and the odometer pose estimation information with the same acquisition time.

10. A terminal device, characterized in that, The terminal device includes a processor and a memory connected to the processor, wherein... The memory stores program instructions; The processor is configured to execute program instructions stored in the memory to implement the method as described in any one of claims 1 to 9.

11. A computer-readable storage medium, characterized in that, The storage medium stores program instructions that, when executed, implement the method as described in any one of claims 1 to 9.

Images