A visual sensor calibration method and device, electronic device and storage medium

Abstract

The present disclosure relates to a calibration method and device for a visual sensor, an electronic device and a storage medium, the method comprising: determining a first coordinate value of a reference point on a calibration object in a world coordinate system, wherein the calibration object comprises a second pattern obtained by perspective transformation of a first pattern, the first pattern comprises a plurality of reference points arranged regularly, and a first distance between reference points in the calibration object is positively correlated with a second distance between the reference points and the visual sensor; determining a second coordinate value of the reference point in an image captured by the visual sensor; and determining a parameter to be calibrated of the visual sensor according to the first coordinate value and the second coordinate value. The embodiments of the present disclosure can improve the calibration accuracy of the visual sensor.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technology, and in particular to a calibration method and device for a visual sensor, an electronic device, and a storage medium. Background Art

[0002] In applications such as image ranging and machine vision based on visual sensors, the camera parameters are often calibrated to determine the correspondence between the position of a surface point in space and the position of its image point in the image. Whether in image ranging or machine vision, the calibration of camera parameters is a very critical link, and the accuracy of the calibration results directly affects the accuracy of the results produced by the camera.

[0003] In related technologies, camera parameters are often calibrated based on a chessboard. During the calibration process, the chessboard is used as a calibration object, and the camera is calibrated by extracting the coordinates of the corner points at the junction of the black and white chessboards in the image space. However, the extraction accuracy of the chessboard corner point coordinates is easily affected by the shooting angle, lighting, and camera depth of field, resulting in low calibration accuracy.

[0004] Especially for low-angle robots (such as sweeping robots), since the low-angle robot body is very low, the visual sensor it carries is also very low. In this case, if the visual sensor of the sweeping robot is calibrated according to the calibration object on the ground plane, it will be very difficult, resulting in low calibration accuracy. Summary of the invention

[0005] The present disclosure proposes a technical solution for calibrating a visual sensor.

[0006] According to one aspect of the present disclosure, a method for calibrating a visual sensor is provided, comprising:

[0007] Determine a first coordinate value of a reference point on a calibration object in a world coordinate system, wherein the calibration object includes a second pattern obtained by perspective transformation of a first pattern, the first pattern includes a plurality of reference points arranged regularly, and a first distance between reference points in the calibration object is positively correlated with a second distance between the reference points and the visual sensor;

[0008] Determine a second coordinate value of the reference point in the image captured by the visual sensor;

[0009] The parameters to be calibrated of the visual sensor are determined according to the first coordinate value and the second coordinate value.

[0010] In a possible implementation manner, before determining the first coordinate value of the reference point on the calibration object in the world coordinate system, the method further includes:

[0011] Determining the perspective transformation relationship between the imaging of the visual sensor and the real object;

[0012] According to the perspective transformation relationship, the first pattern is perspective transformed to obtain the second pattern.

[0013] In a possible implementation, determining the perspective transformation relationship between the imaging of the visual sensor and the real object includes:

[0014] The perspective transformation relationship is determined according to the height of the visual sensor relative to the ground and the shooting angle.

[0015] In a possible implementation, determining the perspective transformation relationship according to the height of the visual sensor relative to the ground and the shooting angle includes:

[0016] Calibrate the visual sensor according to a first pattern located on the ground to obtain an initial internal parameter of the visual sensor;

[0017] Obtaining initial external parameters of the visual sensor according to the height of the visual sensor relative to the ground and the shooting angle;

[0018] An initial homography matrix of the visual sensor is determined by using the initial external parameters and the initial internal parameters as the perspective transformation relationship.

[0019] In a possible implementation, performing perspective transformation on the first pattern according to the perspective transformation relationship to obtain the second pattern includes:

[0020] The first pattern is projected into a world coordinate system using the initial homography matrix, and the projected pattern is used as the second pattern.

[0021] In a possible implementation manner, determining the second coordinate value of the reference point in the image captured by the visual sensor includes:

[0022] detecting the first pattern in the image;

[0023] identifying a location of a reference point in the detected first pattern in the image;

[0024] The coordinate value of the position is used as the second coordinate value of the reference point in the image.

[0025] In a possible implementation, the parameters to be calibrated include a homography matrix;

[0026] The step of determining the parameters to be calibrated of the visual sensor according to the first coordinate value and the second coordinate value includes:

[0027] According to the correspondence between the first coordinate value and the second coordinate value of the same reference point, a homography matrix for characterizing the projection relationship between the position in the image and the position in the world coordinate system is determined.

[0028] According to one aspect of the present disclosure, a calibration device for a visual sensor is provided, comprising:

[0029] a first coordinate value determination module, configured to determine a first coordinate value of a reference point on a calibration object in a world coordinate system, wherein the calibration object includes a second pattern obtained by perspective transformation of the first pattern, the first pattern includes a plurality of reference points arranged regularly, and a first distance between the reference points in the calibration object is positively correlated with a second distance between the reference points and the visual sensor;

[0030] A second coordinate value determination module, used to determine a second coordinate value of the reference point in the image captured by the visual sensor;

[0031] A parameter determination module is used to determine the parameters to be calibrated of the visual sensor according to the first coordinate value and the second coordinate value.

[0032] In a possible implementation manner, the device further includes:

[0033] A perspective transformation relationship determination module, used to determine the perspective transformation relationship between the imaging of the visual sensor and the real object;

[0034] The second pattern determination module is used to perform perspective transformation on the first pattern according to the perspective transformation relationship to obtain a second pattern.

[0035] In a possible implementation, the perspective transformation relationship determination module is used to determine the perspective transformation relationship according to the height of the visual sensor relative to the ground and the shooting angle.

[0036] In a possible implementation, the perspective transformation relationship determination module includes:

[0037] An initial internal parameter determination submodule, used to calibrate the visual sensor according to a first pattern located on the ground to obtain an initial internal parameter of the visual sensor;

[0038] An initial external parameter determination submodule, used to obtain the initial external parameters of the visual sensor according to the height of the visual sensor relative to the ground and the shooting angle;

[0039] The initial homography matrix determination submodule determines the initial homography matrix of the visual sensor using the initial external parameters and the initial internal parameters as the perspective transformation relationship.

[0040] In a possible implementation, the second pattern determination module is configured to project the first pattern into a world coordinate system using the initial homography matrix, and use the projected pattern as the second pattern.

[0041] In a possible implementation, the second coordinate value determining module includes:

[0042] A first pattern detection submodule, configured to detect the first pattern in the image;

[0043] a position identification submodule, configured to identify a position of a reference point in the detected first pattern in the image;

[0044] The second coordinate value determination submodule is used to use the coordinate value of the position as the second coordinate value of the reference point in the image.

[0045] In a possible implementation, the parameters to be calibrated include a homography matrix;

[0046] The parameter determination module is used to determine a homography matrix used to characterize the projection relationship between the position in the image and the position in the world coordinate system according to the corresponding relationship between the first coordinate value and the second coordinate value of the same reference point.

[0047] According to one aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing processor-executable instructions; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0048] According to one aspect of the present disclosure, a computer-readable storage medium is provided, on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor.

[0049] In the disclosed embodiment, by determining the first coordinate value of the reference point on the calibration object in the world coordinate system, the calibration object includes a second pattern obtained by perspective transformation of the first pattern, the first pattern includes a plurality of regularly arranged reference points, and the first distance between the reference points in the calibration object is positively correlated with the second distance of the reference point from the visual sensor; then determining the second coordinate value of the reference point in the image collected by the visual sensor; according to the first coordinate value and the second coordinate value, determining the calibration parameter of the visual sensor. Thus, since the first pattern includes a plurality of regularly arranged reference points, and the second pattern is obtained after perspective transformation of the first pattern, in the process of calibrating the visual sensor using the second pattern, the second pattern will be perspectively transformed again by the visual sensor, so that the reference points in the image collected by the visual sensor are regularly arranged, which is convenient for accurately and quickly detecting the second coordinate value of the reference point in the image. In addition, since the first distance between the reference points is positively correlated with the second distance of the reference points from the visual sensor, the reference points far away in the image collected by the visual sensor are also easy to identify, thereby improving the calibration accuracy of the visual sensor (especially the visual sensor with a low viewing angle).

[0050] It should be understood that the above general description and the following detailed description are exemplary and explanatory only and do not limit the present disclosure. Other features and aspects of the present disclosure will become clear from the following detailed description of exemplary embodiments with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used to illustrate the technical solutions of the present disclosure together with the specification.

[0052] Figure 1 A flow chart of a method for calibrating a visual sensor according to an embodiment of the present disclosure is shown.

[0053] Figure 2 A schematic diagram of a pattern of a calibration object according to an embodiment of the present disclosure is shown.

[0054] Figure 3 A block diagram of a visual sensor calibration device according to an embodiment of the present disclosure is shown.

[0055] Figure 4 A block diagram of an electronic device according to an embodiment of the present disclosure is shown.

[0056] Figure 5 A block diagram of an electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0057] Various exemplary embodiments, features and aspects of the present disclosure will be described in detail below with reference to the accompanying drawings. The same reference numerals in the accompanying drawings represent elements with the same or similar functions. Although various aspects of the embodiments are shown in the accompanying drawings, the drawings are not necessarily drawn to scale unless otherwise specified.

[0058] The word “exemplary” is used exclusively herein to mean “serving as an example, example, or illustration.” Any embodiment described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments.

[0059] The term "and / or" herein is only a description of the association relationship of the associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C can represent including any one or more elements selected from the set consisting of A, B, and C.

[0060] In addition, in order to better illustrate the present disclosure, numerous specific details are given in the following specific embodiments. It should be understood by those skilled in the art that the present disclosure can also be implemented without certain specific details. In some examples, methods, means, components and circuits well known to those skilled in the art are not described in detail in order to highlight the subject matter of the present disclosure.

[0061] In applications such as image ranging, machine vision, and three-dimensional scene reconstruction based on vision sensors, a geometric model of vision sensor imaging is often established in order to correct the lens distortion of the vision sensor, determine the conversion relationship between the physical size in the three-dimensional space and the pixel size of the image, and determine the correspondence between the three-dimensional geometric position of a spatial object (or a point on the surface of a spatial object) and the coordinates of the corresponding pixel point of the spatial object (or a point on the surface of a spatial object) in the image. The parameters of the geometric model are the vision sensor parameters, and the process of solving the parameters is the calibration process of the vision sensor. The accuracy of the calibration result directly affects the accuracy of the results produced by the vision sensor.

[0062] In recent years, the technology related to low-angle robots (such as sweeping robots) has developed rapidly. In order to further improve the user experience and create smarter products, low-angle robots need to be equipped with some visual sensors to achieve tasks such as detection and ranging. These sensors include visible light (RGB) visual sensors, time of flight (TOF) visual sensors, etc., which can work normally after calibration.

[0063] Since the low-angle robot (such as a sweeping robot) has a very low body, the visual sensor it carries is also very low. In this case, it will be very difficult to calibrate the visual sensor of the sweeping robot based on the calibration object on the ground plane.

[0064] In the related art, when calibrating the visual sensor of a low-view robot, a checkerboard calibration plate is placed on the ground plane, and the visual sensor of the low-view robot is calibrated through the correspondence between the positions of multiple corner points on the checkerboard in the world coordinate system and the positions in the image coordinate system. However, since the distance between the visual sensor of the low-view robot and the ground is very small, when the calibration plate is placed on the ground, due to the perspective principle, many corner points at a distance (for example, more than 40 cm) are very blurred, occupy very few pixels in the image, and are very close in distance, making them difficult to detect and distinguish, which leads to a decrease in calibration accuracy.

[0065] The embodiment of the present disclosure provides a calibration method for a visual sensor, by determining the first coordinate value of a reference point on a calibration object in a world coordinate system, the calibration object includes a second pattern obtained by perspective transformation of a first pattern, the first pattern includes a plurality of reference points arranged regularly, and the first distance between the reference points in the calibration object is positively correlated with the second distance between the reference points and the visual sensor; then determining the second coordinate value of the reference point in the image collected by the visual sensor; and determining the parameters to be calibrated of the visual sensor according to the first coordinate value and the second coordinate value. Thus, since the first pattern includes a plurality of reference points arranged regularly, and the second pattern is obtained after perspective transformation of the first pattern, in the process of calibrating the visual sensor using the second pattern, the second pattern will be perspectively transformed again by the visual sensor, so that the reference points in the image collected by the visual sensor are regularly arranged, which is convenient for accurately and quickly detecting the second coordinate value of the reference point in the image. In addition, since the first distance between the reference points is positively correlated with the second distance between the reference points and the visual sensor, the reference points far away in the image collected by the visual sensor are also easy to identify, thereby improving the calibration accuracy of the visual sensor (especially the visual sensor with a low viewing angle).

[0066] In a possible implementation, the visual sensor calibration method can be executed by an electronic device such as a terminal device or a server, and the terminal device can be a user equipment (UE), a mobile device, a user terminal, a terminal, a cellular phone, a cordless phone, a personal digital assistant (PDA), a handheld device, a computing device, a vehicle-mounted device, a wearable device, etc. The method can be implemented by a processor calling a computer-readable instruction stored in a memory. Alternatively, the method can be executed by a server.

[0067] Figure 1 A flow chart showing a method for calibrating a visual sensor according to an embodiment of the present disclosure is shown as follows: Figure 1 As shown, the calibration method of the visual sensor includes:

[0068] In step S11, a first coordinate value of a reference point on the calibration object in the world coordinate system is determined.

[0069] The calibration object includes a second pattern obtained by perspective transformation of the first pattern, the first pattern includes a plurality of regularly arranged reference points, and a first distance between the reference points in the calibration object is positively correlated with a second distance between the reference points and the visual sensor.

[0070] The calibration object is placed in the real space. It often includes multiple reference points. The visual sensor can be calibrated by using the correspondence between the position of the reference point in the world coordinate system and the position in the image coordinate system. The calibration object includes a pattern. By selecting easily identifiable points in the pattern as reference points, the pattern on the calibration object can be one or more of a circle, rectangle, square, etc. The pattern can be divided into different areas. Adjacent areas can be represented by different colors, and the reference point can be located at the junction of adjacent areas, so as to facilitate the identification of the reference point on the image of the visual sensor.

[0071] After the first pattern is transformed by perspective, due to the perspective relationship that objects near are large and objects far are small, the distances between the regularly arranged reference points will change after the perspective transformation. For example, the distances between the reference points will gradually increase from one side of the pattern to the other side. Here, the first pattern is a chessboard as an example. Figure 2 As shown, the checkerboard pattern (first pattern) on the left, after perspective transformation, will become the second pattern on the right. In the first pattern, the reference points are the corner points of the square, the reference points are arranged regularly, and the distances between the reference points are the same; while in the second pattern, the distances between the reference points gradually increase from the bottom to the top of the second pattern.

[0072] Then, when placing a calibration object including a second pattern on the ground for visual sensor calibration, the first distance between reference points in the calibration object can be positively correlated with the second distance between the reference points and the visual sensor (i.e., smaller when closer and larger when farther away). In this way, when the visual sensor collects the second pattern on the ground, due to the optical perspective transformation relationship (i.e., larger when closer and smaller when farther away), the pattern in the image obtained on the imaging plane of the visual sensor is closer to the shape of the first pattern, its reference points are regularly arranged, and distant reference points are also easy to identify in the image.

[0073] The world coordinate system is the coordinate system used by the robot's electronic map. The world coordinate system is a three-dimensional coordinate system. The first coordinate value of the world coordinate system can represent the position of the reference point in the real geographic space. The world coordinate system here can be a coordinate system constructed with the robot where the visual sensor is located as the origin. Then, the first coordinate value can represent the position of the reference point relative to the robot's origin. The first coordinate value can be obtained by manual measurement or by a ranging sensor.

[0074] In step S12, a second coordinate value of the reference point in the image captured by the vision sensor is determined.

[0075] When the visual sensor collects the second pattern on the ground, due to the optical perspective transformation relationship, the pattern in the image obtained on the imaging plane of the visual sensor is closer to the shape of the first pattern, its reference points are regularly arranged, and reference points in the distance are also easy to distinguish.

[0076] The second coordinate value is the coordinate value of the reference point in the coordinate system of the image. The coordinate system of the image can select an origin in the image and construct a two-dimensional xy coordinate system. For example, the upper left corner of the image can be selected as the origin O, the horizontal direction as the x-axis, and the vertical direction as the y-axis to construct the coordinate system.

[0077] The second coordinate value may be determined by the user through a marking operation in the image, or may be obtained by detecting a reference point on the image. The specific process may refer to possible implementation methods described below and will not be described in detail here.

[0078] In step S13, the parameters to be calibrated of the visual sensor are determined according to the first coordinate value and the second coordinate value.

[0079] After determining the first coordinate value of the reference point in the world coordinate system and the second coordinate value in the image captured by the vision sensor, the parameters to be calibrated of the vision sensor can be calibrated.

[0080] In the process of calibrating the visual sensor, the parameters to be calibrated may include the intrinsic parameters and / or extrinsic parameters of the visual sensor, and may also include the homography matrix of the visual sensor, wherein each value of the intrinsic parameter is related to the internal parameters of the camera and does not change with the position of the object. The intrinsic parameter is used to indicate the conversion relationship between the camera coordinate system and the image coordinate system; the extrinsic parameter is used to indicate the position conversion relationship between the world coordinate system and the camera coordinate system; and the homography matrix is ​​used to indicate the conversion relationship between the world coordinate system and the image coordinate system.

[0081] The specific process of calibrating the parameters to be calibrated of the visual sensor can be found in the relevant technology, which will not be described here.

[0082] In the disclosed embodiment, since the first pattern includes a plurality of reference points arranged regularly, and the second pattern is obtained after perspective transformation of the first pattern, in the process of calibrating the visual sensor using the second pattern, the second pattern will be perspectively transformed again by the visual sensor, so that the reference points in the image captured by the visual sensor are regularly arranged, which is convenient for accurately and quickly detecting the second coordinate value of the reference point in the image. In addition, since the first distance between the reference points is positively correlated with the second distance of the reference point from the visual sensor, the reference points in the distance in the image captured by the visual sensor are also easy to identify, which improves the calibration accuracy of the visual sensor (especially the low-angle visual sensor).

[0083] In a possible implementation, before determining the first coordinate value of the reference point on the calibration object in the world coordinate system, the method also includes: determining the perspective transformation relationship between the imaging of the visual sensor and the real object; and performing perspective transformation on the first pattern according to the perspective transformation relationship to obtain a second pattern.

[0084] During the perspective transformation process, the pattern will be projected onto a new viewing plane, also known as projective mapping. Then, the second pattern obtained by the perspective transformation of the first pattern can be obtained by projecting the first pattern onto a new viewing plane. This new viewing plane can be the plane where the calibration object is located, and the original plane where the first pattern is located can be the image plane of the visual sensor.

[0085] Then, the second pattern can be obtained by projecting the first pattern in the image plane imaged by the visual sensor onto the plane where the calibration object is located. In the case where the calibration object is located on the ground, the new visual plane can be the ground, and then the second pattern can be obtained by projecting the first pattern in the image plane imaged by the visual sensor onto the ground.

[0086] For the visual sensor calibration process, the perspective transformation relationship is the mapping relationship between the visual sensor imaging plane and the plane where the calibration object is located, and the mapping relationship can be obtained based on the relative relationship between the two planes. The relative relationship may include, for example: the angle between the imaging plane and the plane where the calibration object is located, and the distance between the imaging plane and the plane where the calibration object is located. Taking the plane where the calibration object is located as the ground as an example, the perspective transformation relationship can be determined based on the height of the visual sensor relative to the ground and the shooting angle. The specific determination process can be referred to the possible implementation methods provided in the present disclosure, which will not be repeated here.

[0087] In the disclosed embodiment, the second pattern is obtained by determining the perspective transformation relationship between the imaging of the visual sensor and the real object, and performing perspective transformation on the first pattern according to the perspective transformation relationship. In other words, the perspective transformation of the first pattern is based on the perspective transformation relationship between the imaging of the visual sensor and the real object, so that the image of the second pattern obtained after passing through the visual sensor again can be closer to the first pattern, and because the reference points in the first pattern are regularly arranged, the reference points regularly arranged in the image can be used for calibration during the calibration of the visual sensor, which makes it easy to distinguish distant reference points in the image, thereby improving the calibration accuracy of the visual sensor (especially the low-angle visual sensor).

[0088] In a possible implementation, determining the perspective transformation relationship based on the height and shooting angle of the visual sensor relative to the ground includes: calibrating the visual sensor according to a first pattern located on the ground to obtain initial intrinsic parameters of the visual sensor; obtaining initial extrinsic parameters of the visual sensor according to the height and shooting angle of the visual sensor relative to the ground; and determining an initial homography matrix of the visual sensor using the initial extrinsic parameters and the initial intrinsic parameters as the perspective transformation relationship.

[0089] In the calibration process of the visual sensor, the homography matrix is ​​used to represent the conversion relationship between the world coordinate system (real object) and the image coordinate system (imaging). That is to say, the homography matrix can be used as the perspective transformation relationship between the imaging of the visual sensor and the real object. Therefore, we can first roughly obtain an initial homography matrix, and then use the homography matrix as the perspective transformation relationship to perform a perspective transformation on the first pattern to obtain a second pattern. The second pattern is then used to calibrate the visual sensor to obtain a homography matrix with higher accuracy.

[0090] The homography matrix can be described by the following expression (1):

[0091] H=M[R,T] (1)

[0092] Where H represents the homography matrix, M represents the intrinsic parameter matrix, [R, T] represents the extrinsic parameter matrix, R represents the shooting angle of the visual sensor in the extrinsic parameter, in three-dimensional space, R can have three components, T represents the offset component of the visual sensor in the extrinsic parameter, and when the calibration object is located on the ground, T will include the height z of the visual sensor relative to the ground.

[0093] In the process of roughly obtaining the initial homography matrix, the visual sensor may be calibrated using the first pattern located on the ground to obtain the initial internal parameters of the visual sensor, and the specific form of the initial internal parameters may be the internal parameter matrix M. For example, the visual sensor may be calibrated using a chessboard located on the ground to obtain the initial internal parameters of the visual sensor, and the calibration process may be implemented based on relevant technologies and will not be described in detail here.

[0094] The initial external parameters can be obtained by measuring the height z of the visual sensor relative to the ground and the shooting angle R. After obtaining the initial internal parameters and the initial external parameters, the initial homography matrix of the visual sensor can be obtained by the above expression (1).

[0095] In the disclosed embodiment, the visual sensor is calibrated according to the first pattern located on the ground to obtain the initial internal parameters of the visual sensor; the initial external parameters of the visual sensor are obtained according to the height of the visual sensor relative to the ground and the shooting angle; the initial homography matrix of the visual sensor is determined by using the initial external parameters and the initial internal parameters as the perspective transformation relationship. Thus, by roughly calibrating the visual sensor, an initial homography matrix is ​​obtained, and the homography matrix is ​​used as the perspective transformation relationship to improve the accuracy of the obtained perspective transformation relationship. Then, by using the second pattern obtained based on the perspective transformation relationship to calibrate the visual sensor, the calibration accuracy of the visual sensor (especially the low-angle visual sensor) can be improved.

[0096] In a possible implementation, performing perspective transformation on the first pattern according to the perspective transformation relationship to obtain the second pattern includes: using the initial homography matrix to project the first pattern into a world coordinate system, and using the projected pattern as the second pattern.

[0097] As mentioned above, the specific implementation form of the perspective transformation relationship can be an initial homography matrix. Based on the initial homography matrix, the position of the projection of the pixel points in the first pattern in the plane where the calibration object is located can be obtained. After projecting each pixel point in the first pattern to the plane where the calibration object is located, the second pattern can be obtained.

[0098] In the disclosed embodiment, the first pattern is projected into the world coordinate system by using the initial homography matrix, and the projected pattern is used as the second pattern. Thus, the second pattern obtained by using the perspective transformation relationship with higher accuracy is used as the pattern of the calibration object in the visual sensor calibration process, which can improve the calibration accuracy of the visual sensor (especially the low-angle visual sensor).

[0099] In a possible implementation, determining the second coordinate value of the reference point in the image captured by the visual sensor includes: detecting the first pattern in the image; identifying the position of the reference point in the detected first pattern in the image; and using the coordinate value of the position as the second coordinate value of the reference point in the image.

[0100] Since the second pattern is obtained through perspective transformation of the first pattern, and in the process of calibrating the visual sensor using the second pattern, the second pattern will be perspective transformed again through the visual sensor, so that the pattern obtained in the image captured by the visual sensor is closer to the first pattern. Therefore, by detecting the first pattern in the image, the reference point in the first pattern can be detected more quickly and accurately.

[0101] As for the specific process of detecting the first pattern, it can be achieved through the target recognition technology based on neural network. As a possible implementation method, multiple candidate regions can be extracted from the image through the Region Proposal Network (RPN), and then a feature vector is obtained for each candidate region using a convolutional neural network. For the feature vector corresponding to each region, the support vector machine (SVM) is used for classification to obtain the candidate region containing the first pattern, and the size of the bounding box is adjusted through bounding box regression to finally obtain the position of the detected first pattern. The above process can be implemented based on the trained neural network, and the specific training process is not described here.

[0102] In a possible implementation, determining the parameters to be calibrated of the visual sensor based on the first coordinate value and the second coordinate value includes: determining a homography matrix used to characterize the projection relationship between the position in the image and the position in the world coordinate system based on the correspondence between the first coordinate value and the second coordinate value of the same reference point.

[0103] The homography matrix is ​​used to indicate the conversion relationship between the world coordinate system and the image coordinate system, and the first coordinate value is the position of the reference point in the world coordinate system, and the second coordinate value is the position of the reference point in the image captured by the visual sensor. Therefore, based on the correspondence between the first coordinate value and the second coordinate value, the homography matrix indicating the conversion relationship between the world coordinate system and the image coordinate system can be determined. The specific determination process can be implemented based on relevant technologies and will not be described in detail here.

[0104] In the disclosed embodiment, based on the correspondence between the first coordinate value of the reference point in the world coordinate system and the second coordinate value in the image coordinate system, a homography matrix indicating the conversion relationship between the world coordinate system and the image coordinate system is determined. Thus, the obtained homography matrix has a high accuracy.

[0105] In the embodiment of the present disclosure, the number of calibration objects can be one or at least two, and the present disclosure does not specifically limit the number of calibration objects. When the number of calibration objects is at least two, the patterns in different calibration objects can be the same or different, and the present disclosure does not limit this.

[0106] In a possible implementation, after obtaining the homography matrix, the homography matrix can also be used to calculate the reprojection error. Specifically, the coordinates of the target object in the image captured by the visual sensor can be projected to the ground through the calculated homography matrix to obtain the position coordinates of the target object on the ground; then the difference between the position coordinates and the actual position coordinates of the target object is determined to obtain the reprojection error. The reprojection error can be used to measure the accuracy of the obtained homography matrix. If the accuracy does not meet the expectations, the homography matrix can be further optimized based on the reprojection error until the reprojection error meets the expectations. The specific implementation process can be referred to the relevant technology, which will not be described here.

[0107] A specific application scenario of the embodiment of the present disclosure is described below. For any details not described in the application scenario, please refer to the corresponding records in the previous text, and no further description is given here. In the application scenario, the visual sensor is a visual sensor provided in a sweeping robot. Since the sweeping robot is traveling on the ground, the calibration object is located on the ground.

[0108] In this application scenario, the specific implementation process of the visual sensor calibration method provided by the present disclosure includes:

[0109] Step S21, calibrating the visual sensor using the first pattern located on the ground to obtain the initial internal parameters of the visual sensor.

[0110] The first pattern here may be a chessboard, and the chessboard may be located on paper and placed on the ground after printing.

[0111] In this step, the checkerboard in the calibration object may be a standard checkerboard, and the initial internal reference of the visual sensor is roughly determined by using the standard checkerboard.

[0112] Step S22, obtaining initial external parameters of the visual sensor according to the height of the visual sensor relative to the ground and the shooting angle.

[0113] Step S23, using the initial internal parameters and the initial external parameters, determine the initial homography matrix of the visual sensor as the perspective transformation relationship between the imaging plane of the visual sensor and the ground plane.

[0114] Step S24: using the perspective transformation relationship, project the first pattern from the imaging plane of the visual sensor to the ground plane to obtain a second pattern.

[0115] After obtaining the second pattern, the second pattern can be printed out as a calibration object and placed on the ground. When placing, the first distance between the reference points in the calibration object can be positively correlated with the second distance between the reference points and the visual sensor, that is, the side with a smaller distance between the reference points is placed close to the visual sensor, and the side with a larger distance between the reference points is placed away from the visual sensor, so that the reference points in the distance in the image collected by the visual sensor are also easy to identify.

[0116] Step S25, determining a first coordinate value of a reference point of the second pattern in the calibration object in the world coordinate system.

[0117] The first coordinate value may be obtained by manual measurement, and the world coordinate system may be a two-dimensional coordinate system constructed with the position of the robot as the origin and the ground as the plane.

[0118] Step S26, determining a second coordinate value of the reference point in the image captured by the visual sensor.

[0119] This process can be implemented based on image detection technology, by detecting the first pattern in the image and determining the reference point in the detected first pattern. Since the reference points in the first pattern in the image captured by the visual sensor are regularly arranged, it is convenient to accurately and quickly detect the second coordinate value of the reference point in the image.

[0120] Step S27, determining a homography matrix for representing a projection relationship between a position in the image and a position in the world coordinate system according to a correspondence between the first coordinate value and the second coordinate value of the same reference point.

[0121] Since the viewing angle of the sweeping robot is very low, the image captured by the visual sensor of the sweeping robot will not be easy to identify the reference points at the far end after the perspective transformation. In the disclosed embodiment, by performing a perspective transformation on the regularly arranged first pattern, and then using the second pattern after the perspective transformation as a calibration object, the visual sensor is calibrated. As a result, the reference points at a distance in the image captured by the visual sensor of the sweeping robot are also easy to identify, which improves the calibration accuracy of the visual sensor (especially the visual sensor with a low viewing angle). In addition, the reference points in the image captured by the visual sensor are regularly arranged, which facilitates accurate and rapid detection of the second coordinate value of the reference point in the image.

[0122] In the embodiments of the present disclosure, subsequent applications can be performed based on the calibrated parameters of the visual sensor. For example, image ranging can be performed based on the calibrated visual sensor. According to the position of the target object in the image, the distance of the target object from the visual sensor or the distance from a robot equipped with the visual sensor can be determined. Image ranging can be applied to applications such as autonomous driving and cleaning work of sweeping robots, and has high application value.

[0123] It can be understood that the above-mentioned various method embodiments mentioned in the present disclosure can be combined with each other to form a combined embodiment without violating the principle logic. Due to space limitations, the present disclosure will not repeat them. It can be understood by those skilled in the art that in the above-mentioned method of the specific implementation method, the specific execution order of each step should be determined according to its function and possible internal logic.

[0124] In addition, the present disclosure also provides a calibration device, an electronic device, a computer-readable storage medium, and a program for a visual sensor, all of which can be used to implement any calibration method for a visual sensor provided by the present disclosure. The corresponding technical solutions and descriptions can be found in the corresponding records in the method section and will not be repeated here.

[0125] Figure 3 A block diagram of a calibration device for a visual sensor according to an embodiment of the present disclosure is shown as follows: Figure 3 As shown, the device 30 comprises:

[0126] A first coordinate value determination module 31 is used to determine a first coordinate value of a reference point on a calibration object in a world coordinate system, wherein the calibration object includes a second pattern obtained by perspective transformation of a first pattern, the first pattern includes a plurality of reference points arranged regularly, and a first distance between reference points in the calibration object is positively correlated with a second distance between the reference points and the visual sensor;

[0127] A second coordinate value determination module 32, used to determine a second coordinate value of the reference point in the image captured by the visual sensor;

[0128] The parameter determination module 33 is used to determine the parameters to be calibrated of the visual sensor according to the first coordinate value and the second coordinate value.

[0129] In a possible implementation, the device 30 further includes:

[0130] A perspective transformation relationship determination module, used to determine the perspective transformation relationship between the imaging of the visual sensor and the real object;

[0131] The second pattern determination module is used to perform perspective transformation on the first pattern according to the perspective transformation relationship to obtain a second pattern.

[0132] In a possible implementation, the perspective transformation relationship determination module is used to determine the perspective transformation relationship according to the height of the visual sensor relative to the ground and the shooting angle.

[0133] In a possible implementation, the perspective transformation relationship determination module includes:

[0134] An initial internal parameter determination submodule, used to calibrate the visual sensor according to a first pattern located on the ground to obtain an initial internal parameter of the visual sensor;

[0135] An initial external parameter determination submodule, used to obtain the initial external parameters of the visual sensor according to the height of the visual sensor relative to the ground and the shooting angle;

[0136] The initial homography matrix determination submodule determines the initial homography matrix of the visual sensor using the initial external parameters and the initial internal parameters as the perspective transformation relationship.

[0137] In a possible implementation, the second pattern determination module is configured to project the first pattern into a world coordinate system using the initial homography matrix, and use the projected pattern as the second pattern.

[0138] In a possible implementation, the second coordinate value determining module 32 includes:

[0139] A first pattern detection submodule, configured to detect the first pattern in the image;

[0140] a position identification submodule, configured to identify a position of a reference point in the detected first pattern in the image;

[0141] The second coordinate value determination submodule is used to use the coordinate value of the position as the second coordinate value of the reference point in the image.

[0142] In a possible implementation, the parameters to be calibrated include a homography matrix;

[0143] The parameter determination module 33 is used to determine a homography matrix for characterizing the projection relationship between the position in the image and the position in the world coordinate system according to the correspondence between the first coordinate value and the second coordinate value of the same reference point.

[0144] In some embodiments, the functions or modules included in the device provided by the embodiments of the present disclosure can be used to execute the method described in the above method embodiments. The specific implementation thereof can refer to the description of the above method embodiments, and for the sake of brevity, it will not be repeated here.

[0145] The embodiment of the present disclosure also provides a computer-readable storage medium on which computer program instructions are stored, and the computer program instructions implement the above method when executed by a processor. The computer-readable storage medium can be a volatile or non-volatile computer-readable storage medium.

[0146] An embodiment of the present disclosure further proposes an electronic device, comprising: a processor; and a memory for storing instructions executable by the processor; wherein the processor is configured to call the instructions stored in the memory to execute the above method.

[0147] The embodiments of the present disclosure also provide a computer program product, including a computer-readable code. When the computer-readable code is executed on a device, a processor in the device executes instructions for implementing the visual sensor calibration method provided in any of the above embodiments.

[0148] The embodiments of the present disclosure also provide another computer program product for storing computer-readable instructions, which, when executed, enable a computer to perform the operations of the visual sensor calibration method provided in any of the above embodiments.

[0149] The electronic device may be provided as a terminal, a server, or a device in other forms.

[0150] Figure 4 A block diagram of an electronic device 800 according to an embodiment of the present disclosure is shown. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, or the like.

[0151] Reference Figure 4 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0152] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0153] The memory 804 is configured to store various types of data to support operations on the electronic device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0154] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0155] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0156] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0157] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: home button, volume button, start button, and lock button.

[0158] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the electronic device 800, the relative positioning of the components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of a nearby object without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a complementary metal oxide semiconductor (CMOS) or a charge coupled device (CCD) image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0159] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as a wireless network (WiFi), a second generation mobile communication technology (2G) or a third generation mobile communication technology (3G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0160] In an exemplary embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above methods.

[0161] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 804 including computer program instructions, which can be executed by a processor 820 of an electronic device 800 to perform the above method.

[0162] Figure 51 is a block diagram of an electronic device 1900 according to an embodiment of the present disclosure. For example, the electronic device 1900 may be provided as a server. Figure 5 , the electronic device 1900 includes a processing component 1922, which further includes one or more processors, and a memory resource represented by a memory 1932 for storing instructions executable by the processing component 1922, such as an application. The application stored in the memory 1932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 1922 is configured to execute instructions to perform the above method.

[0163] The electronic device 1900 may also include a power supply component 1926 configured to perform power management of the electronic device 1900, a wired or wireless network interface 1950 configured to connect the electronic device 1900 to a network, and an input / output (I / O) interface 1958. The electronic device 1900 may operate based on an operating system stored in the memory 1932, such as a Microsoft Server operating system (Windows Server 2003). TM ), a graphical user interface operating system launched by Apple (Mac OSX TM ), a multi-user, multi-process computer operating system (Unix TM ), a free and open source Unix-like operating system (Linux TM ), an open source Unix-like operating system (FreeBSD TM ) or similar.

[0164] In an exemplary embodiment, a non-volatile computer-readable storage medium is also provided, such as a memory 1932 including computer program instructions, which can be executed by the processing component 1922 of the electronic device 1900 to perform the above method.

[0165] The present disclosure may be a system, a method and / or a computer program product. The computer program product may include a computer-readable storage medium carrying computer-readable program instructions for causing a processor to implement various aspects of the present disclosure.

[0166] Computer readable storage medium can be a tangible device that can hold and store instructions used by an instruction execution device. Computer readable storage medium can be, for example, (but not limited to) an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage medium include: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a static random access memory (SRAM), a portable compact disk read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanical encoding device, for example, a punch card or a convex structure in a groove on which instructions are stored, and any suitable combination thereof. The computer readable storage medium used here is not interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagated by a waveguide or other transmission medium (for example, a light pulse by an optical fiber cable), or an electrical signal transmitted by a wire.

[0167] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to each computing / processing device, or downloaded to an external computer or external storage device via a network, such as the Internet, a local area network, a wide area network, and / or a wireless network. The network can include copper transmission cables, optical fiber transmissions, wireless transmissions, routers, firewalls, switches, gateway computers, and / or edge servers. The network adapter card or network interface in each computing / processing device receives the computer-readable program instructions from the network and forwards the computer-readable program instructions for storage in the computer-readable storage medium in each computing / processing device.

[0168] The computer program instructions for performing the operation of the present disclosure may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine-related instructions, microcode, firmware instructions, state setting data, or source code or object code written in any combination of one or more programming languages, including object-oriented programming languages, such as Smalltalk, C++, etc., and conventional procedural programming languages, such as "C" language or similar programming languages. Computer-readable program instructions may be executed completely on a user's computer, partially on a user's computer, as an independent software package, partially on a user's computer, partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., using an Internet service provider to connect via the Internet). In some embodiments, an electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), may be customized by utilizing the state information of the computer-readable program instructions, and the electronic circuit may execute the computer-readable program instructions, thereby realizing various aspects of the present disclosure.

[0169] Various aspects of the present disclosure are described herein with reference to the flowcharts and / or block diagrams of the methods, devices (systems) and computer program products according to the embodiments of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer-readable program instructions.

[0170] These computer-readable program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, thereby producing a machine, so that when these instructions are executed by the processor of the computer or other programmable data processing device, a device that implements the functions / actions specified in one or more boxes in the flowchart and / or block diagram is generated. These computer-readable program instructions can also be stored in a computer-readable storage medium, and these instructions cause the computer, programmable data processing device, and / or other equipment to work in a specific manner, so that the computer-readable medium storing the instructions includes a manufactured product, which includes instructions for implementing various aspects of the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0171] Computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device so that a series of operating steps are performed on the computer, other programmable data processing apparatus, or other device to produce a computer-implemented process, thereby causing the instructions executed on the computer, other programmable data processing apparatus, or other device to implement the functions / actions specified in one or more boxes in the flowchart and / or block diagram.

[0172] The flow chart and block diagram in the accompanying drawings show the possible architecture, function and operation of the system, method and computer program product according to multiple embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a part of a module, program segment or instruction, and a part of the module, program segment or instruction includes one or more executable instructions for realizing the specified logical function. In some alternative implementations, the function marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two continuous square boxes can actually be executed substantially in parallel, and they can sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs the specified function or action, or can be implemented with a combination of special hardware and computer instructions.

[0173] The computer program product may be implemented in hardware, software or a combination thereof. In one optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK) and the like.

[0174] The embodiments of the present disclosure have been described above, and the above description is exemplary, not exhaustive, and is not limited to the disclosed embodiments. Many modifications and changes will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The selection of terms used herein is intended to best explain the principles of the embodiments, practical applications, or improvements to the technology in the market, or to enable other persons of ordinary skill in the art to understand the embodiments disclosed herein.

Claims

1. A calibration method for a visual sensor, It is characterized in that include: Determine a first coordinate value of a reference point on a calibration object in a world coordinate system, wherein the calibration object includes a second pattern obtained by perspective transformation of a first pattern, the first pattern includes a plurality of reference points arranged regularly, and a first distance between reference points in the calibration object is positively correlated with a second distance between the reference points and the visual sensor; Determine a second coordinate value of the reference point in the image captured by the visual sensor; The parameters to be calibrated of the visual sensor are determined according to the first coordinate value and the second coordinate value, wherein the first coordinate value represents the position of the reference point in the real geographic space, and the visual sensor is a visual sensor carried by a low-viewing angle robot.

2. The method according to claim 1, It is characterized in that Before determining the first coordinate value of the reference point on the calibration object in the world coordinate system, the method further includes: Determining the perspective transformation relationship between the imaging of the visual sensor and the real object; According to the perspective transformation relationship, the first pattern is perspective transformed to obtain the second pattern.

3. The method according to claim 2, It is characterized in that Determining the perspective transformation relationship between the imaging of the visual sensor and the real object includes: The perspective transformation relationship is determined according to the height of the visual sensor relative to the ground and the shooting angle.

4. The method according to claim 3, It is characterized in that Determining the perspective transformation relationship according to the height of the visual sensor relative to the ground and the shooting angle includes: Calibrate the visual sensor according to a first pattern located on the ground to obtain an initial internal parameter of the visual sensor; Obtaining initial external parameters of the visual sensor according to the height of the visual sensor relative to the ground and the shooting angle; An initial homography matrix of the visual sensor is determined by using the initial external parameters and the initial internal parameters as the perspective transformation relationship.

5. The method according to claim 4, It is characterized in that The step of performing perspective transformation on the first pattern according to the perspective transformation relationship to obtain the second pattern comprises: The first pattern is projected into a world coordinate system using the initial homography matrix, and the projected pattern is used as the second pattern.

6. The method according to any one of claims 1 to 5, It is characterized in that The determining a second coordinate value of the reference point in the image captured by the visual sensor includes: detecting the first pattern in the image; identifying a location of a reference point in the detected first pattern in the image; The coordinate value of the position is used as the second coordinate value of the reference point in the image.

7. The method according to any one of claims 1 to 5, It is characterized in that The parameters to be calibrated include a homography matrix; The step of determining the parameters to be calibrated of the visual sensor according to the first coordinate value and the second coordinate value includes: According to the correspondence between the first coordinate value and the second coordinate value of the same reference point, a homography matrix for characterizing the projection relationship between the position in the image and the position in the world coordinate system is determined.

8. A calibration device for a visual sensor, It is characterized in that include: A first coordinate value determination module, used to determine a first coordinate value of a reference point on a calibration object in a world coordinate system, wherein the calibration object includes a second pattern obtained by perspective transformation of a first pattern, the first pattern includes a plurality of reference points arranged regularly, a first distance between reference points in the calibration object is positively correlated with a second distance between the reference points and the visual sensor, the first coordinate value represents a position of the reference point in a real geographic space, and the visual sensor is a visual sensor carried by a low-viewing angle robot; A second coordinate value determination module, used to determine a second coordinate value of the reference point in the image captured by the visual sensor; A parameter determination module is used to determine the parameters to be calibrated of the visual sensor according to the first coordinate value and the second coordinate value.

9. An electronic device, It is characterized in that include: processor; a memory for storing processor-executable instructions; The processor is configured to call the instructions stored in the memory to execute the method described in any one of claims 1 to 7.

10. A computer-readable storage medium having computer program instructions stored thereon, It is characterized in that When the computer program instructions are executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images