A mobile robot positioning method, system and product

Abstract

The application discloses a mobile robot positioning method, system and product, and relates to the field of mobile robot positioning. The method comprises the following steps: determining a to-be-recognized pattern according to a 2D image captured by a camera; determining an extreme point of the to-be-recognized pattern, and determining a pattern center according to the extreme point; acquiring a horizontal view angle, a horizontal pixel resolution of the camera and a pixel equivalent of an object projection on the camera; determining an included angle formed by the pattern center and a camera focal point; acquiring laser radar data, and determining a position of the pattern center relative to the laser radar according to the included angle; determining a pose of the pattern center relative to the laser radar according to the position of the pattern center relative to the laser radar; determining a relative pose of the laser radar relative to the pattern center according to the pose; determining an absolute pose of the laser radar according to an absolute pose of the pattern; and determining an absolute pose of the mobile robot according to the pose of the laser radar relative to a base coordinate and the absolute pose of the laser radar. The application can realize accurate positioning of the mobile robot.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of mobile robot positioning, and particularly to a mobile robot positioning method, system and product. BACKGROUND

[0002] The laser radar is a laser radar system for detecting the position, speed and other characteristic quantities of a target by emitting a laser beam. The laser radar is an advanced detection method combining laser technology and modern photoelectric detection technology, and is composed of a transmitting system, a receiving system and an information processing part. The laser radar is a mainstream sensor for positioning selection. According to the principle of a 2D laser radar, a two-dimensional point cloud map of the current environment can basically be obtained. The points scanned by the laser have high accuracy depth information. The laser can obtain the 2D contour of a certain height level of an object at any time, but it cannot find a picture attached to a plane wall because the plane picture does not have contour information.

[0003] The 2D camera is usually used in the field of plane vision in visual applications, which is determined by the inherent characteristics of the 2D imaging theory because there is no depth information. Therefore, most of the mobile robots on the market use 2D cameras to scan two-dimensional codes attached to the ground, because the two-dimensional code on the ground and the mobile robot are in the same coordinate system, while the picture attached to the wall and the mobile robot are not in the same coordinate system. Therefore, the 2D camera cannot accurately position the pattern on the wall. SUMMARY

[0004] The purpose of the present application is to provide a mobile robot positioning method, system and product to solve the problem of inaccurate positioning.

[0005] To achieve the above purpose, the present application provides the following solutions.

[0006] A mobile robot positioning method, the camera focal point and the sensing center of the laser radar are on the same perpendicular line, comprising:

[0007] determining a to-be-recognized pattern according to a 2D image captured by a camera;

[0008] determining an extreme point of the to-be-recognized pattern, and determining a pattern center according to the extreme point;

[0009] obtaining a horizontal viewing angle, a horizontal pixel resolution of the camera, and a pixel equivalent of a projection of an object on the camera;

[0010] determining an included angle formed by the pattern center and the camera focal point according to the horizontal viewing angle, the horizontal pixel resolution, the pixel equivalent and an abscissa of the pattern center;

[0011] acquiring laser radar data, and determining the position of the pattern center relative to the laser radar according to the included angle;

[0012] determining the pose of the pattern center relative to the laser radar according to the position of the pattern center relative to the laser radar; the pose includes the position of the pattern center relative to the laser radar and the angle of declination of the pattern center relative to the laser radar;

[0013] determining the relative pose of the laser radar relative to the pattern center according to the pose; the relative pose includes the relative position of the laser radar relative to the pattern center and the relative angle of declination of the laser radar relative to the pattern center;

[0014] acquiring the absolute pose of the pattern center, and determining the absolute pose of the laser radar according to the relative pose and the absolute pose of the pattern center; the absolute pose includes the absolute position of the laser radar and the absolute angle of declination of the laser radar;

[0015] acquiring the pose of the laser radar relative to the base coordinate, and determining the absolute pose of the mobile robot according to the pose of the laser radar relative to the base coordinate, the relative pose and the absolute pose.

[0016] Optionally, the determining the included angle formed by the pattern center and the camera focus point according to the horizontal visual angle, the horizontal pixel resolution, the pixel equivalent and the horizontal coordinate of the pattern center specifically comprises:

[0017] determining the distance from the projection of the object on the camera to the camera focus point by using the horizontal visual angle, the horizontal pixel resolution and the pixel equivalent;

[0018] determining the distance from the projection of the pattern center on the horizontal direction to the image center by using the horizontal pixel resolution, the pixel equivalent and the horizontal coordinate of the pattern center;

[0019] determining the included angle formed by the pattern center and the camera focus point according to the distance from the projection of the object on the camera to the camera focus point and the distance from the projection of the pattern center on the horizontal direction to the image center.

[0020] Optionally, the determining the position of the pattern center relative to the laser radar according to the included angle specifically comprises:

[0021] setting a laser radar data set;

[0022] acquiring the straight-line distance from the pattern center to the laser radar;

[0023] determining a position of the pattern center relative to the lidar according to the lidar dataset, the included angle, and the straight-line distance from the pattern center to the lidar.

[0024] Optionally, the determining the pose of the pattern center relative to the lidar according to the position of the pattern center relative to the lidar specifically comprises:

[0025] selecting, from the lidar dataset, a piece of lidar data of any length with the position of the pattern center relative to the lidar as the center;

[0026] performing straight-line fitting according to the piece of lidar data to determine a fitting straight line;

[0027] obtaining a slope of the fitting straight line;

[0028] determining an angle of deflection of the pattern center relative to the lidar according to the slope; and the position of the pattern center relative to the lidar and the angle of deflection of the pattern center relative to the lidar are the pose of the pattern center relative to the lidar.

[0029] Optionally, the determining the relative pose of the lidar relative to the pattern center according to the pose specifically comprises:

[0030] determining a relative position of the lidar relative to the pattern center according to the pattern center and the angle of deflection of the pattern center relative to the lidar;

[0031] determining a relative angle of deflection of the lidar relative to the pattern center according to the angle of deflection of the pattern center relative to the lidar.

[0032] Optionally, the determining the absolute pose of the lidar according to the relative pose and the absolute pose of the pattern center specifically comprises:

[0033] setting the absolute pose of the pattern center; the absolute pose of the pattern center comprises an absolute position of the pattern center and an absolute angle of deflection of the pattern center;

[0034] determining an absolute position of the lidar according to the relative position of the lidar relative to the pattern center and the absolute position of the pattern center;

[0035] determining an absolute angle of deflection of the lidar according to the relative angle of deflection of the lidar relative to the pattern center and the absolute angle of deflection of the pattern center.

[0036] A mobile robot positioning system, a camera focal point and a sensing center of a lidar are on the same perpendicular line, comprising:

[0037] a to-be-identified pattern determination module configured to determine a to-be-identified pattern according to a 2D image captured by a camera;

[0038] a pattern center determination module configured to determine an extreme point of the to-be-identified pattern and determine a pattern center according to the extreme point;

[0039] a pixel equivalent determination module configured to obtain a horizontal view angle, a horizontal pixel resolution of the camera, and a pixel equivalent of a projection of the object on the camera;

[0040] an included angle determination module configured to determine an included angle formed by the pattern center and a focal point of the camera according to the horizontal view angle, the horizontal pixel resolution, the pixel equivalent, and an abscissa of the pattern center;

[0041] a pattern center position relative to the laser radar determination module configured to obtain laser radar data and determine a position of the pattern center relative to the laser radar according to the included angle;

[0042] a pattern center pose relative to the laser radar determination module configured to determine a pose of the pattern center relative to the laser radar according to the position of the pattern center relative to the laser radar; the pose includes the position of the pattern center relative to the laser radar and an angle of declination of the pattern center relative to the laser radar;

[0043] a relative pose of the laser radar relative to the pattern center determination module configured to determine a relative pose of the laser radar relative to the pattern center according to the pose; the relative pose includes a relative position of the laser radar relative to the pattern center and a relative angle of declination of the laser radar relative to the pattern center;

[0044] an absolute pose of the laser radar determination module configured to obtain an absolute pose of the pattern center, and determine an absolute pose of the laser radar according to the relative pose and the absolute pose of the pattern center; the absolute pose includes an absolute position of the laser radar and an absolute angle of declination of the laser radar;

[0045] an absolute pose of the mobile robot determination module configured to obtain a pose of the laser radar relative to a base coordinate, and determine an absolute pose of the mobile robot according to the pose of the laser radar relative to the base coordinate, the relative pose, and the absolute pose.

[0046] Optionally, the included angle determination module specifically includes:

[0047] a distance determination unit configured to determine a distance from a projection of the object on the camera to the focal point of the camera by using the horizontal view angle, the horizontal pixel resolution, and the pixel equivalent;

[0048] A pattern center projection distance on a horizontal direction from an image center determination unit is configured to determine the distance of the pattern center projection on the horizontal direction from the image center by using the horizontal pixel resolution, the pixel equivalent, and the horizontal coordinate of the pattern center;

[0049] An angle determination unit is configured to determine the angle formed by the pattern center and the camera focus point according to the distance from the camera focus point of the projection of the object on the camera and the distance of the pattern center projection on the horizontal direction from the image center.

[0050] An electronic device includes a memory for storing a computer program and a processor for running the computer program to make the electronic device execute the mobile robot positioning method.

[0051] A computer readable storage medium stores a computer program, and the computer program is executed by a processor to implement the mobile robot positioning method.

[0052] According to the specific embodiments of the present application, the following technical effects are achieved: the present application provides a mobile robot positioning method, system and product, the camera focus point and the sensing center of the laser radar are on the same perpendicular line, the depth information is obtained by the laser radar, the angle information is obtained by the 2D camera, the 2D laser radar and 2D camera data are fused, and accurate positioning of the mobile robot is realized. BRIEF DESCRIPTION OF DRAWINGS

[0053] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings needed in the embodiments will be briefly introduced below. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0054] Figure 1 A flowchart of the mobile robot positioning method provided by the present application;

[0055] Figure 2 A schematic diagram of a mobile robot positioning device with the camera focus point and the sensing center of the laser radar on the same perpendicular line;

[0056] Figure 3 A schematic diagram of a pattern on the plane shot by the camera;

[0057] Figure 4 A schematic diagram of obtaining surrounding contour data by 2D laser radar scanning a vertical plane;

[0058] Figure 5 A schematic diagram of the absolute position of the mobile robot;

[0059] Figure 6 a schematic diagram of a to-be-identified pattern;

[0060] Figure 7 a schematic diagram of a mobile robot positioning device for the projection of an object on a camera at a moment T. DETAILED DESCRIPTION

[0061] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments of the present application. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative work fall within the protection scope of the present application.

[0062] The object of the present application is to provide a mobile robot positioning method, system and product, which can realize accurate positioning of a mobile robot.

[0063] In order to make the above-mentioned objects, features and advantages of the present application more obvious and easy to understand, the present application will be further described in detail below with reference to the drawings and specific embodiments.

[0064] Embodiment One

[0065] Figure 1 a flowchart of the mobile robot positioning method provided by the present application, Figure 2 a schematic diagram of a device in which the focal point of a camera and the sensing center of a laser radar are on the same vertical line, as Figures 1-2 The present application provides a mobile robot positioning method, in which the focal point of a camera and the sensing center of a laser radar are on the same vertical line, comprising:

[0066] Step 101: determining a to-be-identified pattern according to a 2D image shot by a camera.

[0067] In actual application, a pattern on a plane is found by shooting with a camera, as Figure 3 The surrounding contour data is obtained by scanning a vertical plane with a 2D laser radar, as Figure 4 The relative position of the wall pattern and the mobile robot is obtained by fusing the 2D laser radar and 2D camera data, and the absolute position of the mobile robot is further obtained, as Figure 5 The arrow in Figure 5 represents the position and direction of the pattern.

[0068] The identification steps of a specific pattern in a 2D image are as follows:

[0069] 1) Using a data window of a specific size (e.g., a 3x3 matrix) to slide across the image data array, the image data covered by the data window is averaged to remove image noise; 2) Local sliding binarization of the image; 3) Connected component calculation; 4) Patch feature point calculation; 5) Filtering out patterns to be identified based on feature data, such as... Figure 6 As shown.

[0070] Step 102: Determine the extreme points of the pattern to be identified, and determine the center of the pattern based on the extreme points.

[0071] In practical applications, the pattern to be identified is located, and its extreme points are determined. Extreme points refer to the top, bottom, right, and leftmost points of the pattern. Let's assume the extreme points of the pattern are:

[0072]

[0073] Among them, (x l y l (xr, yr) is the leftmost point, (xt, yr) is the rightmost point, and (xt, yt) is the leftmost point. t (x) is the topmost point, (x) b y b Let P be the bottommost point. Let the coordinates of the center of the pattern in the photograph be: P c =(x c y c ),

[0074] Step 103: Obtain the camera's horizontal field of view, horizontal pixel resolution, and pixel equivalent of the object's projection onto the camera.

[0075] In practical applications, since LiDAR scans data in the horizontal direction, we need to determine the horizontal angle between the pattern center and the camera focus. The coordinates of the pattern in the camera image are currently known to be: P. c =(x c y c ).

[0076] Figure 7 The horizontal angle of view α can be obtained from camera parameters or calculated, and is a constant.

[0077] Let the horizontal pixel resolution of the camera be n, such as Figure 7 As shown, assume that the pixel equivalent of the projection of object T onto the camera at a certain moment is E.

[0078] Step 104: Determine the angle between the pattern center and the camera focus based on the horizontal viewing angle, the horizontal pixel resolution, the pixel equivalent, and the horizontal coordinate of the pattern center.

[0079] In practical applications, the step 104 specifically includes: determining the distance from the object projection on the camera to the camera focus point by using the horizontal angle of view, the horizontal pixel resolution, and the pixel equivalent; determining the distance from the pattern center projection to the image center in the horizontal direction by using the horizontal pixel resolution, the pixel equivalent, and the horizontal coordinate of the pattern center; and determining the angle between the pattern center and the camera focus point according to the distance from the object projection on the camera to the camera focus point and the distance from the pattern center projection to the image center in the horizontal direction.

[0080]

[0081] L2 = (nE / 2) - x c E

[0082] θ = arctan(L2 / L1)

[0083] wherein L1 is the distance from the object projection on the camera to the camera focus point, L2 is the distance from the pattern center projection to the image center in the horizontal direction, and θ is the angle between the pattern center projection and the camera focus point.

[0084] Because the camera focus point and the sensing center of the laser radar are on the same perpendicular line, the horizontal angle between the pattern center and the laser radar is equal to the angle between the pattern center and the camera focus point, i.e., θ.

[0085] Step 105: acquiring laser radar data and determining the position of the pattern center relative to the laser radar according to the angle.

[0086] In practical applications, the step 105 specifically includes: setting a laser radar data set; acquiring the straight-line distance from the pattern center to the laser radar; and determining the position of the pattern center relative to the laser radar according to the laser radar data set, the angle, and the straight-line distance from the pattern center to the laser radar.

[0087] Suppose the laser radar data at the above time T is as follows:

[0088]

[0089] In the laser radar data, angle_min is the minimum angle of the laser radar data angle, which is a constant value, angle_max is the maximum angle of the laser radar data angle, which is a constant value, angle_increment is the angle resolution of the laser radar data, and ranges is the distance data set of the obstacle to the sensing center of the laser radar at each angle.

[0090] m = (angle_max - angle_min) / angle_increment

[0091] wherein m is the number of data in ranges.

[0092] angles = (angle_min, angle_min+1*angle_increment, angle_min+2*angle_increment,.............., angle_min+(m-1)*angle_increment)

[0093]

[0094] wherein angles is the angle set of the lidar data, the angle data in angles and the distance data in ranges are one-to-one corresponding, the relative position of each point in the lidar data at time T relative to the lidar is obtained by a trigonometric function.

[0095] positions is the set of the relative position of each point in the lidar data at time T relative to the lidar.

[0096] The position P of the pattern center in the lidar data is obtained by θ, and the Pth data in positions is the horizontal position of the pattern center relative to the lidar, which is denoted by position p .

[0097] P = (θ - angle_min) / angle_increment

[0098] position p = (cos(θ)*L sin(θ)*L)

[0099] wherein position p is the position of the pattern center relative to the lidar; θ is the included angle; L is the straight-line distance from the pattern center to the lidar; L is the Pth data in the lidar data ranges, which is the straight-line distance from the pattern center to the lidar.

[0100] A piece of data with position p as the center and length w is taken out from positions, which is theoretically a straight line, and can be first judged as a straight line and then fitted as a straight line, and the piece of data is denoted by P line .

[0101] P line = ((x1 y1) (x2 y2)..........(x w-1 y w-1) (x w y w ))

[0102] The fitting equation of the straight line is obtained by using the least square method: y=kx+b.

[0103]

[0104]

[0105] wherein k is the slope of the fitting straight line with position p as the center and length w.

[0106] Step 106: determining the pose of the pattern center relative to the laser radar according to the position of the pattern center relative to the laser radar; the pose includes the position of the pattern center relative to the laser radar and the yaw angle of the pattern center relative to the laser radar.

[0107] In actual application, the step 106 specifically includes: selecting a piece of laser radar data with any length with the position of the pattern center relative to the laser radar as the center from the laser radar data set; performing straight line fitting according to the piece of laser radar data to determine a fitting straight line; obtaining the slope of the fitting straight line; determining the yaw angle of the pattern center relative to the laser radar according to the slope; and the position and the yaw angle of the pattern center relative to the laser radar are the pose of the pattern center relative to the laser radar.

[0108] Because the image is pasted on the vertical plane, the direction of the image is perpendicular to the vertical plane, and the direction angle of the image is set as σ: σ=arctan(k)+π / 2, and the direction angle σ of the image is the yaw angle of the pattern center relative to the laser radar.

[0109] In the coordinate system of the mobile robot, the pose includes a two-dimensional coordinate and a yaw angle, and the pose of the pattern center relative to the laser radar is set as positionctl: position ctl =(position p σ).

[0110] wherein position p is the position of the pattern center relative to the laser radar.

[0111] Step 107: determining the relative pose of the laser radar relative to the pattern center according to the pose; the relative pose includes the relative position of the laser radar relative to the pattern center and the relative yaw angle of the laser radar relative to the pattern center.

[0112] In practical application, the step 107 specifically includes: determining the relative position of the laser radar relative to the pattern center according to the pattern center and the deflection angle of the pattern center relative to the laser radar; and determining the relative deflection angle of the laser radar relative to the pattern center according to the deflection angle of the pattern center relative to the laser radar.

[0113] Supposing the pose of the laser radar relative to the pattern center is position ltc = ((x lp y lp ) β).

[0114]

[0115] β = -σ

[0116] wherein (x lp y lp ) is the relative position of the laser radar relative to the pattern center, and β is the relative deflection angle of the laser radar relative to the pattern center.

[0117] Step 108: obtaining the absolute pose of the pattern center, and determining the absolute pose of the laser radar according to the relative pose and the absolute pose of the pattern center; the absolute pose includes the absolute position of the laser radar and the absolute deflection angle of the laser radar.

[0118] In practical application, the step 108 specifically includes: setting the absolute pose of the pattern center; the absolute pose of the pattern center includes the absolute position of the pattern center and the absolute deflection angle of the pattern center; determining the absolute position of the laser radar according to the relative position of the laser radar relative to the pattern center and the absolute position of the pattern center; and determining the absolute deflection angle of the laser radar according to the relative deflection angle of the laser radar relative to the pattern center and the deflection angle of the pattern center.

[0119] In the present application, the absolute pose of the pattern center is known, and it is assumed that the absolute pose of the pattern center is:

[0120] position cjd = ((x cjd y cjd ) γ)

[0121] wherein (x cjd y cjd ) is the position of the laser radar relative to the pattern center, and γ is the deflection angle of the laser radar relative to the pattern center.

[0122] Supposing the absolute pose of the laser radar is: position ljd The calculation formulae of the parameters in the above equation are as follows:

[0123]

[0124] x ljd =x lj +x cjd

[0125] y ljd =y lj +y cjd

[0126]

[0127] wherein (x ljd y ljd ) is an absolute position of the lidar; is an absolute angle of the lidar; (x lj y lj ) is a difference between the absolute position of the lidar and an absolute position of the pattern center;

[0128] Step 109: obtaining a pose of the lidar relative to a base coordinate, and determining an absolute pose of the mobile robot according to the pose of the lidar relative to the base coordinate, the relative pose, and the absolute pose.

[0129] In actual applications, the principle of pose compensation is to convert the calculated absolute pose of the lidar to the base coordinate of the mobile robot. Assuming that the pose of the lidar relative to the base coordinate is ((x d y d )η), and the absolute pose of the base coordinate is position j , the absolute pose of the mobile robot is:

[0130]

[0131] wherein (x d y d ) is a position of the lidar relative to the base coordinate; η is an angle of the lidar relative to the base coordinate.

[0132] The absolute pose of the base coordinate is the absolute pose of the mobile robot.

[0133] Embodiment Two

[0134] In order to perform the method corresponding to the above-mentioned embodiment one to achieve the corresponding functions and technical effects, a mobile robot positioning system is provided below.

[0135] A mobile robot positioning system, the focal point of the camera and the sensing center of the lidar are on the same perpendicular line, comprising:

[0136] A to-be-identified pattern determination module is configured to determine a to-be-identified pattern according to a 2D image captured by a camera.

[0137] A pattern center determination module is configured to determine an extreme point of the to-be-identified pattern, and determine a pattern center according to the extreme point.

[0138] A pixel equivalent determination module is configured to obtain a horizontal view angle, a horizontal pixel resolution of the camera, and a pixel equivalent of a projection of the object on the camera.

[0139] An included angle determination module is configured to determine an included angle formed by the pattern center and a focal point of the camera according to the horizontal view angle, the horizontal pixel resolution, the pixel equivalent, and an abscissa of the pattern center.

[0140] In actual applications, the included angle determination module specifically includes: a distance determination unit for determining a distance from a projection of the object on the camera to the focal point of the camera by using the horizontal view angle, the horizontal pixel resolution, and the pixel equivalent; a distance determination unit for determining a distance from the projection of the pattern center on the horizontal direction to the image center by using the horizontal pixel resolution, the pixel equivalent, and the abscissa of the pattern center; and an included angle determination unit for determining the included angle formed by the pattern center and the focal point of the camera according to the distance from the projection of the object on the camera to the focal point of the camera and the distance from the projection of the pattern center on the horizontal direction to the image center.

[0141] A pattern center position relative to a laser radar determination module is configured to obtain laser radar data, and determine a position of the pattern center relative to the laser radar according to the included angle.

[0142] A pattern center pose relative to a laser radar determination module is configured to determine a pose of the pattern center relative to the laser radar according to the position of the pattern center relative to the laser radar; the pose includes the position of the pattern center relative to the laser radar and an angle of the pattern center relative to the laser radar.

[0143] A relative pose of a laser radar relative to a pattern center determination module is configured to determine a relative pose of the laser radar relative to the pattern center according to the pose; the relative pose includes a relative position of the laser radar relative to the pattern center and a relative angle of the laser radar relative to the pattern center.

[0144] An absolute pose of a laser radar determination module is configured to obtain an absolute pose of the pattern center, and determine an absolute pose of the laser radar according to the relative pose and the absolute pose of the pattern center; the absolute pose includes an absolute position of the laser radar and an absolute angle of the laser radar.

[0145] The absolute pose determination module of the mobile robot is configured to obtain a pose of the laser radar relative to a base coordinate, and determine an absolute pose of the mobile robot according to the pose of the laser radar relative to the base coordinate, the relative pose, and the absolute pose.

[0146] Embodiment three

[0147] The electronic device provided in the embodiment of the present application comprises a memory for storing a computer program and a processor for executing the computer program to enable the electronic device to perform the mobile robot positioning method provided in the embodiment one.

[0148] In actual application, the electronic device can be a server.

[0149] In actual application, the electronic device comprises at least one processor, a memory, a bus, and a communications interface.

[0150] The processor, the communications interface, and the memory can communicate with each other through the communications bus.

[0151] The communications interface is configured to communicate with other devices.

[0152] The processor is configured to execute a program, and specifically, the program can be the method described in the above embodiments.

[0153] Specifically, the program can comprise program codes, and the program codes comprise computer operation instructions.

[0154] The processor can be a central processing unit (CPU) or an application specific integrated circuit (ASIC) or one or more integrated circuits configured to implement the embodiments of the present application. The one or more processors included in the electronic device can be processors of the same type, such as one or more CPUs, or processors of different types, such as one or more CPUs and one or more ASICs.

[0155] The memory is configured to store the program. The memory can comprise a high-speed RAM memory and can further comprise a non-volatile memory, such as at least one disk memory.

[0156] Based on the above description of the embodiments, the present application provides a storage medium having computer program instructions stored thereon, and the computer program instructions can be executed by a processor to implement the method described in any of the embodiments.

[0157] The mobile robot positioning system provided by the embodiments of the present application exists in various forms, including but not limited to:

[0158] (1) Mobile communication device: This kind of device is characterized by having mobile communication function, and providing voice and data communication as the main target. This kind of terminal includes: smart phone (such as iPhone), multimedia phone, functional phone, and low-end phone, etc.

[0159] (2) Ultra-mobile personal computer device: This kind of device belongs to the category of personal computer, has computing and processing function, and generally has mobile Internet performance. This kind of terminal includes: PDA, MID and UMPC device, etc., such as iPad.

[0160] (3) Portable entertainment device: This kind of device can display and play multimedia content. This kind of device includes: audio and video player (such as iPod), palm game machine, electronic book, and smart toy and portable car navigation device.

[0161] (4) Other electronic devices with data interaction function.

[0162] So far, specific embodiments of the present subject matter have been described. Other embodiments are within the scope of the following claims. In some cases, the actions recited in the claims can be performed in a different order and still achieve desirable results. In addition, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve the desired results. In certain implementations, multitasking and parallel processing can be advantageous.

[0163] The systems, apparatuses, modules or units illustrated by the above embodiments can be specifically implemented by a computer chip or entity, or by a product with certain functions. A typical implementation device is a computer. Specifically, the computer may, for example, be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0164] For the sake of description, the above-described apparatus is described with various units in function for convenience. Of course, in the implementation of the present application, the functions of the units can be implemented in one or more software and / or hardware. Those skilled in the art should understand that the embodiments of the present application can be provided as a method, a system or a computer program product. Therefore, the present application can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware. In addition, the present application can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0165] The present application is described with reference to flowcharts and / or block diagrams of methods, apparatus (system) and computer program products according to embodiments of the present application. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as combinations of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor or other programmable data processing apparatus to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing apparatus generate a means for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flowcharts and / or block diagrams.

[0166] These computer program instructions can also be stored in a computer-readable memory that can direct the computer or other programmable data processing apparatus to work in a specific manner, so that the instructions stored in the computer-readable memory produce a product including an instruction apparatus, which implements the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 an apparatus that implements the functions specified in the flowcharts and / or block diagrams.

[0167] These computer program instructions can also be loaded into a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable data processing apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable data processing apparatus provide a process for implementing the functions specified in the flowcharts and / or block diagrams. Figure 1 one or more flows and / or blocks Figure 1 Figure 1 an apparatus that implements the functions specified in the flowcharts and / or block diagrams.

[0168] In a typical configuration, the computing device includes one or more processors (CPUs), input / output interfaces, network interfaces and memories.

[0169] Memory can include, without being limited to, non- transitory storage in computer-readable media, random access memory (RAM), and / or read-only memory (ROM), such as flash memory, etc. Memory is an example of computer-readable media.

[0170] Computer-readable media includes permanent and non- permanent, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, without being limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disc read-only memory (CD-ROM), digital versatile disks (DVDs) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices

[0171] digital versatile disks (DVDs) or other optical storage, magnetic cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices

[0172] or any other non-transitory medium that can be used to store information that is accessible to computing devices. According to the definitions provided herein, computer-readable media does not include transitory computer-readable media, such as modulated data signals and carrier waves.

[0173] It is also important to note that the terms "comprises", "comprising", or other variations thereof are intended to cover a non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements does not include only those elements but can include other elements not expressly listed or inherent to such process, method, article, or apparatus. An element proceeded by "comprises... a" does not, without more constraints, exclude the existence of additional identical elements in the process, method, article, or apparatus that comprises the element.

[0174] The application can be described in the general context of computer- executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform particular tasks or implement particular abstract data types. The application can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are linked through a communications network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including memory storage devices.

[0175] The various embodiments described in this specification are presented for the purpose of illustrating the principles of the present application and its best mode of operation. Each of the embodiments described in this specification has been provided for the purpose of illustration only and the various embodiments are not intended to limit the present application in any way unless otherwise specifically indicated. The same parts and / or features of the various embodiments described in this specification can be referenced using the same reference numerals for the ease of understanding of the present application.

[0176] The principles and implementations of the present application have been described above with the specific examples. The above description of the embodiments is only for the purpose of helping to understand the method of the present application and its core idea. Meanwhile, for those skilled in the art, the specific implementation and application range of the present application can be changed according to the idea of the present application. In summary, the content of the specification should not be understood as a limitation of the present application.

Claims

1. A mobile robot localization method, characterized in that, The camera focus and the LiDAR sensing center are on the same vertical line, including: The pattern to be identified is determined based on 2D images captured by the camera; Determine the extreme points of the pattern to be identified, and determine the center of the pattern based on the extreme points; Obtain the camera's horizontal field of view, horizontal pixel resolution, and the pixel equivalent of the object's projection onto the camera; The angle between the pattern center and the camera focus is determined based on the horizontal viewing angle, the horizontal pixel resolution, the pixel equivalent, and the horizontal coordinate of the pattern center. Acquire lidar data and determine the position of the pattern center relative to the lidar based on the included angle; The pose of the pattern center relative to the lidar is determined based on the position of the pattern center relative to the lidar; the pose includes the position of the pattern center relative to the lidar and the deflection angle of the pattern center relative to the lidar. The relative pose of the lidar with respect to the center of the pattern is determined based on the pose; the relative pose includes the relative position of the lidar with respect to the center of the pattern and the relative deflection angle of the lidar with respect to the center of the pattern; The absolute pose of the pattern center is obtained, and the absolute pose of the lidar is determined based on the relative pose and the absolute pose of the pattern center; the absolute pose includes the absolute position of the lidar and the absolute deflection angle of the lidar. The pose of the lidar relative to the base coordinates is obtained, and the absolute pose of the mobile robot is determined based on the pose of the lidar relative to the base coordinates, the relative pose, and the absolute pose.

2. The mobile robot positioning method according to claim 1, characterized in that, The step of determining the angle between the pattern center and the camera focus based on the horizontal viewing angle, the horizontal pixel resolution, the pixel equivalent, and the horizontal coordinate of the pattern center specifically includes: The distance from the object's projection onto the camera to the camera's focal point is determined using the horizontal viewing angle, the horizontal pixel resolution, and the pixel equivalent. The distance between the pattern center projection and the image center in the horizontal direction is determined using the horizontal pixel resolution, the pixel equivalent, and the horizontal coordinate of the pattern center. The angle between the pattern center and the camera focus is determined based on the distance between the object's projection onto the camera and the camera focus, and the distance between the pattern center's projection in the horizontal direction and the image center.

3. The mobile robot positioning method according to claim 1, characterized in that, Determining the position of the pattern center relative to the lidar based on the included angle specifically includes: Configure the LiDAR dataset; Obtain the straight-line distance from the center of the pattern to the lidar; The position of the pattern center relative to the lidar is determined based on the lidar dataset, the included angle, and the straight-line distance from the pattern center to the lidar.

4. The mobile robot positioning method according to claim 3, characterized in that, Determining the pose of the pattern center relative to the lidar based on the position of the pattern center relative to the lidar specifically includes: From the LiDAR dataset, select a segment of LiDAR data of any length, centered on the position of the pattern center relative to the LiDAR. Based on the aforementioned LiDAR data, a straight line is fitted to determine the fitted straight line; Obtain the slope of the fitted line; The deflection angle of the pattern center relative to the lidar is determined based on the slope; the position and deflection angle of the pattern center relative to the lidar constitute the pose of the pattern center relative to the lidar.

5. The mobile robot positioning method according to claim 4, characterized in that, Determining the relative pose of the lidar with respect to the pattern center based on the pose specifically includes: The relative position of the lidar to the center of the pattern is determined based on the center of the pattern and the angle of deviation of the center of the pattern relative to the lidar. The relative angle of the lidar to the center of the pattern is determined based on the angle of the pattern center relative to the lidar.

6. The mobile robot positioning method according to claim 5, characterized in that, Determining the absolute pose of the lidar based on the relative pose and the absolute pose of the pattern center specifically includes: Define the absolute pose of the pattern center; the absolute pose of the pattern center includes the absolute position of the pattern center and the absolute deflection angle of the pattern center; The absolute position of the lidar is determined based on its relative position to the center of the pattern and the absolute position of the center of the pattern. The absolute angle of the lidar is determined based on the relative angle of the lidar to the center of the pattern and the absolute angle of the center of the pattern.

7. A mobile robot positioning system, characterized in that, The camera focus and the LiDAR sensing center are on the same vertical line, including: The pattern to be identified module is used to determine the pattern to be identified based on the 2D image captured by the camera; The pattern center determination module is used to determine the extreme points of the pattern to be identified, and determine the pattern center based on the extreme points; A pixel equivalent determination module is used to obtain the camera's horizontal field of view, horizontal pixel resolution, and the pixel equivalent of the object's projection onto the camera. An angle determination module is used to determine the angle between the pattern center and the camera focus based on the horizontal viewing angle, the horizontal pixel resolution, the pixel equivalent, and the horizontal coordinate of the pattern center. The pattern center position determination module relative to the lidar is used to acquire lidar data and determine the position of the pattern center relative to the lidar based on the included angle. A pattern center pose determination module relative to a lidar is used to determine the pose of the pattern center relative to the lidar based on the position of the pattern center relative to the lidar; the pose includes the position of the pattern center relative to the lidar and the deflection angle of the pattern center relative to the lidar. A relative pose determination module for a lidar relative to the center of a pattern is used to determine the relative pose of the lidar relative to the center of the pattern based on the pose; the relative pose includes the relative position of the lidar relative to the center of the pattern and the relative deflection angle of the lidar relative to the center of the pattern; The absolute pose determination module of the lidar is used to obtain the absolute pose of the pattern center and determine the absolute pose of the lidar based on the relative pose and the absolute pose of the pattern center; the absolute pose includes the absolute position of the lidar and the absolute deflection angle of the lidar. The absolute pose determination module for the mobile robot is used to acquire the pose of the lidar relative to the base coordinates, and determine the absolute pose of the mobile robot based on the pose of the lidar relative to the base coordinates, the relative pose, and the absolute pose.

8. The mobile robot positioning system according to claim 7, characterized in that, The included angle determination module specifically includes: A distance determination unit for the projection of an object onto a camera to the camera focus is used to determine the distance from the projection of the object onto the camera to the camera focus using the horizontal viewing angle, the horizontal pixel resolution, and the pixel equivalent. The distance determination unit between the pattern center projection and the image center in the horizontal direction is used to determine the distance between the pattern center projection and the image center in the horizontal direction using the horizontal pixel resolution, the pixel equivalent, and the horizontal coordinate of the pattern center; Angle determination unit is used to determine the angle formed between the pattern center and the camera focus based on the distance from the projection of the object onto the camera to the camera focus and the distance from the projection of the pattern center onto the image center in the horizontal direction.

9. An electronic device, characterized in that, The device includes a memory and a processor, the memory being used to store a computer program, and the processor running the computer program to cause the electronic device to perform the mobile robot positioning method as described in any one of claims 1-6.

10. A computer-readable storage medium, characterized in that, It stores a computer program that, when executed by a processor, implements the mobile robot positioning method as described in any one of claims 1-6.

Images