A positioning method and device

Abstract

The present disclosure provides a positioning method and device, including: determining first relative position and posture information of the robot relative to the ground marking line based on an original image containing ground marking lines collected by the robot during driving; determining second relative position and posture information of the robot relative to the ground marking line based on synchronous positioning and mapping SLAM data collected by the robot; and performing auxiliary positioning of the robot based on the first relative position and posture information and the second relative position and posture information.

Description

Technical Field

[0001] The present disclosure relates to the field of information processing technology, and in particular to a positioning method and device. Background Art

[0002] Robots based on simultaneous localization and mapping (SLAM) can be divided into robots based on laser SLAM and robots based on visual SLAM according to different sensors. The robots can achieve real-time positioning based on the received SLAM data.

[0003] Robots based on laser SLAM are positioned through reflected laser data, and robots based on visual SLAM are positioned by identifying feature points in the captured images. When these two types of robots are positioning, the accuracy of the positioning data is easily affected by the external environment. Summary of the invention

[0004] The embodiments of the present disclosure at least provide a positioning method and device.

[0005] In a first aspect, an embodiment of the present disclosure provides a positioning method, including:

[0006] Determine first relative position information of the robot relative to the ground marking line based on an original image containing ground marking lines collected by the robot during driving;

[0007] Determine second relative position information of the robot relative to a ground marking line based on the synchronous positioning and mapping SLAM data collected by the robot;

[0008] Based on the first relative position information and the second relative position information, auxiliary positioning of the robot is performed.

[0009] In a possible implementation, determining first relative position information of the robot relative to the ground marking line based on an original image containing ground marking lines collected by the robot during driving includes:

[0010] Converting the original image into a bird's-eye view;

[0011] Acquire preset position information of the robot in the bird's-eye view and ground marking information in the bird's-eye view;

[0012] According to the preset position information of the robot in the bird's-eye view and the ground marking information in the bird's-eye view, first relative position information of the robot relative to the ground marking is determined.

[0013] In a possible implementation manner, the ground marking information includes a target straight line used to represent the ground marking;

[0014] Before determining the first relative position information of the robot relative to the ground marking line, the method further includes:

[0015] Determine the target location area corresponding to the ground marking line in the bird's-eye view based on a color detection algorithm;

[0016] Determine a target straight line corresponding to a ground marking line in the target location area;

[0017] The determining of first relative position information of the robot relative to a ground marking line includes:

[0018] Determine first relative position information of the robot relative to the target line.

[0019] In a possible implementation manner, determining a target straight line corresponding to a ground marking line in the target position area includes:

[0020] Converting a target position region image corresponding to the target position region into a binary image;

[0021] Performing edge detection on the binary image to determine a plurality of edge points, wherein the edge points are points located at the edge of the ground marking line;

[0022] Perform straight line fitting based on the multiple edge points to determine at least one straight line to be screened;

[0023] The target straight line is selected from the at least one straight line to be selected.

[0024] In a possible implementation manner, the step of selecting the target straight line from the at least one straight line to be selected includes:

[0025] For each line to be screened, determine the sum of the distances from each edge point to the line to be screened;

[0026] The straight line to be screened with the smallest sum of corresponding distances is taken as the target straight line.

[0027] In a possible implementation manner, the step of selecting the target straight line from the at least one straight line to be selected includes:

[0028] For each straight line to be screened, determine the number of edge points covered by the straight line to be screened;

[0029] The straight line to be screened with the largest number of corresponding ones is taken as the target straight line.

[0030] In a possible implementation manner, the preset position information includes coordinate information of the robot in an image coordinate system corresponding to the bird's-eye view;

[0031] The determining, based on preset position information of the robot in the bird's-eye view and ground marking information in the bird's-eye view, first relative position information of the robot relative to the ground marking line comprises:

[0032] Determine the angle information of the ground marking line in the image coordinate system corresponding to the bird's-eye view; and determine the image lateral distance between the robot and the ground marking line based on the coordinate information of the robot in the image coordinate system corresponding to the bird's-eye view;

[0033] Determine the real lateral distance between the robot and the ground marking line based on a preset ratio of the image distance to the real distance and the determined image lateral distance between the robot and the ground marking line;

[0034] The first relative position information is determined according to the angle information of the ground marking line in the image coordinate system corresponding to the bird's-eye view and the real lateral distance.

[0035] In a possible implementation, determining the second relative position information of the robot relative to the ground marking line based on the synchronous positioning and mapping SLAM data collected by the robot includes:

[0036] Determine the position and posture information of the robot in a pre-built map based on the synchronous positioning and mapping SLAM data collected by the robot;

[0037] The second relative posture information is determined based on the position information of the ground markings pre-stored in the map and the posture information of the robot in the map.

[0038] In a possible implementation manner, the auxiliary positioning of the robot based on the first relative position information and the second relative position information includes:

[0039] Determining target relative posture information based on the first relative posture information and the second relative posture information;

[0040] Based on the position information of the ground marking line and the target relative posture information, the current posture information of the robot is re-determined.

[0041] In a second aspect, an embodiment of the present disclosure further provides a positioning device, including:

[0042] A first determination module is used to determine first relative position information of the robot relative to the ground marking line based on an original image containing the ground marking line collected by the robot during the driving process;

[0043] A second determination module is used to determine second relative posture information of the robot relative to the ground marking line based on the synchronous positioning and mapping SLAM data collected by the robot;

[0044] A positioning module is used to assist in positioning the robot based on the first relative position information and the second relative position information.

[0045] In a possible implementation manner, the first determination module, when determining the first relative position information of the robot relative to the ground marking line based on the original image containing the ground marking line collected by the robot during driving, is used to:

[0046] Converting the original image into a bird's-eye view;

[0047] Acquire preset position information of the robot in the bird's-eye view and ground marking information in the bird's-eye view;

[0048] According to the preset position information of the robot in the bird's-eye view and the ground marking information in the bird's-eye view, first relative position information of the robot relative to the ground marking is determined.

[0049] In a possible implementation manner, the ground marking information includes a target straight line used to represent the ground marking;

[0050] The first determination module, before determining the first relative position information of the robot relative to the ground marking line, is further used to:

[0051] Determine the target location area corresponding to the ground marking line in the bird's-eye view based on a color detection algorithm;

[0052] Determine a target straight line corresponding to a ground marking line in the target location area;

[0053] The first determination module, when determining the first relative position information of the robot relative to the ground marking line, is used to:

[0054] Determine first relative position information of the robot relative to the target line.

[0055] In a possible implementation manner, the first determination module, when determining the target straight line corresponding to the ground marking line in the target location area, is used to:

[0056] Converting a target position region image corresponding to the target position region into a binary image;

[0057] Performing edge detection on the binary image to determine a plurality of edge points, wherein the edge points are points located at the edge of the ground marking line;

[0058] Perform straight line fitting based on the multiple edge points to determine at least one straight line to be screened;

[0059] The target straight line is selected from the at least one straight line to be selected.

[0060] In a possible implementation manner, when the first determining module selects the target straight line from the at least one straight line to be selected, it is configured to:

[0061] For each line to be screened, determine the sum of the distances from each edge point to the line to be screened;

[0062] The straight line to be screened with the smallest sum of corresponding distances is taken as the target straight line.

[0063] In a possible implementation manner, when the first determining module selects the target straight line from the at least one straight line to be selected, it is configured to:

[0064] For each straight line to be screened, determine the number of edge points covered by the straight line to be screened;

[0065] The straight line to be screened with the largest number of corresponding ones is taken as the target straight line.

[0066] In a possible implementation manner, the preset position information includes coordinate information of the robot in an image coordinate system corresponding to the bird's-eye view;

[0067] The first determination module, when determining first relative position information of the robot relative to a ground marking line according to preset position information of the robot in the bird's-eye view and ground marking line information in the bird's-eye view, is used to:

[0068] Determine the angle information of the ground marking line in the image coordinate system corresponding to the bird's-eye view; and determine the image lateral distance between the robot and the ground marking line based on the coordinate information of the robot in the image coordinate system corresponding to the bird's-eye view;

[0069] Determine the real lateral distance between the robot and the ground marking line based on a preset ratio of the image distance to the real distance and the determined image lateral distance between the robot and the ground marking line;

[0070] The first relative position information is determined according to the angle information of the ground marking line in the image coordinate system corresponding to the bird's-eye view and the real lateral distance.

[0071] In a possible implementation manner, the second determination module, when determining the second relative position information of the robot relative to the ground marking line based on the synchronous positioning and mapping SLAM data collected by the robot, is used to:

[0072] Determine the position and posture information of the robot in a pre-built map based on the synchronous positioning and mapping SLAM data collected by the robot;

[0073] The second relative posture information is determined based on the position information of the ground markings pre-stored in the map and the posture information of the robot in the map.

[0074] In a possible implementation manner, the positioning module, when performing auxiliary positioning of the robot based on the first relative posture information and the second relative posture information, is used to:

[0075] Determining target relative posture information based on the first relative posture information and the second relative posture information;

[0076] Based on the position information of the ground marking line and the target relative posture information, the current posture information of the robot is re-determined.

[0077] In a third aspect, an embodiment of the present disclosure further provides a computer device, comprising: a processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the above-mentioned first aspect, or any possible implementation of the first aspect are performed.

[0078] In a fourth aspect, an embodiment of the present disclosure further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned first aspect or any possible implementation of the first aspect are executed.

[0079] The positioning method and device provided in the embodiments of the present disclosure can utilize the ground markings during the robot's driving process to determine the first relative posture information of the robot relative to the ground markings based on the collected original images, and determine the second relative posture information of the robot relative to the ground markings based on the SLAM data, and then assist in positioning the robot based on the first relative posture information and the second relative posture information. In this way, the robot can be prevented from being affected by the external environment during the positioning process, thereby improving the positioning accuracy of the robot.

[0080] In order to make the above-mentioned objectives, features and advantages of the present disclosure more obvious and easy to understand, preferred embodiments are specifically cited below and described in detail with reference to the attached drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] In order to more clearly illustrate the technical solutions of the embodiments of the present disclosure, the following is a brief introduction to the drawings required for use in the embodiments. The drawings herein are incorporated into the specification and constitute a part of the specification. These drawings illustrate embodiments consistent with the present disclosure and are used together with the specification to illustrate the technical solutions of the present disclosure. It should be understood that the following drawings only illustrate certain embodiments of the present disclosure and should not be regarded as limiting the scope. For ordinary technicians in this field, other relevant drawings can also be obtained based on these drawings without creative work.

[0082] Figure 1 A flowchart of a positioning method provided by an embodiment of the present disclosure is shown;

[0083] Figure 2 A flowchart of a first relative posture information determination method provided by an embodiment of the present disclosure is shown;

[0084] Figure 3 A schematic diagram of the architecture of a positioning device provided by an embodiment of the present disclosure is shown;

[0085] Figure 4 A schematic structural diagram of a computer device 400 provided in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0086] In order to make the purpose, technical scheme and advantages of the embodiments of the present disclosure clearer, the technical scheme in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are only part of the embodiments of the present disclosure, rather than all of the embodiments. The components of the embodiments of the present disclosure generally described and shown in the drawings here can be arranged and designed in various different configurations. Therefore, the following detailed description of the embodiments of the present disclosure provided in the drawings is not intended to limit the scope of the present disclosure for protection, but merely represents the selected embodiments of the present disclosure. Based on the embodiments of the present disclosure, all other embodiments obtained by those skilled in the art without making creative work belong to the scope of protection of the present disclosure.

[0087] The robot based on laser SLAM is positioned by the reflected laser data. However, when the robot based on laser SLAM is located in a wide area, the reflected laser data signal is weak, and the positioning data of the robot based on laser SLAM will be affected. The robot based on visual SLAM is positioned by identifying the feature points in the captured image. However, when the robot based on visual SLAM is blocked by obstacles or the light is weak, the accuracy of the captured image will be affected, which in turn affects the positioning data. Therefore, the positioning data of these two robots will be affected by the external environment.

[0088] Based on this, the present disclosure provides a positioning method and device, which can utilize the ground markings during the robot's driving process, determine the first relative posture information of the robot relative to the ground markings based on the collected original images, and determine the second relative posture information of the robot relative to the ground markings based on SLAM data, and then assist in positioning the robot based on the first relative posture information and the second relative posture information. In this way, the robot can be prevented from being affected by the external environment during the positioning process, thereby improving the positioning accuracy of the robot.

[0089] The defects existing in the above solutions are the results obtained by the inventor after practice and careful research. Therefore, the discovery process of the above problems and the solutions proposed by the present disclosure for the above problems below should be the contributions made by the inventor to the present disclosure during the disclosure process.

[0090] It should be noted that similar reference numerals and letters denote similar items in the following drawings, and therefore, once an item is defined in one drawing, further definition and explanation thereof is not required in subsequent drawings.

[0091] To facilitate understanding of this embodiment, a positioning method disclosed in the embodiment of the present disclosure is first introduced in detail. The executor of the positioning method provided in the embodiment of the present disclosure is generally a robot, and the robot is equipped with an image acquisition device, such as a camera. In a possible implementation, the robot may also be equipped with a laser radar device.

[0092] See also Figure 1 As shown, it is a flowchart of a positioning method provided by an embodiment of the present disclosure, and the method includes steps 101 to 103, specifically:

[0093] Step 101: Determine first relative position information of the robot relative to the ground marking line based on an original image containing ground marking lines collected during the robot's driving.

[0094] Step 102: Determine second relative position information of the robot relative to the ground marking based on the synchronous positioning and mapping SLAM data collected by the robot.

[0095] Step 103: Based on the first relative position information and the second relative position information, assist in positioning the robot.

[0096] The following is a detailed description of the above steps 101 to 103.

[0097] For step 101:

[0098] In a specific implementation, the robot may be equipped with a camera, which may be an ordinary camera or a fisheye camera. When the camera for capturing the original image is a fisheye camera, after acquiring the original image captured by the fisheye camera, the original image may be subjected to distortion correction processing.

[0099] When the robot collects original images during driving, it can collect them once every preset time period. The collection frequency can be adjusted according to the driving speed of the robot. When the robot drives faster, the collection frequency can be increased and the preset time period can be set shorter. When the robot drives slower, the collection frequency can be lowered and the preset time period can be set longer. The specific preset time period can be set manually.

[0100] The first relative position information may be the lateral distance and angle of the robot relative to the ground marking line. In a possible implementation, when determining the first relative position information of the robot relative to the ground marking line based on the original image, the following may be referred to: Figure 2 The first relative posture information determination method shown includes the following steps:

[0101] Step 201: Convert the original image into a bird's-eye view.

[0102] Since the robot takes the original image at a certain tilt angle, the pixel distance between the two objects in the original image is not the real pixel distance. For example, if the two objects are far apart, but when the image is taken at a certain tilt angle, the two objects may overlap, which makes it impossible to correctly detect the distance between the two objects based on the captured image. Therefore, the original image needs to be converted into a bird's-eye view first.

[0103] In a specific implementation, the original image may be converted into a bird's-eye view by, but not limited to, an inverse perspective transformation method.

[0104] Step 202: Obtain preset position information of the robot in the bird's-eye view and ground marking information in the bird's-eye view.

[0105] The preset position information of the robot in the bird's-eye view includes the coordinate information of the robot in the image coordinate system corresponding to the bird's-eye view. In actual applications, since the position of the robot's camera is fixed, the original image taken by the same camera, after the original image is converted into a bird's-eye view, the coordinates of the camera in the image coordinate system of the bird's-eye view corresponding to different original images are the same. Therefore, the coordinate information of the robot in the image coordinate system corresponding to the bird's-eye view can be determined in advance.

[0106] In a specific implementation, when determining the coordinate information of the robot in the image coordinate system corresponding to the bird's-eye view, the robot's camera can be calibrated first to obtain the camera's intrinsic parameters, extrinsic parameters, and distortion parameters, etc., and then the camera can be used to capture an image containing a calibration plate, and then the robot's coordinate information can be determined based on the calibration information of the calibration plate and the camera's intrinsic parameters, extrinsic parameters, and distortion parameters.

[0107] It should be noted here that, since the camera is installed on the robot, and in the image coordinate system, the robot and the camera are the same point, the coordinate information of the robot and the coordinate information of the camera can be the same.

[0108] The ground marking line information in the bird's-eye view includes a target straight line for representing the ground marking line. Therefore, the first relative posture of the robot relative to the ground marking line is also the first relative posture of the robot relative to the target straight line.

[0109] In a possible implementation, before determining the first relative posture of the robot relative to the ground markings, the target position area corresponding to the ground markings in the bird's-eye view can be determined based on a color detection algorithm, and then the target straight line corresponding to the ground markings in the target position area can be determined. When determining the first relative posture information of the robot relative to the ground markings, the first relative posture information of the robot relative to the target straight line can be determined.

[0110] Among them, the color detection algorithm includes but is not limited to the Hue, Saturation, Value (HSV) model. In practical applications, the color to be detected can be set in advance. For example, if the ground marking is yellow, the color to be detected is set to yellow. The color detection algorithm can detect the yellow area in the bird's-eye view, which is the target position area where the ground marking is located.

[0111] When determining the target straight line corresponding to the ground marking in the target position area, the target position area image corresponding to the target position area can be first converted into a binary image, and then edge detection is performed on the binary image to determine multiple edge points, and then straight line fitting is performed based on the multiple edge points to determine at least one straight line to be screened, and then the target straight line is screened out from the at least one straight line to be screened. The edge point is a point located at the edge of the ground marking.

[0112] In actual applications, the ground markings included in the target position area image may include multiple edges. When determining multiple edge points, the edge point on the edge closest to the robot can be determined. For example, if the ground marking is located on the left side of the robot, the edge closest to the robot is the edge on the right side of the ground marking. If the ground marking is located on the right side of the robot, the edge closest to the robot is the edge on the left side of the ground marking.

[0113] When edge detection is performed on a binary image, exemplary methods that may be used include but are not limited to Sobel operator, Laplacian operator, Canny operator, Roberts Cross operator, Prewitt operator, Kirsch operator, compass operator, etc. to perform edge detection on the binary image.

[0114] When straight line fitting is performed based on multiple edge points, the Hough line detection algorithm, the line segment detection algorithm (a Line Segment Detector, LSD), etc. can be used as an example. For other methods that can detect straight lines, the method provided in the present disclosure is also applicable, and the present disclosure does not limit this.

[0115] Since multiple straight lines may be fitted when performing straight line fitting based on edge points, it is necessary to select the target straight line from at least one straight line to be selected that is fitted. Specifically, any of the following methods may be used:

[0116] Method 1: for each straight line to be screened, determine the sum of the distances from each edge point to the straight line to be screened; and take the straight line to be screened with the smallest sum of corresponding distances as the target straight line.

[0117] Method 2: for each straight line to be screened, determine the number of edge points covered by the straight line to be screened; and use the straight line to be screened with the largest number of corresponding edge points as the target straight line.

[0118] The edge points covered by the straight line to be screened are edge points passed by the straight line to be detected, or the edge points are components of the straight line to be screened.

[0119] Step 203: Determine first relative position information of the robot relative to the ground marking line according to the preset position information of the robot in the bird's-eye view and the ground marking line information in the bird's-eye view.

[0120] In a specific implementation, an image coordinate system can be first established in the bird's-eye view. The image coordinate system is a two-dimensional coordinate system. For example, the upper left corner of the image can be used as the coordinate origin, and the two sides of the bird's-eye view adjacent to the upper left corner can be used as the x-axis and y-axis respectively.

[0121] When determining the first relative posture information of the robot relative to the ground marking line based on the preset position information of the robot in the bird's-eye view and the ground marking line information in the bird's-eye view, the angle information of the ground marking line in the image coordinate system corresponding to the bird's-eye view can be first determined, and, based on the coordinate information of the robot in the image coordinate system corresponding to the robot in the bird's-eye view, the image lateral distance between the robot and the ground marking line can be determined; then, based on the ratio of the preset image distance to the actual distance and the determined image lateral distance between the robot and the ground marking line, the actual lateral distance between the robot and the ground marking line can be determined, and then the first relative posture information can be determined according to the angle information of the ground marking line in the image coordinate system corresponding to the bird's-eye view and the actual lateral distance.

[0122] In a specific implementation, after determining the target straight line in the bird's-eye view, the equation corresponding to the target straight line in the image coordinate system corresponding to the bird's-eye view can be determined, which can be exemplarily expressed in the form of y=ax+b, and then the inclination angle θ of the target straight line in the image coordinate system can be determined. When determining the image lateral distance between the robot and the ground marking line, since the coordinate information of the robot in the image coordinate system is known (pre-set), the method corresponding to the target straight line is also known, so the image lateral distance between the robot and the ground marking line can be calculated by the method of calculating the distance from a point to a straight line.

[0123] When determining the first relative posture information based on the angle information of the ground marking line in the image coordinate system corresponding to the bird's-eye view and the actual lateral distance, the relative angle information of the robot relative to the ground marking line can be determined based on the preset angle information of the robot in the image coordinate system and the angle information of the ground marking line in the image coordinate system corresponding to the bird's-eye view, and then the relative angle information and the actual lateral distance are determined as the first relative posture information.

[0124] Exemplarily, if the inclination angle of the target straight line in the image coordinate system is θ1 (i.e., the angle between the target straight line and the positive direction of the x-axis of the image coordinate system is θ1), and the preset angle information of the robot in the image coordinate system is θ2, then the relative angle information is θ1-θ2.

[0125] Regarding step 102:

[0126] The simultaneous positioning and mapping SLAM data collected by the robot may include laser SLAM data and visual SLAM data collected by the robot. After obtaining the SLAM data collected by the robot, the geographical location of the robot in the pre-constructed map can be determined. The pre-constructed map also stores ground markings and the position of the target straight line corresponding to the ground markings. Here, the target straight line is a virtual straight line in the map.

[0127] After determining the geographic location of the robot in the pre-constructed map and the position of the target line in the map, the second relative posture information of the robot relative to the target line can be determined, wherein the method for determining the second relative posture information is the same as the method for determining the first relative posture information, which will not be repeated here. However, the difference is that when calculating the lateral distance in the second relative posture, since the ratio of the map to the real world is 1:1 when constructing the map, the lateral distance calculated here is the actual lateral distance.

[0128] Regarding step 103:

[0129] In a possible implementation, when assisting the positioning of the robot based on the first relative posture information and the second relative posture information, the target relative posture information can be first determined based on the first relative posture information and the second relative posture information, and then the robot can be assisted in positioning based on the target relative posture information and the position information of the ground markings.

[0130] Among them, when determining the target relative posture information based on the first relative posture information and the second relative posture information, the first relative posture information and the second relative posture information can be fused through the Kalman filtering algorithm to obtain the target relative posture information.

[0131] The target relative posture information includes the target relative angle information of the robot relative to the ground marking and the target real lateral distance of the robot from the target straight line. The target straight line can be used to represent the ground marking. Therefore, the target relative posture information includes the target relative angle information of the robot relative to the target straight line in the pre-constructed map and the target real lateral distance of the robot from the target straight line.

[0132] When assisting the robot in positioning based on the target relative posture information, the relative position of the robot and the target straight line can be determined according to the position information of the target straight line stored in the map, as well as the target relative angle information and the target real lateral distance. Then, the real geographical location of the robot can be determined based on the relative position, thereby realizing the correction of the robot's geographical location.

[0133] The positioning method provided by the embodiment of the present disclosure can utilize the ground markings during the robot's driving process to determine the first relative posture information of the robot relative to the ground markings based on the collected original images, and determine the second relative posture information of the robot relative to the ground markings based on the SLAM data, and then assist in positioning the robot based on the first relative posture information and the second relative posture information. In this way, the robot can be prevented from being affected by the external environment during the positioning process, thereby improving the positioning accuracy of the robot.

[0134] Those skilled in the art will appreciate that, in the above method of specific implementation, the order in which the steps are written does not imply a strict execution order and does not constitute any limitation on the implementation process. The specific execution order of the steps should be determined by their functions and possible internal logic.

[0135] Based on the same inventive concept, a positioning device corresponding to the positioning method is also provided in the embodiment of the present disclosure. Since the principle of solving the problem by the device in the embodiment of the present disclosure is similar to the above-mentioned positioning method in the embodiment of the present disclosure, the implementation of the device can refer to the implementation of the method, and the repeated parts will not be repeated.

[0136] Reference Figure 3 , which is a schematic diagram of the architecture of a positioning device provided by an embodiment of the present disclosure, the device comprises: a first determination module 301, a second determination module 302, and a positioning module 303; wherein,

[0137] A first determination module 301 is used to determine first relative position information of the robot relative to the ground marking line based on an original image containing the ground marking line collected by the robot during the driving process;

[0138] A second determination module 302 is used to determine second relative position information of the robot relative to the ground marking line based on the synchronous positioning and mapping SLAM data collected by the robot;

[0139] The positioning module 303 is used to assist in positioning the robot based on the first relative position information and the second relative position information.

[0140] In a possible implementation manner, the first determination module 301, when determining the first relative position information of the robot relative to the ground marking line based on the original image containing the ground marking line collected by the robot during driving, is used to:

[0141] Converting the original image into a bird's-eye view;

[0142] Acquire preset position information of the robot in the bird's-eye view and ground marking information in the bird's-eye view;

[0143] According to the preset position information of the robot in the bird's-eye view and the ground marking information in the bird's-eye view, first relative position information of the robot relative to the ground marking is determined.

[0144] In a possible implementation manner, the ground marking information includes a target straight line used to represent the ground marking;

[0145] The first determination module 301, before determining the first relative position information of the robot relative to the ground marking line, is further used to:

[0146] Determine the target location area corresponding to the ground marking line in the bird's-eye view based on a color detection algorithm;

[0147] Determine a target straight line corresponding to a ground marking line in the target location area;

[0148] The first determination module 301, when determining the first relative position information of the robot relative to the ground marking line, is used to:

[0149] Determine first relative position information of the robot relative to the target line.

[0150] In a possible implementation manner, the first determination module 301, when determining the target straight line corresponding to the ground marking line in the target location area, is used to:

[0151] Converting a target position region image corresponding to the target position region into a binary image;

[0152] Performing edge detection on the binary image to determine a plurality of edge points, wherein the edge points are points located at the edge of the ground marking line;

[0153] Perform straight line fitting based on the multiple edge points to determine at least one straight line to be screened;

[0154] The target straight line is selected from the at least one straight line to be selected.

[0155] In a possible implementation manner, when the first determining module 301 selects the target straight line from the at least one straight line to be selected, it is configured to:

[0156] For each line to be screened, determine the sum of the distances from each edge point to the line to be screened;

[0157] The straight line to be screened with the smallest sum of corresponding distances is taken as the target straight line.

[0158] In a possible implementation manner, when the first determining module 301 selects the target straight line from the at least one straight line to be selected, it is configured to:

[0159] For each straight line to be screened, determine the number of edge points covered by the straight line to be screened;

[0160] The straight line to be screened with the largest number of corresponding ones is taken as the target straight line.

[0161] In a possible implementation manner, the preset position information includes coordinate information of the robot in an image coordinate system corresponding to the bird's-eye view;

[0162] The first determination module 301, when determining the first relative position information of the robot relative to the ground marking line according to the preset position information of the robot in the bird's-eye view and the ground marking line information in the bird's-eye view, is used to:

[0163] Determine the angle information of the ground marking line in the image coordinate system corresponding to the bird's-eye view; and determine the image lateral distance between the robot and the ground marking line based on the coordinate information of the robot in the image coordinate system corresponding to the bird's-eye view;

[0164] Determine the real lateral distance between the robot and the ground marking line based on a preset ratio of the image distance to the real distance and the determined image lateral distance between the robot and the ground marking line;

[0165] The first relative position information is determined according to the angle information of the ground marking line in the image coordinate system corresponding to the bird's-eye view and the real lateral distance.

[0166] In a possible implementation manner, the second determination module 302, when determining the second relative position information of the robot relative to the ground marking line based on the synchronous positioning and mapping SLAM data collected by the robot, is used to:

[0167] Determine the position and posture information of the robot in a pre-built map based on the synchronous positioning and mapping SLAM data collected by the robot;

[0168] The second relative posture information is determined based on the position information of the ground markings pre-stored in the map and the posture information of the robot in the map.

[0169] In a possible implementation manner, the positioning module 303, when performing auxiliary positioning of the robot based on the first relative posture information and the second relative posture information, is used to:

[0170] Determining target relative posture information based on the first relative posture information and the second relative posture information;

[0171] Based on the position information of the ground marking line and the target relative posture information, the current posture information of the robot is re-determined.

[0172] For descriptions of the processing flow of each module in the device and the interaction flow between each module, reference may be made to the relevant descriptions in the above method embodiment, which will not be described in detail here.

[0173] Based on the same technical concept, the embodiment of the present application also provides a computer device. Figure 4As shown, it is a schematic diagram of the structure of the computer device 400 provided in the embodiment of the present application, including a processor 401, a memory 402, and a bus 403. Among them, the memory 402 is used to store execution instructions, including a memory 4021 and an external memory 4022; the memory 4021 here is also called an internal memory, which is used to temporarily store the operation data in the processor 401, and the data exchanged with the external memory 4022 such as a hard disk. The processor 401 exchanges data with the external memory 4022 through the memory 4021. When the computer device 400 is running, the processor 401 communicates with the memory 402 through the bus 403, so that the processor 401 executes the following instructions

[0174] Determine first relative position information of the robot relative to the ground marking line based on an original image containing ground marking lines collected by the robot during driving;

[0175] Determine second relative position information of the robot relative to a ground marking line based on the synchronous positioning and mapping SLAM data collected by the robot;

[0176] Based on the first relative position information and the second relative position information, auxiliary positioning of the robot is performed.

[0177] In a possible implementation manner, the instructions executed by the processor 401 include determining first relative position information of the robot relative to the ground marking line based on an original image containing the ground marking line collected by the robot during driving, including:

[0178] Converting the original image into a bird's-eye view;

[0179] Acquire preset position information of the robot in the bird's-eye view and ground marking information in the bird's-eye view;

[0180] According to the preset position information of the robot in the bird's-eye view and the ground marking information in the bird's-eye view, first relative position information of the robot relative to the ground marking is determined.

[0181] In a possible implementation manner, in the instruction executed by the processor 401, the ground marking information includes a target straight line for representing the ground marking;

[0182] Before determining the first relative position information of the robot relative to the ground marking line, the method further includes:

[0183] Determine the target location area corresponding to the ground marking line in the bird's-eye view based on a color detection algorithm;

[0184] Determine a target straight line corresponding to a ground marking line in the target location area;

[0185] The determining of first relative position information of the robot relative to a ground marking line includes:

[0186] Determine first relative position information of the robot relative to the target line.

[0187] In a possible implementation manner, the instructions executed by the processor 401 include determining the target straight line corresponding to the ground marking line in the target position area, including:

[0188] Converting a target position region image corresponding to the target position region into a binary image;

[0189] Performing edge detection on the binary image to determine a plurality of edge points, wherein the edge points are points located at the edge of the ground marking line;

[0190] Perform straight line fitting based on the multiple edge points to determine at least one straight line to be screened;

[0191] The target straight line is selected from the at least one straight line to be selected.

[0192] In a possible implementation manner, in the instruction executed by the processor 401, the step of selecting the target straight line from the at least one straight line to be selected includes:

[0193] For each line to be screened, determine the sum of the distances from each edge point to the line to be screened;

[0194] The straight line to be screened with the smallest sum of corresponding distances is taken as the target straight line.

[0195] In a possible implementation manner, in the instruction executed by the processor 401, the step of selecting the target straight line from the at least one straight line to be selected includes:

[0196] For each straight line to be screened, determine the number of edge points covered by the straight line to be screened;

[0197] The straight line to be screened with the largest number of corresponding ones is taken as the target straight line.

[0198] In a possible implementation, in the instruction executed by the processor 401, the preset position information includes coordinate information of the robot in an image coordinate system corresponding to the bird's-eye view;

[0199] The determining, based on preset position information of the robot in the bird's-eye view and ground marking information in the bird's-eye view, first relative position information of the robot relative to the ground marking line comprises:

[0200] Determine the angle information of the ground marking line in the image coordinate system corresponding to the bird's-eye view; and determine the image lateral distance between the robot and the ground marking line based on the coordinate information of the robot in the image coordinate system corresponding to the bird's-eye view;

[0201] Determine the real lateral distance between the robot and the ground marking line based on a preset ratio of the image distance to the real distance and the determined image lateral distance between the robot and the ground marking line;

[0202] The first relative position information is determined according to the angle information of the ground marking line in the image coordinate system corresponding to the bird's-eye view and the real lateral distance.

[0203] In a possible implementation, the instructions executed by the processor 401 include determining the second relative position information of the robot relative to the ground marking line based on the synchronous positioning and mapping SLAM data collected by the robot, including:

[0204] Determine the position and posture information of the robot in a pre-built map based on the synchronous positioning and mapping SLAM data collected by the robot;

[0205] The second relative posture information is determined based on the position information of the ground markings pre-stored in the map and the posture information of the robot in the map.

[0206] In a possible implementation manner, in the instructions executed by the processor 401, the auxiliary positioning of the robot based on the first relative posture information and the second relative posture information includes:

[0207] Determining target relative posture information based on the first relative posture information and the second relative posture information;

[0208] Based on the position information of the ground marking line and the target relative posture information, the current posture information of the robot is re-determined.

[0209] The present disclosure also provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the steps of the positioning method described in the above method embodiment are executed. The storage medium may be a volatile or non-volatile computer-readable storage medium.

[0210] The computer program product of the positioning method provided in the embodiments of the present disclosure includes a computer-readable storage medium storing program code, and the instructions included in the program code can be used to execute the steps of the positioning method described in the above method embodiments. For details, please refer to the above method embodiments, which will not be repeated here.

[0211] The present disclosure also provides a computer program, which implements any one of the methods of the aforementioned embodiments when executed by a processor. The computer program product can be implemented in hardware, software, or a combination thereof. In an optional embodiment, the computer program product is embodied as a computer storage medium, and in another optional embodiment, the computer program product is embodied as a software product, such as a software development kit (SDK), etc.

[0212] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, the specific working process of the system and device described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here. In the several embodiments provided in the present disclosure, it should be understood that the disclosed system, device and method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some communication interfaces, and the indirect coupling or communication connection of the device or unit can be electrical, mechanical or other forms.

[0213] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0214] In addition, each functional unit in each embodiment of the present disclosure may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0215] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present disclosure, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present disclosure. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0216] Finally, it should be noted that the above-described embodiments are only specific implementation methods of the present disclosure, which are used to illustrate the technical solutions of the present disclosure, rather than to limit them. The protection scope of the present disclosure is not limited thereto. Although the present disclosure is described in detail with reference to the above-described embodiments, ordinary technicians in the field should understand that any technician familiar with the technical field can still modify the technical solutions recorded in the above-described embodiments within the technical scope disclosed in the present disclosure, or can easily think of changes, or make equivalent replacements for some of the technical features therein; and these modifications, changes or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present disclosure, and should be included in the protection scope of the present disclosure. Therefore, the protection scope of the present disclosure should be based on the protection scope of the claims.

Claims

1. A positioning method, characterized in that: include: Determine first relative position information of the robot relative to the ground marking line based on an original image containing ground marking lines collected by the robot during driving; Determine second relative position information of the robot relative to a ground marking line based on the synchronous positioning and mapping SLAM data collected by the robot; Based on the first relative position information and the second relative position information, the robot is assisted in positioning; The determining of the first relative position information of the robot relative to the ground marking line based on the original image containing the ground marking line collected by the robot during the driving process includes: Converting the original image into a bird's-eye view; Acquire preset position information of the robot in the bird's-eye view and ground marking information in the bird's-eye view; According to the preset position information of the robot in the bird's-eye view and the ground marking information in the bird's-eye view, the first relative posture information of the robot relative to the ground marking is determined; wherein the first relative posture information includes the relative angle information and the actual lateral distance of the robot relative to the ground marking.

2. The method according to claim 1, characterized in that The ground marking information includes a target straight line used to represent the ground marking; Before determining the first relative position information of the robot relative to the ground marking line, the method further includes: Determine the target location area corresponding to the ground marking line in the bird's-eye view based on a color detection algorithm; Determine a target straight line corresponding to a ground marking line in the target location area; The determining of first relative position information of the robot relative to a ground marking line includes: Determine first relative position information of the robot relative to the target line.

3. The method according to claim 2, characterized in that Determining a target straight line corresponding to a ground marking line in the target location area includes: Converting a target position region image corresponding to the target position region into a binary image; Performing edge detection on the binary image to determine a plurality of edge points, wherein the edge points are points located at the edge of the ground marking line; Perform straight line fitting based on the multiple edge points to determine at least one straight line to be screened; The target straight line is selected from the at least one straight line to be selected.

4. The method according to claim 3, characterized in that The step of selecting the target straight line from the at least one straight line to be selected includes: For each line to be screened, determine the sum of the distances from each edge point to the line to be screened; The straight line to be screened with the smallest sum of corresponding distances is taken as the target straight line.

5. The method according to claim 3, characterized in that: The step of selecting the target straight line from the at least one straight line to be selected includes: For each straight line to be screened, determine the number of edge points covered by the straight line to be screened; The straight line to be screened with the largest number of corresponding ones is taken as the target straight line.

6. The method according to claim 2, characterized in that The determining, based on preset position information of the robot in the bird's-eye view and ground marking information in the bird's-eye view, first relative position information of the robot relative to the ground marking line comprises: Determine the angle information of the ground marking line in the image coordinate system corresponding to the bird's-eye view; and determine the image lateral distance between the robot and the ground marking line based on the coordinate information of the robot in the image coordinate system corresponding to the bird's-eye view; Determine the real lateral distance between the robot and the ground marking line based on a preset ratio of the image distance to the real distance and the determined image lateral distance between the robot and the ground marking line; The first relative position information is determined according to the angle information of the ground marking line in the image coordinate system corresponding to the bird's-eye view and the real lateral distance.

7. The method according to claim 1, characterized in that Determining second relative position information of the robot relative to a ground marking line based on the synchronous positioning and mapping SLAM data collected by the robot includes: Determine the position and posture information of the robot in a pre-built map based on the synchronous positioning and mapping SLAM data collected by the robot; The second relative posture information is determined based on the position information of the ground markings pre-stored in the map and the posture information of the robot in the map.

8. The method according to claim 1, characterized in that: The auxiliary positioning of the robot based on the first relative position information and the second relative position information includes: Determining target relative posture information based on the first relative posture information and the second relative posture information; Based on the position information of the ground marking line and the target relative posture information, the current posture information of the robot is re-determined.

9. A positioning device, characterized in that: include: A first determination module is used to determine first relative position information of the robot relative to the ground marking line based on an original image containing the ground marking line collected by the robot during the driving process; A second determination module is used to determine second relative position information of the robot relative to the ground marking line based on the synchronous positioning and mapping SLAM data collected by the robot; A positioning module, used for assisting the positioning of the robot based on the first relative position information and the second relative position information; Wherein, the first determining module is used to: Converting the original image into a bird's-eye view; Acquire preset position information of the robot in the bird's-eye view and ground marking information in the bird's-eye view; According to the preset position information of the robot in the bird's-eye view and the ground marking information in the bird's-eye view, the first relative posture information of the robot relative to the ground marking is determined; wherein the first relative posture information includes the relative angle information and the actual lateral distance of the robot relative to the ground marking.

10. A computer device, characterized in that: include: A processor, a memory and a bus, wherein the memory stores machine-readable instructions executable by the processor, and when the computer device is running, the processor and the memory communicate via the bus, and when the machine-readable instructions are executed by the processor, the steps of the positioning method as described in any one of claims 1 to 8 are performed.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the positioning method according to any one of claims 1 to 8 are executed.

Images