Robot pose localization methods, devices, cleaning robots, media and products

Abstract

This application relates to a robot pose localization method, apparatus, cleaning robot, medium, and product. The method includes: acquiring an image of tile seams captured by the robot in an initial pose; determining reference tile seam segments in a tile seam map that match the position information of each first tile seam segment based on at least one first tile seam segment in the tile seam image; and iteratively optimizing the robot's initial pose based on the position information of each first tile seam segment and each reference tile seam segment to obtain the robot's pose localization result. In this method, the entire process relies on a tile seam map including the position information of the tile seam segments to perform robot pose localization, which can accurately adapt to the characteristics of tile seam cleaning scenarios, improving the robot's localization accuracy in tile seam cleaning scenarios to meet the refined requirements of tile seam cleaning operations.

Description

Technical Field

[0001] This application relates to the field of robotics technology, and in particular to robot pose localization methods, devices, cleaning robots, media, and products. Background Technology

[0002] With the development of technology, robots are being used more and more in people's lives and production.

[0003] Taking cleaning robots as an example, in related technologies, cleaning robots usually rely on point cloud grid maps or visual feature maps to locate their own pose in order to perform cleaning tasks.

[0004] However, the positioning accuracy of the robots in the relevant technologies is poor, which cannot meet the positioning requirements in the scenario of cleaning the gaps between floor tiles. Summary of the Invention

[0005] Therefore, it is necessary to provide a robot pose localization method, device, cleaning robot, medium, and product to address the aforementioned technical problems.

[0006] Firstly, this application provides a robot pose localization method, including:

[0007] Acquire images of the tile seams captured by the robot in its initial pose;

[0008] Based on at least one first tile joint line segment in the tile joint image, determine a reference tile joint line segment in the tile joint map that matches the position information of each first tile joint line segment.

[0009] Based on the position information of each first tile seam segment and each reference tile seam segment, the robot's initial pose is iteratively optimized to obtain the robot's pose localization result.

[0010] In one embodiment, determining reference tile joint segments in a tile joint map that match the positional information of each first tile joint segment based on at least one first tile joint segment in a tile joint image includes:

[0011] For each first tile joint line segment in the tile joint image, obtain at least one of the endpoint position coordinates and line vector parameters of the first tile joint line segment in the world coordinate system, as the position information of the first tile joint line segment;

[0012] Determine the reference tile joint line in the tile joint map based on the location information of the first tile joint line.

[0013] In one embodiment, obtaining the endpoint coordinates and line vector parameters of the first tile seam segment in the tile seam image in the world coordinate system includes:

[0014] Obtain the tile joint pixels in the tile joint image, and determine the first tile joint line segment based on the tile joint pixels in the tile joint image;

[0015] Transform the tile seam pixels in the first tile seam segment to the world coordinate system, fit the straight line vector parameters of the first tile seam segment, and obtain the coordinates of the endpoints of the first tile seam segment.

[0016] In one embodiment, the tile joint map includes location information of multiple second tile joint segments; determining a reference tile joint segment in the tile joint map based on the location information of the first tile joint segments includes:

[0017] Determine the target search range based on the initial pose;

[0018] Based on the location information of each second tile joint segment in the tile joint map, at least one candidate tile joint segment is determined to be located within the target search range, with at least a portion of the area being the candidate.

[0019] Based on the position information of the first tile seam segment and the position information of at least one candidate tile seam segment, a reference tile seam segment is determined from at least one candidate tile seam segment.

[0020] In one embodiment, the location information includes a line vector parameter. Based on the location information of the first tile seam segment and the location information of at least one candidate tile seam segment, determining a reference tile seam segment from the at least one candidate tile seam segment includes:

[0021] Based on the linear vector parameters of the first tile seam segment and the linear vector parameters of each candidate tile seam segment, determine the angle between the first tile seam segment and each candidate tile seam segment.

[0022] Based on the angle between the first tile seam segment and each candidate tile seam segment, a reference tile seam segment is determined from at least one candidate tile seam segment.

[0023] In one embodiment, the location information further includes endpoint location coordinates, and determining a reference tile joint segment among at least one candidate tile joint segment based on the angle between the first tile joint segment and each candidate tile joint segment, including:

[0024] If there is a pending tile joint segment among at least one candidate tile joint segment, the pending tile joint segment is determined as the reference tile joint segment; the pending tile joint segment is a candidate tile joint segment whose included angle is less than a preset included angle threshold.

[0025] If at least two undetermined tile joint segments exist among at least one candidate tile joint segment, a reference tile joint segment is determined among the at least two undetermined tile joint segments based on the endpoint coordinates of the first tile joint segment.

[0026] In one embodiment, determining a reference tile joint segment from at least two undetermined tile joint segments based on the endpoint coordinates of the first tile joint segment includes:

[0027] Based on the coordinates of the endpoints of the first tile seam segment and the linear vector parameters of each undetermined tile seam segment, determine the distances from the first tile seam segment to each undetermined tile seam segment.

[0028] Among at least two undetermined tile seam segments, determine the undetermined tile seam segment corresponding to the smallest distance that is less than a preset distance threshold, and use it as the reference tile seam segment.

[0029] In one embodiment, the robot's initial pose is iteratively optimized based on the position information of each first tile seam segment and the position information of each reference tile seam segment to obtain the robot's pose localization result, including:

[0030] Construct the pose deviation relationship between the variable pose and the initial pose, and obtain the sum expression of the distance difference between the position information of each first tile seam segment and the position information of each reference tile seam segment under the variable pose, as the distance deviation relationship;

[0031] Construct an objective function based on the pose deviation relationship and the distance deviation relationship;

[0032] The robot's pose is used as a variable to iteratively optimize the objective function, thus obtaining the robot's pose localization result.

[0033] In one embodiment, the initial pose includes a first initial pose determined based on environmental perception data and a second initial pose determined based on robot motion parameters. Constructing the pose deviation relationship between the variable pose and the initial pose includes:

[0034] Obtain the expression for the difference between the variable pose and the first initial pose, and use it as the first pose deviation relationship;

[0035] Obtain the expression for the difference between the variable pose and the second initial pose, and use it as the second pose deviation relationship.

[0036] In one embodiment, an objective function is constructed based on the pose deviation relationship and the distance deviation relationship, including:

[0037] The weights of the first pose deviation relationship, the second pose deviation relationship, and the distance deviation relationship are obtained respectively.

[0038] Based on the weights of the first pose deviation relationship, the second pose deviation relationship, and the distance deviation relationship, obtain an expression for the weighted sum of the squares of the first pose deviation relationship, the second pose deviation relationship, and the distance deviation relationship, which serves as the objective function.

[0039] In one embodiment, the robot's pose is used as a variable to iteratively optimize the objective function, resulting in the robot's pose localization result, including:

[0040] The variable pose is used as a variable to iteratively optimize the objective function, and the reference pose that meets the iteration cutoff condition is obtained.

[0041] The reference pose and the second initial pose determined based on the robot's motion parameters are fused to obtain the target pose, which is then used as the robot's pose localization result.

[0042] Secondly, this application also provides a robot pose localization device, comprising:

[0043] The image acquisition module is used to acquire images of the tile seams captured by the robot in its initial pose.

[0044] The line segment matching module is used to determine, in the tile joint map, a reference tile joint line segment that matches the position information of each first tile joint line segment based on at least one first tile joint line segment in the tile joint image.

[0045] The pose optimization module is used to iteratively optimize the robot's initial pose based on the position information of each first tile seam segment and each reference tile seam segment, so as to obtain the robot's pose localization result.

[0046] Thirdly, this application also provides a cleaning robot, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in any of the above-mentioned robot pose localization methods.

[0047] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps in any of the above-described robot pose localization methods.

[0048] Fifthly, this application also provides a computer program product, including a computer program that, when executed by a processor, implements the steps in any of the above-described robot pose localization methods.

[0049] In the aforementioned robot pose localization method, apparatus, cleaning robot, medium, and product, by acquiring an image of the tile seam captured by the robot in its initial pose, and determining reference tile seam segments in a tile seam map based on at least one first tile seam segment in the tile seam image, the robot's initial pose is iteratively optimized based on the position information of each first tile seam segment and each reference tile seam segment, thereby obtaining the robot's pose localization result. In the aforementioned method, based on at least one first tile seam segment in the tile seam image in the initial pose, a reference tile seam segment is determined in the tile seam map that matches the position information of each first tile seam segment. Then, based on the position information of each first tile seam segment and each reference tile seam segment, the robot's initial pose is iteratively optimized to obtain the robot's pose localization result. The robot obtains reference tile seam segments in the tile seam map by using the position information of the tile seam segments. Then, iteratively optimizes the initial pose using the position information of both to obtain the robot's pose localization result. The entire process relies on the tile seam map, which includes the position information of the tile seam segments, to perform robot pose localization. This can accurately adapt to the characteristics of the tile seam cleaning scenario, improve the robot's positioning accuracy in the tile seam cleaning scenario, meet the fine requirements of the tile seam cleaning operation, reduce the occurrence of missed or misaligned seams, and correspondingly improve the cleaning effect of the robot in the tile seam cleaning scenario. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the description of the embodiments of this application or related technologies will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a diagram of the internal structure of the robot in one embodiment;

[0052] Figure 2 This is a flowchart illustrating a robot pose localization method in one embodiment;

[0053] Figure 3 This is a schematic diagram illustrating a scenario in which a robot collects images of the seams between floor tiles, as shown in one embodiment.

[0054] Figure 4 This is a flowchart illustrating the process of determining a reference tile seam segment in one embodiment;

[0055] Figure 5 This is a flowchart illustrating the process of obtaining endpoint position coordinates and line vector parameters in one embodiment;

[0056] Figure 6 This is a schematic diagram showing the distribution of floor tile pixels in a floor tile seam image in one embodiment.

[0057] Figure 7 This is a flowchart illustrating the process of determining a reference tile seam segment in another embodiment;

[0058] Figure 8 This is a flowchart illustrating the process of determining a reference tile seam segment in another embodiment;

[0059] Figure 9 This is a flowchart illustrating the process of determining a reference tile seam segment in another embodiment;

[0060] Figure 10 This is a flowchart illustrating the process of determining a reference tile seam segment in another embodiment;

[0061] Figure 11 This is a flowchart illustrating the process of determining the pose localization result in one embodiment;

[0062] Figure 12 This is a flowchart illustrating the process of constructing pose deviation relationships in one embodiment;

[0063] Figure 13 This is a flowchart illustrating the process of constructing the objective function in one embodiment;

[0064] Figure 14 This is a flowchart illustrating the process of determining the pose localization result in another embodiment;

[0065] Figure 15 This is a flowchart illustrating a robot pose localization method in another embodiment;

[0066] Figure 16 This is a structural block diagram of a robot pose localization device in one embodiment. Detailed Implementation

[0067] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0068] It should be noted that the terms "first," "second," etc., used in this application can be used to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish the first element from the second element. The terms "comprising" and "having," and any variations thereof, used in this application, are intended to cover non-exclusive inclusion. The term "multiple" used in this application refers to two or more. The term "and / or" used in this application refers to one of the embodiments, or any combination of multiple embodiments.

[0069] The robot pose localization method provided in this application embodiment can be applied to, for example, Figure 1The robot shown includes a processor, memory, input / output interface, communication interface, and input device. The processor, memory, and input / output interface are connected via a system bus, and the communication interface and input device are also connected to the system bus via the input / output interface. The robot's processor provides computational and control capabilities. The robot's memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The robot's input / output interface is used for exchanging information between the processor and external devices. The robot's communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, mobile cellular networks, Near Field Communication (NFC), or other technologies. When the computer program is executed by the processor, it implements a robot pose localization method. The robot's input device can be a touch layer covering a display screen, buttons, a trackball, or a touchpad on the robot's shell, or an external keyboard, touchpad, or mouse.

[0070] It should be noted that various types of sensors can be configured on the robot to achieve the robot pose localization method described above based on data collected by these sensors. These various types of sensors include, but are not limited to, vision sensors, detection sensors, and motion sensors.

[0071] For example, the vision sensor includes a visible light camera, such as a monocular camera or a binocular camera; the detection sensor may include radar, such as lidar or millimeter-wave radar, and may also include a time-of-flight (TOF) sensor; the motion sensor may include an odometry and may also include an inertial measurement unit (IMU).

[0072] Those skilled in the art will understand that Figure 1 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0073] This application does not specify the type of robot, but it can be a cleaning robot used for cleaning tile grout, including but not limited to sweeping robots or mopping robots. Tile grout cleaning includes, but is not limited to, grout sealing cleaning.

[0074] In one exemplary embodiment, such as Figure 2As shown, a robot pose localization method is provided, which can be applied to... Figure 1 Taking the robot in the example, the following steps are included:

[0075] S210. Acquire the image of the tile seams captured by the robot in its initial pose.

[0076] The initial pose refers to the initially determined robot pose. For example, the initial pose can be a robot pose determined based on environmental perception data, or it can be a robot pose determined based on robot motion parameters. For instance, the environmental perception data can be a point cloud; the robot can match the collected point cloud with a point cloud grid map of the driving scene to obtain the initial pose. The environmental perception data can also be an image; the robot can match the collected image with a visual feature map of the driving scene to obtain the initial pose. The robot can also recursively obtain the initial pose based on robot motion parameters (such as speed and displacement information) collected by the odometry / IMU. The tile seam image is a visible light image collected by the camera mounted on the robot. For example, the tile seam image is an RGB image.

[0077] Optionally, when pose localization is required, the robot can collect environmental perception data or obtain robot motion parameters to initially determine the robot pose, which is recorded as the initial pose, and obtain the image of the tile seam captured by the camera under the initial pose.

[0078] S220. Based on at least one first tile joint line segment in the tile joint image, determine a reference tile joint line segment in the tile joint map that matches the position information of each first tile joint line segment.

[0079] The first tile seam segment is a tile seam segment in the tile seam image, and the tile seam image includes at least one first tile seam segment. The tile seam map includes the location information of multiple second tile seam segments, and the location information of each second tile seam segment is stored as a tile seam landmark in the tile seam map. The location information of the tile seam segments is used to characterize the spatial location of the tile seam segments. For example, the location information of the tile seam segments may include the spatial coordinates of at least one pixel point on the tile seam segment, such as the 3D coordinates of the endpoints and midpoints.

[0080] Optionally, after obtaining the tile seam image, the robot can identify the first tile seam segment in the tile seam image, and perform spatial transformation on the first tile seam segment based on the two-dimensional coordinates of at least one identified first tile seam segment in the tile seam image and the camera's intrinsic and extrinsic parameters. This transformation converts the first tile seam segment to the world coordinate system used to construct the tile seam map, obtaining the position information of the first tile seam segment in the tile seam image. Then, the position information of each first tile seam segment is matched with the position information of each second tile seam segment (i.e., tile seam markers) in the tile seam map to determine the second tile seam segment in the tile seam map that matches the position information of the first tile seam segment in the tile seam image. This second tile seam segment is denoted as the reference tile seam segment that matches the corresponding first tile seam segment in the tile seam image. The matched first and second tile seam segments correspond to the same tile seam segment in the scene.

[0081] For example, such as Figure 3 As shown, the tile seam image includes first tile seam segment 1, first tile seam segment 2, and first tile seam segment 3. The tile seam map includes tile seam marker A, tile seam marker B, tile seam marker C, and tile seam marker D. The robot can match the position information of first tile seam segments 1, 2, and 3 with tile seam markers A, B, C, and D, respectively. When first tile seam segment 1 successfully matches with tile seam marker A, first tile seam segment 2 successfully matches with tile seam marker B, and first tile seam segment 3 successfully matches with tile seam marker C, the second tile seam segment corresponding to tile seam marker A is used as the reference tile seam segment for first tile seam segment 1, the second tile seam segment corresponding to tile seam marker B is used as the reference tile seam segment for first tile seam segment 2, and the second tile seam segment corresponding to tile seam marker C is used as the reference tile seam segment for first tile seam segment 3.

[0082] It should be noted that the tile seam map can be a global map of the driving scene pre-built by the robot before pose localization, or a local map of the driving scene built synchronously by the robot during pose localization. If no reference tile seam segment matching the first tile seam segment in the tile seam image is identified in the tile seam map, the robot can use the position information of the first tile seam segment as a new tile seam landmark and add it to the tile seam map.

[0083] S230. Based on the position information of each first tile seam segment and each reference tile seam segment, iteratively optimize the robot's initial pose to obtain the robot's pose localization result.

[0084] Optionally, after determining the matching first tile seam segment and reference tile seam segment, the robot can construct a distance deviation relationship based on the position information of the first tile seam segment and the matching reference tile seam segment in the tile seam image. Using this distance deviation relationship as the objective function and the initial pose as the first variable pose, the robot iteratively optimizes the objective function to obtain the variable pose that meets the iteration cutoff condition, which is then used as the robot's pose localization result.

[0085] For example, the distance deviation relationship between the matching first tile seam segment and the reference tile seam segment can be characterized based on the distance difference between the matching first tile seam segment and the reference tile seam segment. The constructed distance deviation relationship is then expressed as the sum of the distance differences between each set of matching first tile seam segments and the reference tile seam segments.

[0086] For example, the iteration cutoff condition may be that the number of iterations reaches a preset iteration threshold (e.g., ≥50 times) and / or the residual change of the objective function is less than or equal to a preset change (e.g., ≤1e-6).

[0087] In this embodiment, by acquiring the tile seam image collected by the robot in its initial pose, and determining reference tile seam segments in the tile seam map that match the position information of each first tile seam segment based on at least one first tile seam segment in the tile seam image, the robot's initial pose is iteratively optimized based on the position information of each first tile seam segment and each reference tile seam segment to obtain the robot's pose localization result. In the above method, based on the position information of the first tile seam segments in the tile seam image in the initial pose, reference tile seam segments in the tile seam map whose position information matches the first tile seam segment are obtained. Then, the initial pose is iteratively optimized using the position information of both to obtain the robot's pose localization result. The entire process relies on the tile seam map including the position information of the tile seam segments to perform robot pose localization, which can accurately adapt to the characteristics of the tile seam cleaning scenario, improve the robot's positioning accuracy in the tile seam cleaning scenario, meet the refined requirements of the tile seam cleaning operation, reduce the occurrence of missed or misaligned seams, and correspondingly improve the cleaning effect of the robot in the tile seam cleaning scenario.

[0088] The location information of the tile seam segment may include at least one of the endpoint coordinates of the tile seam segment and a line vector parameter. Based on this, in one embodiment, such as Figure 4 As shown, S220 above, determining a reference tile joint line segment in the tile joint map that matches the position information of each first tile joint line segment based on at least one first tile joint line segment in the tile joint image, includes:

[0089] S410. For each first tile joint line segment in the tile joint image, obtain at least one of the endpoint position coordinates and line vector parameters of the first tile joint line segment in the world coordinate system, as the position information of the first tile joint line segment.

[0090] The world coordinate system is the three-dimensional coordinate system used to construct the tile seam map. For example, the origin of the world coordinate system is the robot pose of the first frame of the tile seam image used to construct the tile seam map. The endpoint coordinates of the tile seam segments include the spatial coordinates of the start and end points of the tile seam segments. The line vector parameters of the tile seam segments are used to characterize the orientation of the tile seam segments in space. For example, the line vector parameters of the tile seam segments include Plücker line parameters.

[0091] Optionally, the robot can use an AI model for identifying tile seams to identify tile seams in the tile seam image, obtain at least one first tile seam segment in the tile seam image, and for each obtained first tile seam segment, perform spatial transformation on the first tile seam segment according to the two-dimensional coordinates of the first tile seam segment in the tile seam image and the camera intrinsic and extrinsic parameters, so as to transform the first tile seam segment in the tile seam image to the world coordinate system, and extract the endpoint position coordinates of the first tile seam segment in the world coordinate system. It can also use a straight line fitting algorithm to fit the straight line vector parameters of the first tile seam segment according to the endpoint position coordinates, so as to use at least one of the endpoint position coordinates and the straight line vector parameters of the first tile seam segment in the world coordinate system as the position information of the first tile seam segment.

[0092] For example, the above-mentioned line fitting algorithm can be the Random Sample Consensus (RANSAC) algorithm, with the number of iterations set to 1000 and the distance threshold set to 2 pixels.

[0093] S420. Determine the reference tile joint segment in the tile joint map based on the location information of the first tile joint segment.

[0094] Optionally, after obtaining the position information of the first tile joint line segment in the tile joint image, the robot matches the position information of the first tile joint line segment with the tile joint landmarks in the tile joint map to determine the second tile joint line segment corresponding to the tile joint landmark that matches the position information of the first tile joint line segment in the tile joint image in the tile joint map, and records it as the reference tile joint line segment.

[0095] In this embodiment, for each first tile seam segment in the tile seam image, at least one of the endpoint position coordinates and line vector parameters of the first tile seam segment in the world coordinate system is obtained as the position information of the first tile seam segment. Based on the position information of the first tile seam segment, a reference tile seam segment is determined in the tile seam map. In the above method, by extracting the endpoint position coordinates and / or line vector parameters of the first tile seam segment in the image in the world coordinate system as position information, a matching reference tile seam segment in the map is determined. The endpoint position coordinates or line vector parameters can accurately reflect the spatial geometric state of the tile seam segment, thereby improving the accuracy and reliability of segment matching, weakening image texture interference, providing reliable data for subsequent robot pose localization, and improving localization accuracy.

[0096] To obtain the endpoint coordinates and line vector parameters of the tile seam segment, in one embodiment, such as Figure 5 As shown, the endpoint coordinates and line vector parameters of the first tile seam segment in the tile seam image are obtained in the world coordinate system, including:

[0097] S510. Obtain the tile joint pixels in the tile joint image, and determine the first tile joint line segment based on the tile joint pixels in the tile joint image.

[0098] Optionally, such as Figure 6 As shown, the robot can use an AI model to identify tile grout pixels in a tile grout image, or it can extract a ground area from the tile grout image, filter out the tile grout pixels from the ground area based on the pixel values ​​of each pixel in the ground area, and determine the first tile grout line segment based on the tile grout pixels in the tile grout image. For example, a set of tile grout pixels that meet preset conditions can be used as a first tile grout line segment. The preset conditions may include that the connected regions formed by the tile grout pixels are continuous, and that the length is greater than a preset length (e.g., 50mm) and the width is within a preset width range (e.g., 1-5mm).

[0099] For example, taking the identification of tile grout pixels in a tile grout image using an AI model as an example, the robot can input the tile grout image into the AI ​​model and obtain a mask image output by the AI ​​model that identifies the tile grout pixels in the tile grout image. The robot then extracts the tile grout pixels from the tile grout image in the mask image. For example, the mask image can be a binarized image in which the tile grout pixels in the tile grout image are represented by a first identifier (such as 1), and other pixels are represented by a second identifier (such as 0).

[0100] It should be noted that before determining the first tile seam segment, the robot can further filter and process the tile seam pixels in the tile seam image to determine the first tile seam segment using the filtered and processed tile seam pixels. For example, the filtering and processing includes, but is not limited to, at least one of isolated point denoising and outlier removal. For instance, a Gaussian filtering algorithm (e.g., a Gaussian kernel size of 3×3 and a standard deviation σ=1.2) is used to remove isolated tile seam pixels considered as noise in the tile seam image; the area of ​​the connected components formed by the tile seam pixels in the tile seam image is obtained, and tile seam pixels with a connected component area less than a preset area threshold are removed as outliers.

[0101] S520. Convert the tile joint pixels in the first tile joint line segment to the world coordinate system, fit the straight line vector parameters of the first tile joint line segment, and obtain the endpoint position coordinates of the first tile joint line segment.

[0102] Optionally, for each first tile seam segment in the tile seam image, the robot can perform spatial transformation based on the two-dimensional coordinates of the tile seam pixels in the first tile seam segment in the tile seam image and the camera's intrinsic and extrinsic parameters to transform the tile seam pixels in the first tile seam segment to the world coordinate system. Then, based on the spatial position coordinates of the tile seam pixels in the transformed first tile seam segment in the world coordinate system, a straight line is fitted to obtain the straight line vector parameters of the corresponding first tile seam segment. Finally, the spatial position coordinates of the starting point and ending point of the tile seam segment, i.e., the endpoint position coordinates, are extracted from the spatial position coordinates of the tile seam pixels in the transformed first tile seam segment in the world coordinate system.

[0103] For example, the process of transforming the two-dimensional coordinates of the tile seam pixels in the tile seam image to the world coordinate system is as follows: Pre-calibrate the camera intrinsic parameters (including focal length f and pixel size px), camera extrinsic parameters (including rotation matrix R and translation vector T), and camera height H; using the center point of the tile seam image as the origin, determine the vertical axis offset y (in pixels) of the tile seam pixels relative to the origin; calculate the depth d of the tile seam pixels relative to the camera lens based on the camera intrinsic parameters, camera height H, and the principle of similar triangles; obtain the two-dimensional coordinates of the tile seam pixels in the tile seam image, and combine the vertical axis offset y to determine the spatial position coordinates P of the tile seam pixels in the camera coordinate system. 相机 Based on the camera's extrinsic parameters, the P-camera coordinates of the tile seam pixels are transformed to the robot's machine coordinate system to obtain P. 机器 Determine the transformation relationship between the machine coordinate system and the world coordinate system, and then convert the P-values ​​of the tile seam pixels according to this transformation relationship. 机器 Transform to the world coordinate system. Where:

[0104] d=(f×H) / y

[0105] P 机器 =R×P 相机 +T

[0106] The robot can also use a binocular camera to capture images of the grout lines between floor tiles and calculate the depth d using parallax.

[0107] In this embodiment, by acquiring the tile seam pixels in the tile seam image, a first tile seam segment is determined based on the tile seam pixels in the tile seam image. The tile seam pixels in the first tile seam segment are then converted to the world coordinate system, and the straight line vector parameters of the first tile seam segment are fitted to obtain the endpoint coordinates. In the above method, the first tile seam segment is extracted from the tile seam pixels in the image, and then the tile seam pixels on the first tile seam segment are converted to the world coordinate system to fit the straight line vector parameters and obtain the endpoint coordinates. The straight line vector parameters and endpoint coordinates of the tile seam segment can accurately quantify the spatial pose state of the tile seam, eliminate the influence of camera viewpoint deviation, and provide reliable data support for subsequent positioning.

[0108] To improve the efficiency of determining the reference tile seam segment, in one embodiment, such as Figure 7 As shown, S420 above, determining the reference tile joint segment in the tile joint map based on the position information of the first tile joint segment, includes:

[0109] S710. Determine the target search range based on the initial pose.

[0110] The target search range is the spatial location range defined based on the robot's initial pose.

[0111] Optionally, the robot can extend a preset distance (e.g., ±5cm) along the X and Y axes of the world coordinate system from the initial pose and expand by a preset angle (e.g., ±10°), and record the resulting spatial position range as the target search range.

[0112] S720. Based on the location information of each second tile joint line segment in the tile joint map, determine at least one candidate tile joint line segment whose area is at least partially located within the target search range.

[0113] Among them, the second tile joint line segment is the tile joint line segment in the tile joint map, and the tile joint map includes the location information of multiple second tile joint line segments.

[0114] Optionally, after determining the target search range, the robot can traverse all tile joint landmarks (i.e., the position information of the second tile joint line segments) in the tile joint map to determine the positional relationship between each second tile joint line segment in the tile joint map and the target search range, and filter out at least one second tile joint line segment whose at least part of the area falls within the target search range, which is recorded as a candidate tile joint line segment.

[0115] S730. Based on the position information of the first tile joint segment and the position information of at least one candidate tile joint segment, determine the reference tile joint segment among the at least one candidate tile joint segment.

[0116] Optionally, after filtering out candidate tile seam segments in the tile seam map, the robot can match the position information of the first tile seam segment with the position information of each candidate tile seam segment to determine the candidate tile seam segment that matches the position information of the first tile seam segment in the tile seam image, i.e., the reference tile seam segment.

[0117] In this embodiment, the tile seam map includes the position information of multiple second tile seam segments. The target search range is determined based on the initial pose. At least one candidate tile seam segment is identified, with at least a portion of the area within the target search range, based on the position information of each second tile seam segment in the tile seam map. Then, based on the position information of the first tile seam segment and the position information of at least one candidate tile seam segment, a reference tile seam segment is determined from the at least one candidate tile seam segment. In this method, defining the target search range based on the initial pose to initially screen candidate tile seam segments from the tile seam map, and then matching and determining the reference tile seam segment from them, effectively narrows the matching retrieval range, reduces the amount of matching computation, and improves the efficiency of determining the reference tile seam segment.

[0118] In one embodiment, where the location information includes a line vector parameter, such as Figure 8 As shown, S730 above, determining a reference tile joint segment from at least one candidate tile joint segment based on the position information of the first tile joint segment and the position information of at least one candidate tile joint segment, includes:

[0119] S810. Determine the angle between the first tile joint segment and each candidate tile joint segment based on the straight line vector parameters of the first tile joint segment and the straight line vector parameters of each candidate tile joint segment.

[0120] Optionally, for each candidate tile seam segment, the robot can perform cosine value calculation on the straight line vector parameters of the first tile seam segment and the candidate tile seam segments, so as to deduce the included angle between the first tile seam segment and the candidate tile seam segment based on the cosine value.

[0121] S820. Based on the angle between the first tile seam segment and each candidate tile seam segment, determine a reference tile seam segment from at least one candidate tile seam segment.

[0122] Optionally, after obtaining the angle between the first tile seam segment and each candidate tile seam segment, the robot can compare the obtained angles, determine the smallest angle that is less than a preset angle threshold, and use the candidate tile seam segment with the obtained angle as a reference tile seam segment that matches the corresponding first tile seam segment.

[0123] For example, taking a tile seam image including a first tile seam segment 1, and at least one candidate tile seam segment in the determined tile seam map including candidate tile seam segments a, b, and c, the robot calculates that the included angles between the first tile seam segment 1 and the candidate tile seam segments a, b, and c are ∠a, ∠b, and ∠c respectively, and determines the smallest included angle ∠c among ∠a, ∠b, and ∠c that is less than a preset included angle threshold. Then, the candidate tile seam segment c with ∠c is used as a reference tile seam segment that matches the corresponding first tile seam segment 1.

[0124] In this embodiment, when the location information includes line vector parameters, the angle between the first tile seam segment and each candidate tile seam segment is determined based on the line vector parameters of the first tile seam segment and each candidate tile seam segment. Then, based on the angle between the first tile seam segment and each candidate tile seam segment, a reference tile seam segment is determined from at least one candidate tile seam segment. In this method, the angle between the first tile seam segment and the candidate tile seam segments is calculated using their line vector parameters, and the reference tile seam segment is selected based on this angle. This effectively reduces mismatches caused by position coordinate deviations, and the calculation of the angle based on line vector parameters is convenient and efficient, thus improving the accuracy and efficiency of line segment matching.

[0125] In one embodiment, where the location information also includes endpoint location coordinates, such as Figure 9 As shown, S820 above, determining a reference tile joint segment from at least one candidate tile joint segment based on the angle between the first tile joint segment and each candidate tile joint segment, includes:

[0126] S910. If there is a pending tile joint segment among at least one candidate tile joint segment, the pending tile joint segment is determined as a reference tile joint segment; the pending tile joint segment is a candidate tile joint segment whose included angle is less than a preset included angle threshold.

[0127] Optionally, after obtaining the angles between the first tile seam segment and each candidate tile seam segment, the robot can compare the obtained angles and determine the angles less than a preset angle threshold, which are recorded as undetermined tile seam segments. If only one undetermined tile seam segment exists among the at least one selected candidate tile seam segment, the robot can directly use the undetermined tile seam segment as a reference tile seam segment that matches the corresponding first tile seam segment.

[0128] S920. If there are at least two undetermined tile joint segments among at least one candidate tile joint segment, determine a reference tile joint segment among the at least two undetermined tile joint segments based on the endpoint position coordinates of the first tile joint segment.

[0129] Optionally, if there are at least two (i.e., only one) undetermined tile joint segments among the at least one candidate tile joint segments selected, the robot can further select a matching reference tile joint segment from the at least two undetermined tile joint segments based on the endpoint position coordinates of the first tile joint segment.

[0130] For example, the robot can determine a reference tile seam segment from at least two undetermined tile seam segments based on the endpoint coordinates of the first tile seam segment and the endpoint coordinates of each undetermined tile seam segment. For instance, for each undetermined tile seam segment, the robot can calculate the average distance between the two endpoints of the first tile seam segment and the two endpoints of the undetermined tile seam segment, and take the undetermined tile seam segment with the smallest average distance that is less than a preset distance threshold as the reference tile seam segment that matches the corresponding first tile seam segment.

[0131] In this embodiment, the location information also includes endpoint coordinates. If one undetermined tile seam segment exists among at least one candidate tile seam segment, the undetermined tile seam segment is determined as a reference tile seam segment. If at least two undetermined tile seam segments exist among at least one candidate tile seam segment, the reference tile seam segment is determined from the at least two undetermined tile seam segments based on the endpoint coordinates of the tile seam segment. Undetermined tile seam segments are candidate tile seam segments with an included angle less than a preset included angle threshold. In the above method, undetermined tile seam segments are first obtained based on the included angle for coarse matching, and then fine matching is performed by combining the endpoint coordinates. This achieves a layered filtering system combining coarse and fine matching, balancing directional consistency and positional fit. This effectively reduces misselection and omissions, improves the accuracy of reference tile seam segment selection, and enhances the algorithm's adaptability to complex tile environments.

[0132] To obtain a reference tile joint segment in the tile joint map that matches the first tile joint segment in the tile joint image, in one embodiment, such as Figure 10 As shown, in S920 above, determining a reference tile joint segment from at least two undetermined tile joint segments based on the endpoint coordinates of the first tile joint segment includes:

[0133] S1010. Based on the coordinates of the endpoints of the first tile seam segment and the linear vector parameters of each undetermined tile seam segment, determine the distances from the first tile seam segment to each undetermined tile seam segment.

[0134] Here, the distance between line segments represents the distance from a point on one line segment to another line segment. For example, the distance from the first tile seam segment to the tile seam segment to be determined can be the maximum or average of two distances from the two endpoints of the first tile seam segment to the tile seam segment to be determined.

[0135] Optionally, for each undetermined tile seam segment, the robot can calculate the maximum distance from the endpoint of the first tile seam segment to the undetermined tile seam segment based on the endpoint position coordinates of the first tile seam segment and the straight line vector parameters of the undetermined tile seam segment, and record it as the distance from the first tile seam segment to the undetermined tile seam segment.

[0136] S1020. Among at least two undetermined tile seam segments, determine the undetermined tile seam segment corresponding to the smallest distance that is less than a preset distance threshold, and use it as a reference tile seam segment.

[0137] Optionally, after obtaining the distances from the first tile seam segment to each undetermined tile seam segment, the robot can compare the distances and determine the smallest distance that is less than a preset distance threshold. The undetermined tile seam segment with the smallest distance that is less than the preset distance threshold is then used as a reference tile seam segment that matches the corresponding first tile seam segment.

[0138] It should be noted that, considering distance calculation errors, the robot can filter out undetermined tile seam segments whose corresponding distances are less than a preset distance threshold from at least two undetermined tile seam segments. If multiple segments are filtered out, the robot can obtain the conversion relationship between the first tile seam segment and each filtered undetermined tile seam segment. For each conversion relationship, the robot can convert all the first tile seam segments in the tile seam image according to the conversion relationship and determine the number of segments that successfully match the tile seam landmarks in the tile seam map after conversion. The undetermined tile seam segment with the conversion relationship that yields the most segments is taken as the reference tile seam segment that matches the aforementioned first tile seam segment.

[0139] For example, the tile seam image includes first tile seam segments 1, 2, and 3. For the first tile seam segment 1, undetermined tile seam segments A and B are identified in the tile seam map. The robot obtains the transformation relationship Fa between the first tile seam segment 1 and the undetermined tile seam segment A, and the transformation relationship Fb between the first tile seam segment 1 and the undetermined tile seam segment B. The robot transforms the first tile seam segments 1, 2, and 3 in the tile seam image according to the transformation relationship Fa. After transformation, it is determined that the first tile seam segments 1 and 2 successfully match the tile seam landmarks in the tile seam map, with a corresponding number of successfully matched segments of 2. Similarly, the robot transforms the first tile seam segments 1, 2, and 3 in the tile seam image according to the transformation relationship Fb. After transformation, it is determined that the first tile seam segments 1, 2, and 3 successfully match the tile seam landmarks in the tile seam map, with a corresponding number of successfully matched segments of 3. The robot can then determine that the undetermined tile seam segment B, corresponding to the transformation relationship Fb of the number of line segments 3, is a reference tile seam segment that matches the aforementioned first tile seam segment 1.

[0140] In this embodiment, the distances from the first tile seam segment to each undetermined tile seam segment are determined based on the endpoint coordinates of the first tile seam segment and the linear vector parameters of each undetermined tile seam segment. This allows for the identification of the undetermined tile seam segment with the smallest distance (less than a preset distance threshold) among at least two undetermined tile seam segments, which serves as a reference tile seam segment. The method combines the endpoint coordinates with the linear vector parameters of the undetermined tile seam segments to calculate the distances from the first tile seam segment to each undetermined tile seam segment. The undetermined tile seam segment with the smallest distance that meets the preset distance threshold is selected as the reference tile seam segment. By fusing the dual features of the endpoint position and vector of the segment for quantitative filtering, interference from segments at the same angle is reduced, overcoming the limitations of single matching and correspondingly improving the matching accuracy of the tile seam segments, thus enhancing positioning stability and scene robustness.

[0141] To obtain the robot's pose localization results, in one embodiment, such as Figure 11 As shown, in step S230 above, based on the position information of each first tile seam segment and each reference tile seam segment, the initial pose of the robot is iteratively optimized to obtain the robot's pose localization result, including:

[0142] S1110. Construct the pose deviation relationship between the variable pose and the initial pose, and obtain the sum expression of the distance difference between the position information of each first tile seam segment and the position information of each reference tile seam segment under the variable pose, as the distance deviation relationship.

[0143] Here, the variable pose represents the robot pose that needs to be optimized. The distance difference represents the difference in distance between the matched first tile seam segment and the reference tile seam segment under different variable poses. Under different robot poses, the positional information of the same first tile seam segment in the tile seam image acquired by the robot differs. The robot can obtain the positional information of the first tile seam segment in the tile seam image based on the variable pose transformation.

[0144] Optionally, the robot can construct a pose deviation expression between the variable pose and the initial pose, denoted as the pose deviation relationship. For each set of matched first tile seam segments and reference tile seam segments, the robot can obtain an expression for the distance difference between them based on the position information of the first tile seam segment and the position information of each reference tile seam segment. The robot can then sum the expressions for the distance differences between the matched first tile seam segments and reference tile seam segments in each set, thus obtaining the sum expression for the distance differences between the matched first tile seam segments and reference tile seam segments in each set, which serves as the distance deviation relationship.

[0145] For example, the pose deviation relationship △Pose is as follows:

[0146] △Pose=Pose_optimize-Pose_initial

[0147] Distance deviation relationship △D:

[0148] △D=

[0149] Pose_optimize represents the variable pose, Pose_initial represents the initial pose, and △d represents the distance difference.

[0150] Taking the Plücker line parameter representation as an example, the Plücker line parameter includes: direction (3dof orientation) + moment (3dof position). The distance d between the matching first tile seam segment and the reference tile seam segment can be obtained based on the following formula:

[0151] d=(pw*direction)-moment).norm()

[0152] pw=R*pb+t

[0153] Pose_optimize=(R, t)

[0154] Where pw represents the spatial coordinates of a point on the tile seam segment in the world coordinate system, pb represents the spatial coordinates of a point on the tile seam segment in the machine coordinate system, and Pose_optimize represents the optimized robot pose, i.e., the above-mentioned variable pose.

[0155] S1120. Construct the objective function based on the pose deviation relationship and the distance deviation relationship.

[0156] Optionally, the robot can obtain an expression that weights and sums the above pose deviation relationship and distance deviation relationship as the objective function.

[0157] S1130. The pose variable is used as a variable to iteratively optimize the objective function to obtain the robot's pose localization result.

[0158] Optionally, after constructing the objective function, the robot can use the initial pose as the first variable pose, iteratively optimize the objective function, and obtain the variable pose that meets the iteration cutoff condition, which is used as the robot's pose localization result.

[0159] In this embodiment, a pose deviation relationship is constructed between the variable pose and the initial pose. The sum of the distance differences between the position information of each first tile seam segment and the position information of each reference tile seam segment under the variable pose is obtained as the distance deviation relationship. An objective function is constructed based on the pose deviation relationship and the distance deviation relationship. The variable pose is used as a variable to iteratively optimize the objective function, resulting in the robot's pose localization result. In the above method, the objective function is jointly constructed by building the pose deviation relationship and the distance deviation relationship. The pose localization result is solved using the variable pose as the iterative optimization variable. This transforms the robot pose solution into a structured function optimization problem. Relying on the iterative optimization mechanism, the optimal pose is automatically approximated, improving the convergence speed and localization accuracy of the pose solution, and enhancing the stability and robustness of the pose localization result.

[0160] The initial pose includes a first initial pose determined based on environmental perception data and a second initial pose determined based on robot motion parameters. Therefore, in one embodiment, such as... Figure 12 As shown, the process of constructing the pose deviation relationship between the variable pose and the initial pose in S1110 above includes:

[0161] S1210. Obtain the expression for the difference between the variable pose and the first initial pose, as the first pose deviation relationship.

[0162] Optionally, if the environmental perception data includes point clouds collected by the robot, the robot can match the collected point clouds with a point cloud grid map of the driving scene to obtain the robot pose, denoted as the first initial pose. If the environmental perception data includes images collected by the robot, the robot can extract feature points (such as Superpoint features) from the collected images and match the extracted feature points with a visual feature map of the driving scene to obtain the robot pose, denoted as the first initial pose. If the environmental perception data includes both point clouds and images collected by the robot, the robot can determine its pose by matching the point cloud with a point cloud grid map and by matching the image with a visual feature map, and then weight the robot poses determined by the point cloud and the image respectively to obtain the first initial pose. The robot then obtains the difference expression between the variable pose and the first initial pose as the first pose deviation relationship.

[0163] S1220. Obtain the expression for the difference between the variable pose and the second initial pose, as the second pose deviation relationship.

[0164] Alternatively, the robot can also recursively derive its pose based on the robot motion parameters (such as motion speed, displacement information, etc.) collected by the odometry, and record it as the second initial pose. The difference expression between the variable pose and the second initial pose is then obtained as the second pose deviation relationship.

[0165] In this embodiment, the initial pose includes a first initial pose determined based on environmental perception data and a second initial pose determined based on robot motion parameters. The difference expression between the variable pose and the first initial pose is obtained as the first pose deviation relationship, and the difference expression between the variable pose and the second initial pose is obtained as the second pose deviation relationship. In the above method, the two types of initial poses—environmental perception and robot motion parameters—are fused, and dual pose deviation relationships are constructed to form a bidirectional constraint. This achieves complementary correction using multi-source pose information, improves the error defects of a single initial pose, suppresses perception noise and motion drift, and thus improves the robot pose positioning accuracy.

[0166] When the pose deviation relationship includes the aforementioned first pose deviation relationship and second pose deviation relationship, in one embodiment, such as Figure 13 As shown, the above S1120, which constructs the objective function based on the pose deviation relationship and the distance deviation relationship, includes:

[0167] S1310. Obtain the weights of the first pose deviation relationship, the second pose deviation relationship, and the distance deviation relationship, respectively.

[0168] Optionally, the robot can directly read the pre-set weights of the first pose deviation relationship, the second pose deviation relationship, and the distance deviation relationship. Alternatively, it can adjust the default weight of the first pose deviation relationship based on the data status information of the point cloud / image collected by the robot to obtain the adjusted weight of the first pose deviation relationship. It can also adjust the default weight of the second pose deviation relationship based on the data status information of the robot's motion parameters to obtain the adjusted weight of the second pose deviation relationship. Furthermore, it can adjust the default weight of the distance deviation relationship based on the data status information of the first tile seam line segment in the tile seam image to obtain the adjusted weight of the distance deviation relationship.

[0169] For example, the data status information of the point cloud may include whether the point cloud exhibits long corridor degradation characteristics. If the point cloud exhibits long corridor degradation characteristics, the robot may reduce the default weight of the first pose deviation relationship (e.g., from 0.3 to 0.1). The data status information of the image may include the number of feature points in the image. If the number of feature points is less than a preset threshold (e.g., 50), the robot may reduce the default weight of the first pose deviation relationship (e.g., from 0.3 to 0.1). The data status information of the first tile seam segment in the tile seam image may include the fitting error of the fitted straight line vector parameters. If the fitting error is greater than a preset error, the robot may correspondingly reduce the default weight of the distance deviation relationship (e.g., from 0.5 to 0.3).

[0170] S1320. Construct an expression for the weighted sum of the squares of the first pose deviation relationship, the second pose deviation relationship, and the distance deviation relationship based on the weights of the first pose deviation relationship, the second pose deviation relationship, and the distance deviation relationship, and use it as the objective function.

[0171] Optionally, after obtaining the weights of the first pose deviation relationship, the second pose deviation relationship, and the distance deviation relationship, the robot can perform a weighted summation of the first pose deviation relationship, the second pose deviation relationship, and the distance deviation relationship according to each weight, and obtain the corresponding expression as the objective function.

[0172] For example, the objective function F satisfies the following equation:

[0173] F = minΣ(ωi×residuali²)

[0174] Where, residuali represents the deviation relationship, and ωi represents the weight of the corresponding deviation relationship.

[0175] In this embodiment, the weights of the first pose deviation relationship, the second pose deviation relationship, and the distance deviation relationship are obtained respectively. An expression for the weighted sum of the squares of the first pose deviation relationship, the second pose deviation relationship, and the distance deviation relationship is constructed based on these weights, serving as the objective function. In the above method, weights are set for the two types of pose deviation relationships and the distance deviation relationship. The objective function is constructed by weighted summation of squares. The weight ratio of each constraint term can be adjusted as needed to achieve adaptive fusion of multi-source constraints, balance the fitting degree of constraints in different dimensions, reduce interference from abnormal data, and improve the rationality of the objective function solution and the iterative convergence effect. This can effectively improve the robot's pose localization accuracy, robustness, and scene adaptability.

[0176] To improve positioning smoothness, in one embodiment, such as Figure 14 As shown, in step S1130 above, the robot's pose is used as a variable to iteratively optimize the objective function, resulting in the robot's pose localization result, including:

[0177] S1410. Use the variable pose as a variable to iteratively optimize the objective function to obtain the reference pose that meets the iteration cutoff condition.

[0178] The reference pose represents the pose of the variable that makes the objective function reach the iteration cutoff condition, i.e., the optimal pose obtained by solving the objective function.

[0179] Optionally, the robot can use the variable pose as a variable and the initial pose as the first variable pose to iteratively optimize the objective function, obtaining the variable pose that meets the iteration cutoff condition, which is denoted as the reference pose.

[0180] For example, the iteration cutoff condition may be that the number of iterations reaches a preset iteration threshold (e.g., ≥50 times) and / or the residual change of the objective function is less than or equal to a preset change (e.g., ≤1e-6).

[0181] S1420. The reference pose and the second initial pose determined based on the robot's motion parameters are fused to obtain the target pose, which is then used as the robot's pose localization result.

[0182] Optionally, based on the reference pose obtained by optimizing the objective function, the robot can further use the Extended Kalman Filter (EKF) algorithm to fuse the reference pose with the second initial pose determined based on the robot's motion parameters to obtain the target pose, which can be used as the robot's pose localization result.

[0183] It should be noted that the reference pose obtained by iterative optimization is based on visual tile seam matching, which is easily affected by image texture and noise, resulting in instantaneous jumps and small jitters, leading to poor smoothness of the optimized reference pose. In contrast, the second initial pose obtained based on robot motion parameters is the result of continuous odometry / IMU recursion, which has the characteristics of continuous time sequence, smooth change and no abrupt changes. By fusing the reference pose and the second initial pose, a target pose with improved smoothness can be obtained.

[0184] In this embodiment, the variable pose is used as a variable to iteratively optimize the objective function to obtain a reference pose that meets the iteration cutoff condition. The reference pose and the second initial pose determined based on the robot's motion parameters are then fused to obtain the target pose, which is used as the robot's pose localization result. In the above method, by fusing the reference pose and the second initial pose, the sudden fluctuations in localization can be effectively smoothed, which helps to improve the continuity and smoothness of the localization trajectory. Consequently, the robot's pose output becomes stable and coherent, improving the smoothness of the robot's motion control and reducing problems such as localization offset, trajectory jumps, and body jamming during the motion process, thereby improving the robot's motion stability.

[0185] To facilitate understanding by those skilled in the art, the robot pose localization method provided in this application is described in detail below, such as... Figure 15 As shown, the method may include:

[0186] S1501. Acquire the image of the tile seam captured by the robot in its initial pose;

[0187] S1502. Obtain the tile joint pixels in the tile joint image, and determine at least one first tile joint line segment based on the tile joint pixels in the tile joint image.

[0188] S1503. For each first tile seam segment, the tile seam pixels in the first tile seam segment are transformed to the world coordinate system, the straight line vector parameters of the first tile seam segment are fitted, and the endpoint position coordinates of the first tile seam segment are obtained. The endpoint position coordinates and the straight line vector parameters are used as the position information of the first tile seam segment.

[0189] S1504. Determine the target search range based on the initial pose;

[0190] S1505. Based on the location information of each second tile joint line segment in the tile joint map, determine at least one candidate tile joint line segment whose at least part of the area is located within the target search range.

[0191] S1506. Determine the angle between the first tile joint segment and each candidate tile joint segment based on the straight line vector parameters of the first tile joint segment and the straight line vector parameters of each candidate tile joint segment.

[0192] S1507. If there is a pending tile joint segment among at least one candidate tile joint segment, the pending tile joint segment shall be determined as a reference tile joint segment; the pending tile joint segment is a candidate tile joint segment whose included angle is less than a preset included angle threshold.

[0193] S1508. In the case where there are at least two undetermined tile joint segments among at least one candidate tile joint segment, the distance from the first tile joint segment to each undetermined tile joint segment is determined according to the endpoint position coordinates of the first tile joint segment and the straight line vector parameters of each undetermined tile joint segment.

[0194] S1509. Among at least two undetermined tile seam segments, determine the undetermined tile seam segment corresponding to the smallest distance that is less than a preset distance threshold, and use it as a reference tile seam segment.

[0195] S1510, the initial pose includes a first initial pose determined based on environmental perception data and a second initial pose determined based on robot motion parameters; obtain the difference expression between the variable pose and the first initial pose as the first pose deviation relationship, and obtain the difference expression between the variable pose and the second initial pose as the second pose deviation relationship.

[0196] S1511. Obtain the sum expression of the distance difference between the position information of each first tile seam segment and the position information of each reference tile seam segment under the variable pose, as the distance deviation relationship;

[0197] S1512. Obtain the weights of the first pose deviation relationship, the second pose deviation relationship, and the distance deviation relationship respectively;

[0198] S1513. Based on the weights of the first pose deviation relationship, the second pose deviation relationship, and the distance deviation relationship, obtain an expression for the weighted sum of squares of the first pose deviation relationship, the second pose deviation relationship, and the distance deviation relationship, and use it as the objective function;

[0199] S1514. Use the variable pose as a variable to iteratively optimize the objective function to obtain the reference pose that meets the iteration cutoff condition.

[0200] S1515. The reference pose and the second initial pose are fused to obtain the target pose, which is used as the pose localization result of the robot.

[0201] It should be noted that the descriptions in S1501-S1515 above can be found in the relevant descriptions in the above embodiments, and their effects are similar, so they will not be repeated here.

[0202] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages in other steps. It is understood that the steps in different embodiments can be freely combined as needed, and all non-contradictory solutions formed by such combinations are within the scope of protection of this application.

[0203] Based on the same inventive concept, this application also provides a robot pose localization device for implementing the robot pose localization method described above. The solution provided by this device is similar to the implementation described in the above method; therefore, the specific limitations in one or more robot pose localization device embodiments provided below can be found in the limitations of the robot pose localization method described above, and will not be repeated here.

[0204] In one exemplary embodiment, such as Figure 16 As shown, a robot pose localization device is provided, including: an image acquisition module 1601, a line segment matching module 1602, and a pose optimization module 1603, wherein:

[0205] The image acquisition module 1601 is used to acquire images of the tile seams collected by the robot in its initial pose;

[0206] The line segment matching module 1602 is used to determine, in the tile joint map, a reference tile joint line segment that matches the position information of each first tile joint line segment based on at least one first tile joint line segment in the tile joint image.

[0207] The pose optimization module 1603 is used to iteratively optimize the robot's initial pose based on the position information of each first tile seam segment and each reference tile seam segment, so as to obtain the robot's pose localization result.

[0208] Each module in the aforementioned robot pose localization device can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device as software, so that the processor can call and execute the corresponding operations of each module.

[0209] In one exemplary embodiment, a cleaning robot is provided, including a memory and a processor, wherein the memory stores a computer program, and the processor executes the computer program to implement the steps in any of the robot pose localization methods described above.

[0210] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in any of the robot pose localization methods described above.

[0211] In one embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements the steps in any of the robot pose localization methods described above.

[0212] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile memory and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM). The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, artificial intelligence (AI) processors, etc., and are not limited to these.

[0213] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this application.

[0214] The embodiments described above are merely illustrative of several implementation methods of this application, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of this patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of this application, and these all fall within the protection scope of this application. Therefore, the protection scope of this application should be determined by the appended claims.

Claims

1. A robot pose localization method, characterized in that, The method includes: Acquire images of the tile seams captured by the robot in its initial pose; Based on at least one first tile joint line segment in the tile joint image, a reference tile joint line segment is determined in the tile joint map that matches the position information of each first tile joint line segment. Based on the position information of each of the first tile seam segments and the position information of each of the reference tile seam segments, the initial pose of the robot is iteratively optimized to obtain the pose localization result of the robot.

2. The method according to claim 1, characterized in that, The step of determining, based on at least one first tile seam segment in the tile seam image, a reference tile seam segment in the tile seam map that matches the position information of each of the first tile seam segments includes: For each first tile seam segment in the tile seam image, at least one of the endpoint position coordinates and line vector parameters of the first tile seam segment in the world coordinate system is obtained as the position information of the first tile seam segment. The reference tile joint segment is determined in the tile joint map based on the location information of the first tile joint segment.

3. The method according to claim 2, characterized in that, Obtaining the endpoint coordinates and line vector parameters of the first tile joint line segment in the tile joint image in the world coordinate system includes: Obtain the tile joint pixels in the tile joint image, and determine the first tile joint line segment based on the tile joint pixels in the tile joint image; The tile seam pixels in the first tile seam segment are transformed to the world coordinate system, the straight line vector parameters of the first tile seam segment are obtained by fitting, and the endpoint position coordinates of the first tile seam segment are obtained.

4. The method according to claim 2 or 3, characterized in that, The tile joint map includes position information of multiple second tile joint segments; determining the reference tile joint segment in the tile joint map based on the position information of the first tile joint segment includes: The target search range is determined based on the initial pose. Based on the location information of each second tile seam segment in the tile seam map, at least one candidate tile seam segment is determined to be located within the target search range, with at least a portion of the area being the candidate. Based on the position information of the first tile seam segment and the position information of the at least one candidate tile seam segment, the reference tile seam segment is determined from the at least one candidate tile seam segment.

5. The method according to claim 4, characterized in that, The location information includes a straight line vector parameter. The step of determining the reference tile seam segment from the at least one candidate tile seam segment based on the location information of the first tile seam segment and the location information of the at least one candidate tile seam segment includes: Based on the straight line vector parameters of the first tile seam segment and the straight line vector parameters of each candidate tile seam segment, the included angle between the first tile seam segment and each candidate tile seam segment is determined. The reference tile seam segment is determined from the at least one candidate tile seam segment based on the angle between the first tile seam segment and each of the candidate tile seam segments.

6. The method according to claim 5, characterized in that, The location information also includes endpoint location coordinates. The step of determining the reference tile seam segment from the at least one candidate tile seam segment based on the angle between the first tile seam segment and each of the candidate tile seam segments includes: If there is an undetermined tile joint segment among the at least one candidate tile joint segment, the undetermined tile joint segment is determined as the reference tile joint segment; the undetermined tile joint segment is a candidate tile joint segment with an included angle less than a preset included angle threshold. If at least two undetermined tile joint segments exist among the at least one candidate tile joint segments, the reference tile joint segment is determined from the at least two undetermined tile joint segments based on the endpoint position coordinates of the first tile joint segment.

7. The method according to claim 6, characterized in that, The step of determining the reference tile joint segment from the at least two undetermined tile joint segments based on the endpoint coordinates of the first tile joint segment includes: Based on the endpoint coordinates of the first tile seam segment and the straight line vector parameters of each of the undetermined tile seam segments, the distances from the first tile seam segment to each of the undetermined tile seam segments are determined. Among the at least two undetermined tile seam segments, the undetermined tile seam segment corresponding to the smallest distance that is less than a preset distance threshold is determined and used as the reference tile seam segment.

8. The method according to any one of claims 1 to 3, characterized in that, The step of iteratively optimizing the robot's initial pose based on the position information of each of the first tile seam segments and the position information of each of the reference tile seam segments to obtain the robot's pose localization result includes: Construct a pose deviation relationship between the variable pose and the initial pose, and obtain a sum expression of the distance difference between the position information of each first tile seam segment and the position information of each reference tile seam segment under the variable pose, as the distance deviation relationship; Construct an objective function based on the pose deviation relationship and the distance deviation relationship; The robot's pose is used as a variable to iteratively optimize the objective function, thereby obtaining the robot's pose localization result.

9. The method according to claim 8, characterized in that, The initial pose includes a first initial pose determined based on environmental perception data and a second initial pose determined based on robot motion parameters. The pose deviation relationship between the constructed variable pose and the initial pose includes: Obtain the difference expression between the variable pose and the first initial pose, and use it as the first pose deviation relationship; Obtain the difference expression between the variable pose and the second initial pose, as the second pose deviation relationship.

10. The method according to claim 9, characterized in that, The step of constructing the objective function based on the pose deviation relationship and the distance deviation relationship includes: The weights of the first pose deviation relationship, the second pose deviation relationship, and the distance deviation relationship are obtained respectively. Based on the weights of the first pose deviation relationship, the second pose deviation relationship, and the distance deviation relationship, an expression for the weighted sum of the squares of the first pose deviation relationship, the second pose deviation relationship, and the distance deviation relationship is obtained, which serves as the objective function.

11. The method according to claim 8, characterized in that, The step of iteratively optimizing the objective function using the variable pose as a variable to obtain the robot's pose localization result includes: The variable pose is used as a variable to iteratively optimize the objective function, and a reference pose that meets the iteration cutoff condition is obtained. The reference pose and the second initial pose determined based on the robot's motion parameters are fused to obtain the target pose, which is then used as the robot's pose localization result.

12. A robot pose positioning device, characterized in that, The device includes: The image acquisition module is used to acquire images of the tile seams captured by the robot in its initial pose. The line segment matching module is used to determine, in the tile joint map, a reference tile joint line segment that matches the position information of each of the first tile joint line segments based on at least one first tile joint line segment in the tile joint image. The pose optimization module is used to iteratively optimize the initial pose of the robot based on the position information of each of the first tile seam segments and the position information of each of the reference tile seam segments, so as to obtain the pose localization result of the robot.

13. A cleaning robot, comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 11.

14. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

15. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 11.

Images