Robot positioning method and device, storage medium and robot

Abstract

The application discloses a robot positioning method and device, a storage medium and a robot, and belongs to the technical field of positioning. The application obtains a target indoor environment image collected by a robot to be positioned; divides the target indoor environment image into a plurality of indoor environment sub-images; calculates a re-projection error between each indoor environment sub-image and a preset reference image; screens a target feature image from the plurality of indoor environment sub-images and the target indoor environment image according to the re-projection error; and performs indoor positioning on the robot to be positioned according to the target feature image. Through the way of region segmentation of an indoor environment image, the re-projection error between the segmented indoor environment sub-image and the preset reference image is used to perform indoor positioning on the robot, so that the calculation resource is reduced, and the cost of robot positioning is greatly reduced.

Description

Technical Field

[0001] This invention relates to the field of positioning technology, and in particular to a robot positioning method, device, storage medium, and robot. Background Technology

[0002] This invention provides an indoor dynamic scene localization method for home mobile robots. When a home mobile robot is building an environmental map, if a person or pet passes through the robot's camera field of view, the static environment appears to be in motion relative to the robot, and the person, pet, or other moving object also appears to be in motion relative to the robot. The robot struggles to determine these motion relationships, easily leading to localization failure. To address this problem, current home mobile robots on the market require sensors such as LiDAR, ultrasonic modules, and Time-of-Flight (TOF) sensors, and often employ deep learning semantic segmentation algorithms. However, deep learning semantic segmentation algorithms are highly complex, and the large number of sensors significantly increases the cost of robot localization.

[0003] The above content is only used to help understand the technical solution of the present invention and does not represent an admission that the above content is prior art. Summary of the Invention

[0004] The main objective of this invention is to provide a robot positioning method, device, storage medium, and robot, aiming to solve the technical problem of high cost in existing robot positioning technologies.

[0005] To achieve the above objectives, the present invention provides a robot localization method, which includes the following steps:

[0006] Acquire images of the target indoor environment collected by the robot to be positioned;

[0007] The target indoor environment image is divided into multiple indoor environment sub-images;

[0008] Calculate the reprojection error between each of the indoor environment sub-images and the preset reference image;

[0009] Based on the reprojection error, a target feature image is selected from multiple indoor environment sub-images and the target indoor environment image; and

[0010] The robot to be located is positioned indoors based on the target feature image.

[0011] Optionally, dividing the target indoor environment image into multiple indoor environment sub-images includes:

[0012] The target indoor environment image is pixel-marked;

[0013] The marked target indoor environment is evenly divided into a preset number of regular hexagonal image regions;

[0014] Based on the pixel labeling results, determine the region type corresponding to each regular hexagonal image region, as well as the number of similar pixels in each regular hexagonal image region;

[0015] The seed pixels in each hexagonal image region are determined based on the number of types;

[0016] Based on the seed pixels, the regular hexagonal region of the region type being non-uniform is re-marked with pixels; and

[0017] The regular hexagonal image regions are merged according to the new pixel labels and the region type to obtain multiple indoor environment sub-images.

[0018] Optionally, the pixel-marking of the target indoor environment image includes:

[0019] A reference pixel is randomly selected from the target indoor environment image in sequence;

[0020] Determine multiple neighborhood pixels corresponding to the reference pixel;

[0021] Obtain the brightness and color of the reference pixel and the pixels in each field;

[0022] Calculate the pixel distance between the reference pixel and each neighborhood pixel based on the brightness and the color.

[0023] Pixels in each region are marked according to the pixel distance.

[0024] Optionally, acquiring the target indoor environment image collected by the robot to be located includes:

[0025] Acquire the camera's intrinsic parameters, camera distortion parameters, and the original indoor environment images captured by the robot to be located; and

[0026] The image points of the original indoor environment image are distorted according to the camera intrinsic parameters and the camera distortion parameters to obtain the target indoor environment image.

[0027] Optionally, the step of filtering the target feature image from the plurality of indoor environment sub-images and the target indoor environment image based on the reprojection error includes:

[0028] Obtain the difference between the maximum and minimum errors in the reprojection errors;

[0029] The difference is compared with a preset difference threshold; and

[0030] Based on the comparison results, a target feature image is selected from multiple indoor environment sub-images and the target indoor environment image.

[0031] Optionally, calculating the reprojection error between each of the indoor environment sub-images and the preset reference image includes:

[0032] Sequentially remove one indoor environment sub-image from the plurality of indoor environment sub-images; and

[0033] Calculate the reprojection error between the remaining indoor environment sub-images and the preset reference image.

[0034] Optionally, after sequentially removing one indoor environment sub-image from the plurality of indoor environment sub-images, the method further includes:

[0035] Obtain the image area of ​​the removed indoor environment sub-image; and

[0036] If the area of ​​the image is smaller than the preset area, the image region of the target indoor environment image will be re-divided.

[0037] Furthermore, to achieve the above objectives, the present invention also proposes a robot positioning device, the robot positioning device comprising:

[0038] The acquisition module is used to acquire images of the target indoor environment collected by the robot to be positioned.

[0039] A segmentation module is used to divide the target indoor environment image into multiple indoor environment sub-images;

[0040] The calculation module is used to calculate the reprojection error between each of the indoor environment sub-images and the preset reference image;

[0041] The filtering module is used to filter out target feature images from multiple indoor environment sub-images and the target indoor environment image based on the reprojection error;

[0042] The positioning module is used to perform indoor positioning of the robot to be positioned based on the target feature image.

[0043] Furthermore, to achieve the above objectives, the present invention also proposes a robot comprising: a memory, a processor, and a robot localization program stored in the memory and executable on the processor, the robot localization program being configured to implement the steps of the robot localization method as described above.

[0044] In addition, to achieve the above objectives, the present invention also proposes a storage medium storing a robot localization program, which, when executed by a processor, implements the steps of the robot localization method as described above.

[0045] This invention acquires a target indoor environment image collected by a robot to be positioned; divides the target indoor environment image into multiple indoor environment sub-images; calculates the reprojection error between each indoor environment sub-image and a preset reference image; filters out a target feature image from the multiple indoor environment sub-images and the target indoor environment image based on the reprojection error; and performs indoor positioning of the robot to be positioned based on the target feature image. By segmenting the indoor environment image into regions and utilizing the reprojection error between the segmented indoor environment sub-images and the preset reference image, the invention reduces computational resources and significantly lowers the cost of robot positioning. Attached Figure Description

[0046] Figure 1 This is a schematic diagram of the structure of the robot in the hardware operating environment involved in the embodiments of the present invention;

[0047] Figure 2 This is a flowchart illustrating the first embodiment of the robot positioning method of the present invention;

[0048] Figure 3 This is a flowchart illustrating the second embodiment of the robot positioning method of the present invention;

[0049] Figure 4 This is a flowchart illustrating the third embodiment of the robot positioning method of the present invention;

[0050] Figure 5 This is a structural block diagram of the first embodiment of the robot positioning device of the present invention.

[0051] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0052] It should be understood that the specific embodiments described herein are for illustrative purposes only and are not intended to limit the scope of the invention.

[0053] Reference Figure 1 , Figure 1 This is a schematic diagram of the robot structure in the hardware operating environment involved in the embodiments of the present invention.

[0054] like Figure 1As shown, the electronic device may include: a processor 1001, such as a central processing unit (CPU), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen or an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be a high-speed random access memory (RAM) or a stable non-volatile memory (NVM), such as a disk drive. The memory 1005 may also optionally be a storage device independent of the aforementioned processor 1001.

[0055] Those skilled in the art will understand that Figure 1 The structure shown does not constitute a limitation on the electronic device and may include more or fewer components than shown, or combine certain components, or have different component arrangements.

[0056] like Figure 1 As shown, the memory 1005, which serves as a storage medium, may include an operating system, a network communication module, a user interface module, and a robot positioning program.

[0057] exist Figure 1 In the robot shown, the network interface 1004 is mainly used for data communication with the network server; the user interface 1003 is mainly used for data interaction with the user; the processor 1001 and the memory 1005 in the electronic device of the present invention can be set in the robot. The electronic device calls the robot positioning program stored in the memory 1005 through the processor 1001 and executes the robot positioning method provided in the embodiment of the present invention.

[0058] This invention provides a robot localization method, referring to... Figure 2 , Figure 2 This is a flowchart illustrating a first embodiment of a robot localization method according to the present invention.

[0059] In this embodiment, the robot localization method includes the following steps:

[0060] Step S10: Obtain the target indoor environment image collected by the robot to be located.

[0061] It should be noted that the executing entity in this embodiment can be a robot positioning device, which can be an electronic device such as a personal computer or server, or other controllers that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, the robot positioning device is used as an example to illustrate the robot positioning method of the present invention.

[0062] In this embodiment, the mobile robot requiring indoor positioning, i.e., the robot to be positioned, is equipped with a vision sensor. The robot can acquire images of the indoor environment through this vision sensor during movement. The user can send positioning commands to the robot via a mobile terminal or via a physical button on the robot. Upon receiving a positioning command, the robot begins acquiring images of the target indoor environment. A preset time can also be set; when the preset time is reached, the robot automatically acquires images of the target indoor environment for positioning. This embodiment allows for the selection of different methods based on actual positioning needs, and this embodiment does not impose any limitations on this method.

[0063] It should be noted that the visual sensor acquires external environmental information through imaging devices such as cameras. When the camera is taking pictures, in order to ensure the accuracy of the image, the camera needs to be calibrated to eliminate the influence of camera distortion. In this embodiment, in order to obtain a more accurate image of the target indoor environment, it can be achieved in the following way.

[0064] In this specific implementation, camera distortion correction is performed based on camera intrinsic parameters and camera distortion parameters. The distortion of image points in the original indoor environment image acquired by the robot to be localized is corrected using these parameters. Specifically, in this embodiment, radial and tangential distortion corrections can be performed on the image points of the original indoor environment image according to the following relationship:

[0065]

[0066] Wherein, the camera intrinsic parameters are [fx, fy, Cx, Cy], and the camera distortion parameters are [k1, k2, k3, p1, p2], r is the distance from any image point on the original indoor environment image to the origin of the coordinate system, and (x, y) are the coordinates of a point projected from the point in three-dimensional space onto the normalized image plane. corrected y correctedThe coordinates of the point after distortion correction are given. The camera's intrinsic parameters are [fx, fy, Cx, Cy], and the camera distortion parameters [k1, k2, k3, p1, p2] can be determined based on the actual camera's performance parameters. The choice of camera can be adjusted accordingly based on actual positioning requirements; this embodiment does not impose any restrictions on this. After obtaining the distortion-corrected point coordinates, the distortion-corrected point coordinates are then transformed to the pixel plane, i.e.:

[0067]

[0068] Where (u, v) are the coordinates of the pixel plane points, and the pixel plane obtained after transformation is the target indoor environment image.

[0069] Step S20: Divide the target indoor environment image into multiple indoor environment sub-images.

[0070] In this implementation, after acquiring the target indoor environment, the image segmentation method used in this embodiment is employed. Utilizing the concept of superpixel segmentation, this algorithm has lower complexity compared to the deep learning semantic segmentation used in existing technologies, effectively saving computational resources during robot localization. In this embodiment, when segmenting the target indoor environment image, it can be uniformly divided into multiple indoor environment sub-images, for example, dividing the target indoor environment image into N average indoor environment sub-images. Flat areas after segmentation can be directly merged, while non-flat areas are re-divided according to the boundaries.

[0071] Step S30: Calculate the reprojection error between each of the indoor environment sub-images and the preset reference image.

[0072] It should be noted that when the camera captures images, factors such as camera position, shooting angle, and camera operating status can cause differences even in images captured from the same location within an indoor environment. This embodiment uses image-based positioning; therefore, to improve positioning accuracy, after obtaining the indoor environment sub-image, the reprojection error between the indoor environment sub-image and a preset reference image is calculated. The preset reference image is an indoor environment image used for reference and can be set according to actual needs; this embodiment does not impose any restrictions on it. Specifically, this embodiment determines the reprojection error between the indoor environment sub-image and the preset reference image by converting the image coordinate points on the indoor environment sub-image and the preset reference image.

[0073] Step S40: Select the target feature image from the multiple indoor environment sub-images and the target indoor environment image based on the reprojection error.

[0074] It should be noted that the reprojection error is the reprojection error between the sub-images of the indoor environment obtained after segmenting the target indoor environment image and the preset reference image, rather than the reprojection error between the entire target indoor environment image and the preset reference image. Based on this reprojection error, this embodiment can determine whether to use a portion of the image, i.e., multiple sub-images of the indoor environment, as the target feature image, or to use the entire image, i.e., the target indoor environment image, as the target feature image. In order to improve the accuracy of target feature image selection in this embodiment, it can be implemented in the following way.

[0075] In this specific implementation, multiple reprojection errors are obtained. An error sequence is constructed from these multiple reprojection errors. The maximum and minimum reprojection errors are selected from this error sequence. The difference between the maximum and minimum reprojection errors is obtained by subtraction. This difference is then compared with a preset difference threshold. Finally, the target feature image is selected based on the comparison result. The preset difference threshold can be set according to actual needs, and this embodiment does not impose any restrictions on it.

[0076] In practice, if the difference is greater than the preset difference threshold, the indoor environment sub-image corresponding to the minimum reprojection error is used as the target feature image; otherwise, if the difference is less than or equal to the preset difference threshold, the target indoor environment image is used as the target feature image.

[0077] Step S50: Perform indoor positioning of the robot to be positioned based on the target feature image.

[0078] In this embodiment, the light streamer method can be used to track feature points from the target feature image to obtain the positions of various scenes and objects in the indoor environment, thereby determining the position of the robot to be located in the entire indoor environment. Of course, other methods can also be used to track feature points from the target feature image in this embodiment, and can be set accordingly according to actual needs. This embodiment does not limit this.

[0079] This embodiment acquires a target indoor environment image collected by the robot to be located; divides the target indoor environment image into multiple indoor environment sub-images; calculates the reprojection error between each indoor environment sub-image and a preset reference image; filters out a target feature image from the multiple indoor environment sub-images and the target indoor environment image based on the reprojection error; and performs indoor localization of the robot to be located based on the target feature image. By segmenting the indoor environment image into regions, the reprojection error between the segmented indoor environment sub-images and the preset reference image is used to perform indoor localization of the robot, reducing computational resources and significantly reducing the cost of robot localization.

[0080] refer to Figure 3 , Figure 3 This is a flowchart illustrating a second embodiment of a robot localization method according to the present invention.

[0081] Based on the first embodiment described above, step S20 in the robot localization method of this embodiment specifically includes the following steps:

[0082] Step S201: Mark the image pixels of the target indoor environment image.

[0083] In this implementation, the same type of pixels are selected from the target indoor environment image by pixel marking. Specifically, a pixel is randomly selected from the target indoor environment image as a reference pixel, and multiple neighboring pixels corresponding to the reference pixel are determined. That is, the other pixels within the nine-square grid centered on the reference pixel are the 8-neighbor pixels of the reference pixel. Then, the pixel distance between the reference pixel and each neighboring pixel is calculated. In this embodiment, the pixel distance is calculated based on the brightness and color of the pixel. Finally, the same type of pixels are found from the neighboring pixels based on the distance. The distance calculation formula is as follows:

[0084]

[0085] Where l represents brightness, with a higher value indicating whiter white; a represents the hue between red and green; b represents the hue between yellow and blue, including both red-green and yellow-blue hues. Then D... ij The value is compared with a preset distance, and pixels are labeled based on the comparison result. For example, if D... ij If the distance is less than or equal to a preset distance, then the neighboring pixel is considered a pixel of the same type and marked as C1. If D ij If the distance is greater than a preset value, the neighboring pixel is marked as C2. After classifying the pixels of the same type for this pixel, the same classification is performed on the 8 neighboring pixels of other pixels. If there is a pixel of the same type among the 8 neighboring pixels of other pixels that does not belong to either C1 or C2, then that pixel is set to C3, and so on, until the classification reaches C1. n Similar pixels.

[0086] Step S202: Divide the marked target indoor environment image into a preset number of regular hexagonal regions.

[0087] In specific implementation, after pixel marking is completed, the target indoor environment image after pixel marking can be evenly divided into regular hexagonal regions according to regular hexagons or other geometric shapes. Each regular hexagonal region is a superpixel. The number of divisions can be preset and adjusted according to actual needs. This embodiment does not limit this.

[0088] Step S203: Determine the region type corresponding to each regular hexagonal image region and the number of similar pixels in each regular hexagonal image region based on the pixel marking results.

[0089] It should be noted that, based on the marking results, the similar pixels in each regular hexagonal region can be identified. Then, the region type of the regular hexagonal region is determined based on the proportion of similar pixels in the region. For example, if the proportion of similar pixels exceeds 70%, the region is considered a uniform region; otherwise, it is a non-uniform region. Furthermore, based on the marking results, the number of types of similar pixels in the regular hexagonal region can also be determined, i.e., how many types of similar pixels are contained in the region. Further, based on the pixel marking results, the similar pixels in each regular hexagonal region can be identified, thus obtaining the proportion of similar pixels to all pixels in the region. If this proportion is greater than a preset proportion, the region type of the regular hexagonal region is a uniform region; otherwise, it is a non-uniform region. The preset proportion can be set to 70%, or it can be set to other values ​​according to actual conditions; this embodiment does not impose any restrictions on this.

[0090] Step S204: Determine the seed pixels in each regular hexagonal image region according to the number of types.

[0091] In practice, different methods are used to determine seed pixels in each hexagonal image region depending on the number of types. Specifically, when there is only one type, the Euclidean distance between each pixel in the corresponding hexagonal image region and the center of the hexagon is obtained; the target Euclidean distance is selected from the Euclidean distances, and the pixel corresponding to the target Euclidean distance is used as the seed pixel. The Euclidean distance formula is as follows:

[0092]

[0093] In this embodiment, the target Euclidean distance is the minimum Euclidean distance between each pixel and the center of the regular hexagon, and the seed pixel is the pixel corresponding to this minimum Euclidean distance.

[0094] Furthermore, when there are two or more types of pixels, a seed metric value is calculated for each pixel in the corresponding hexagonal region; a target seed metric value is selected from the seed metric values, and the pixel corresponding to the target seed metric value is used as the seed pixel. Specifically, if there are two or more types of pixels of the same type, the second-order difference quotients in the four directions of the l, a, and b channels are calculated, taking l as an example as follows.

[0095]

[0096] Averaging the second order in the four directions yields Then calculate sequentially according to the above calculation process. Finally, the seed metric is calculated, and the formula for calculating the metric is as follows:

[0097]

[0098] Based on this metric calculation formula, the seed metric value corresponding to each pixel in the regular hexagonal region can be obtained. In this embodiment, the target metric value is the minimum metric value, and the pixel corresponding to the minimum metric value is used as the seed pixel of the corresponding regular hexagonal region.

[0099] Step S205: Re-mark the pixels of the regular hexagonal region whose region type is non-uniform region according to the seed pixels.

[0100] It should be noted that a second pixel labeling is required for the non-uniform hexagonal region. Specifically, an arbitrary pixel to be labeled is selected from the non-uniform region, and the seed distance between the pixel to be labeled and each seed pixel within a preset range is calculated. The minimum seed distance is selected from these seed distances, and the corresponding target seed pixel is determined based on this minimum seed distance. The pixel label of the target seed pixel is used as the pixel label of the pixel to be labeled. This embodiment uses a regular hexagonal region as an example for explanation. In the non-uniform region, a pixel p is defined. qj The seed pixel is S. u Calculate pixel p qj With seed S u The distance between them is calculated using the following formula:

[0101]

[0102] Where w1-w4 are weighting coefficients, which can be adjusted according to actual needs, and l, a, b, u, v are normalized values ​​used to calculate pixel p. qj The distance between seed pixels within a circle with a radius twice the side length of the regular hexagon and its nearest neighbor is used as the label of the seed pixel corresponding to the minimum distance. qjThe pixels are marked, thereby remarking the pixels in the non-uniform area.

[0103] Step S206: Merge the regular hexagonal regions according to the new pixel labels and the region type to obtain multiple indoor environment sub-images.

[0104] In practice, merging the regular hexagonal regions can yield multiple indoor environment sub-images. In this embodiment, the merging is performed according to pixel labels and region types. For example, regular hexagonal regions with the same pixel labels can be merged, or regular hexagonal regions with the same region type can be merged. Of course, other methods can also be adopted, and this embodiment does not limit them.

[0105] In this implementation, uniform regions are selected based on region type. Then, the pixel labels of each uniform region are detected, and adjacent uniform regions with the same pixel label are merged. Among the remaining pixels in the unmerged uniform and non-uniform regions, pixels with the same label in their four neighborhoods can be merged into a single superpixel according to a preset order. This preset order can be from top to bottom, from left to right, or any other order; this embodiment does not impose any restrictions. Furthermore, if no two neighborhoods have the same label, they are treated as a single superpixel. The centroid (x, y) of each superpixel is calculated, and its neighboring superpixels are traversed sequentially. The distance between the centroid (x, y) and each neighboring superpixel is calculated using the following formula:

[0106]

[0107] Here, w5 is a weighting coefficient, which can be set according to actual needs. When Ds is less than the set threshold, superpixels are merged; otherwise, they are not merged. After merging, the superpixel centroids are recalculated, and the loop continues until the total number of superpixels is less than N. S The merge stops. It's important to emphasize that setting a threshold and N is necessary. S The specific value can be set according to actual needs, and this embodiment does not impose any restrictions on it.

[0108] This embodiment marks the target indoor environment image with pixels, then divides it evenly. Based on the pixel marking results, it determines the seed pixels and region type corresponding to the regular hexagonal region. Based on the seed pixels, it re-marks the regular hexagonal region with the region type of non-uniform region. Finally, it merges the regular hexagonal regions according to the new pixel marking and region type to obtain multiple indoor environment sub-images, making the division of the target indoor environment image more reasonable and accurate.

[0109] refer to Figure 4 , Figure 4This is a flowchart illustrating a third embodiment of a robot localization method according to the present invention.

[0110] Based on the first or second embodiment described above, a third embodiment of the robot localization method of the present invention is proposed.

[0111] Taking the first embodiment described above as an example, in this embodiment, step S30 specifically includes the following steps:

[0112] Step S301: Remove one indoor environment sub-image from the plurality of indoor environment sub-images in sequence.

[0113] It should be noted that in calculating the reprojection error, this embodiment randomly removes one indoor environment sub-image from multiple indoor environment sub-images sequentially. Then, the remaining image area and a preset reference image are used to calculate the reprojection error. For example, if there are K indoor environment sub-images, K-1 indoor environment sub-images are used to calculate the reprojection error, and this removal is done sequentially. For instance, if there are three indoor environment sub-images A, B, and C, A, B, and C are removed sequentially, and the remaining three types of indoor environment sub-images—A and B, B and C, and A and C—are used to calculate the reprojection error. Furthermore, this embodiment detects the image area of ​​the removed indoor environment sub-image. If this area is smaller than a preset area, it indicates that the removed indoor environment sub-image is unreasonable and the target indoor environment image needs to be re-divided. The preset area can be set according to actual needs; this embodiment does not impose any restrictions on this.

[0114] Step S302: Calculate the reprojection error between the remaining indoor environment sub-image and the preset reference image.

[0115] In specific implementation, let the time of the remaining indoor environment sub-image frame be c, the time of the preset reference image frame be k, and the coordinates of the remaining indoor environment sub-image frame point be (u c ,v c ), with a depth of z c Then the coordinates of the midpoint of the remaining indoor environment sub-image frame in 3D space are: Let's assume that the camera rotation between the remaining indoor environment sub-image frames and the preset reference image frame is... Translation The coordinates of the corresponding 3D point in the current frame are then transformed to the coordinates in the preset reference image frame. Then P k Projected onto the reference keyframe image plane as u k =(z k f x +c x ,z k f y +cy Then the reprojection error can be obtained as e(u). k ).

[0116] This embodiment can accurately calculate the projection error by sequentially removing one image region from the plurality of indoor environment sub-images and calculating the projection error between the remaining image region and the preset reference image. At the same time, it can obtain the area of ​​the removed image region. If the area is smaller than the preset area, the target indoor environment image is re-divided into image regions to ensure the rationality of the image region division.

[0117] Reference Figure 5 , Figure 5 This is a structural block diagram of the first embodiment of the robot positioning device of the present invention.

[0118] like Figure 5 As shown, the robot positioning device proposed in this embodiment of the invention includes:

[0119] The acquisition module 10 is used to acquire images of the target indoor environment collected by the robot to be positioned.

[0120] It should be noted that the executing entity in this embodiment can be a robot positioning device, which can be an electronic device such as a personal computer or server, or other controllers that can achieve the same or similar functions. This embodiment does not limit this. In this embodiment and the following embodiments, the robot positioning device is used as an example to illustrate the robot positioning method of the present invention.

[0121] In this embodiment, the mobile robot requiring indoor positioning, i.e., the robot to be positioned, is equipped with a vision sensor. The robot can acquire images of the indoor environment through this vision sensor during movement. The user can send positioning commands to the robot via a mobile terminal or via a physical button on the robot. Upon receiving a positioning command, the robot begins acquiring images of the target indoor environment. A preset time can also be set; when the preset time is reached, the robot automatically acquires images of the target indoor environment for positioning. This embodiment allows for the selection of different methods based on actual positioning needs, and this embodiment does not impose any limitations on this method.

[0122] It should be noted that the visual sensor acquires external environmental information through imaging devices such as cameras. When the camera is taking pictures, in order to ensure the accuracy of the image, the camera needs to be calibrated to eliminate the influence of camera distortion. In this embodiment, in order to obtain a more accurate image of the target indoor environment, it can be achieved in the following way.

[0123] In this specific implementation, camera distortion correction is performed based on camera intrinsic parameters and camera distortion parameters. The distortion of image points in the original indoor environment image acquired by the robot to be localized is corrected using these parameters. Specifically, in this embodiment, radial and tangential distortion corrections can be performed on the image points of the original indoor environment image according to the following relationship:

[0124]

[0125] Wherein, the camera intrinsic parameters are [fx, fy, Cx, Cy], and the camera distortion parameters are [k1, k2, k3, p1, p2], r is the distance from any image point on the original indoor environment image to the origin of the coordinate system, and (x, y) are the coordinates of a point projected from the point in three-dimensional space onto the normalized image plane. corrected y corrected The coordinates of the point after distortion correction are given. The camera's intrinsic parameters are [fx, fy, Cx, Cy], and the camera distortion parameters [k1, k2, k3, p1, p2] can be determined based on the actual camera's performance parameters. The camera selection can be adjusted accordingly based on actual positioning requirements; this embodiment does not impose any restrictions on this. After obtaining the distortion-corrected point coordinates, the distortion-corrected point coordinates are then transformed to the pixel plane, i.e. Where (u, v) are the coordinates of the pixel plane points, and the pixel plane obtained after transformation is the target indoor environment image.

[0126] The segmentation module 20 is used to divide the target indoor environment image into multiple indoor environment sub-images.

[0127] In this implementation, after acquiring the target indoor environment, the image segmentation method used in this embodiment is employed. Utilizing the concept of superpixel segmentation, this algorithm has lower complexity compared to the deep learning semantic segmentation used in existing technologies, effectively saving computational resources during robot localization. In this embodiment, when segmenting the target indoor environment image, it can be uniformly divided into multiple indoor environment sub-images, for example, divided into n average indoor environment sub-images, or it can be irregularly divided into multiple indoor environment sub-images. The specific image segmentation method can be selected according to actual needs, and this embodiment does not impose any restrictions on it.

[0128] The calculation module 30 is used to calculate the reprojection error between each of the indoor environment sub-images and the preset reference image.

[0129] It should be noted that when the camera captures images, factors such as camera position, shooting angle, and camera operating status can cause differences even in images captured from the same location within an indoor environment. This embodiment uses image-based positioning; therefore, to improve positioning accuracy, after obtaining the indoor environment sub-image, the reprojection error between the indoor environment sub-image and a preset reference image is calculated. The preset reference image is an indoor environment image used for reference and can be set according to actual needs; this embodiment does not impose any restrictions on it. Specifically, this embodiment determines the reprojection error between the indoor environment sub-image and the preset reference image by converting the image coordinate points on the indoor environment sub-image and the preset reference image.

[0130] The filtering module 40 is used to filter out the target feature image from multiple indoor environment sub-images and the target indoor environment image based on the reprojection error.

[0131] It should be noted that the reprojection error is the reprojection error between the sub-images of the indoor environment obtained after segmenting the target indoor environment image and the preset reference image, rather than the reprojection error between the entire target indoor environment image and the preset reference image. Based on this reprojection error, this embodiment can determine whether to use a portion of the image, i.e., multiple sub-images of the indoor environment, as the target feature image, or to use the entire image, i.e., the target indoor environment image, as the target feature image. In order to improve the accuracy of target feature image selection in this embodiment, it can be implemented in the following way.

[0132] In this specific implementation, multiple reprojection errors are obtained. An error sequence is constructed from these multiple reprojection errors. The maximum and minimum reprojection errors are selected from this error sequence. The difference between the maximum and minimum reprojection errors is obtained by subtraction. This difference is then compared with a preset difference threshold. Finally, the target feature image is selected based on the comparison result. The preset difference threshold can be set according to actual needs, and this embodiment does not impose any restrictions on it.

[0133] In practice, if the difference is greater than the preset difference threshold, the indoor environment sub-image corresponding to the minimum reprojection error is used as the target feature image; otherwise, if the difference is less than or equal to the preset difference threshold, the target indoor environment image is used as the target feature image.

[0134] The positioning module 50 is used to perform indoor positioning of the robot to be positioned based on the target feature image.

[0135] In this embodiment, the light streamer method can be used to track feature points from the target feature image to obtain the positions of various scenes and objects in the indoor environment, thereby determining the position of the robot to be located in the entire indoor environment. Of course, other methods can also be used to track feature points from the target feature image in this embodiment, and can be set accordingly according to actual needs. This embodiment does not limit this.

[0136] This embodiment acquires a target indoor environment image collected by the robot to be located; divides the target indoor environment image into multiple indoor environment sub-images; calculates the reprojection error between each indoor environment sub-image and a preset reference image; filters out a target feature image from the multiple indoor environment sub-images and the target indoor environment image based on the reprojection error; and performs indoor localization of the robot to be located based on the target feature image. By segmenting the indoor environment image into regions, the reprojection error between the segmented indoor environment sub-images and the preset reference image is used to perform indoor localization of the robot, reducing computational resources and significantly reducing the cost of robot localization.

[0137] Furthermore, this embodiment of the invention also proposes a storage medium storing a robot localization program, which, when executed by a processor, implements the steps of the robot localization method described above.

[0138] It should be understood that the above are merely illustrative examples and do not constitute any limitation on the technical solution of the present invention. In specific applications, those skilled in the art can make settings as needed, and the present invention does not impose any restrictions on this.

[0139] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of this invention. In practical applications, those skilled in the art can select some or all of the workflow to achieve the purpose of this embodiment according to actual needs, and no restrictions are imposed here.

[0140] In addition, for technical details not described in detail in this embodiment, please refer to the robot localization method provided in any embodiment of the present invention, which will not be repeated here.

[0141] Furthermore, it should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.

[0142] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.

[0143] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as read-only memory (ROM) / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.

[0144] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.

Claims

1. A robot positioning method, characterized by, The robot positioning method comprises: acquiring a target indoor environment image collected by a robot to be positioned; dividing the target indoor environment image into a plurality of indoor environment sub-images; calculating re-projection errors between each of the indoor environment sub-images and a preset reference image; acquiring a difference between a maximum error and a minimum error in the re-projection errors; comparing the difference with a preset difference threshold; and selecting a target feature image from the plurality of indoor environment sub-images and the target indoor environment image according to a comparison result; and performing indoor positioning on the robot to be positioned according to the target feature image.

2. The robot positioning method of claim 1, wherein, The dividing of the target indoor environment image into a plurality of indoor environment sub-images comprises: pixel labeling of the target indoor environment image; uniformly dividing the labeled target indoor environment into a preset number of regular hexagon image regions; determining, according to a pixel labeling result, a region type corresponding to each of the regular hexagon image regions and a category number of the same type of pixels in each of the regular hexagon image regions; determining seed pixels in each of the regular hexagon image regions according to the category number; re-labeling pixels of the regular hexagon region with the region type of non-uniform region according to the seed pixels; and merging the regular hexagon image regions according to the new pixel labeling and the region type to obtain a plurality of indoor environment sub-images.

3. The robot positioning method of claim 2, wherein, The pixel labeling of the target indoor environment image comprises: sequentially selecting a reference pixel point from the target indoor environment image; determining a plurality of field pixel points corresponding to the reference pixel point; acquiring brightness and color of the reference pixel point and each of the field pixel points; calculating pixel distances between the reference pixel point and each of the field pixel points according to the brightness and the color; and pixel labeling each of the field pixel points according to the pixel distances.

4. The robot positioning method of claim 1, wherein, The acquiring of the target indoor environment image collected by the robot to be positioned comprises: acquiring camera internal parameters, camera distortion parameters of the robot to be positioned and an original indoor environment image collected by the robot to be positioned; and correcting image points of the original indoor environment image according to the camera internal parameters and the camera distortion parameters to obtain the target indoor environment image.

5. The robot positioning method of any one of claims 1 to 4, wherein, The calculating of the re-projection errors between each of the indoor environment sub-images and the preset reference image comprises: sequentially removing one indoor environment sub-image from the plurality of indoor environment sub-images; and calculating re-projection errors between the remaining indoor environment sub-images and the preset reference image.

6. The robot positioning method of claim 5, wherein, After the sequential removal of one indoor environment sub-image from the plurality of indoor environment sub-images, the method further comprises: acquiring an image area of the removed indoor environment sub-image; and if the image area is smaller than a preset area, re-dividing the target indoor environment image into image regions.

7. A robot positioning device, characterized in that The robot positioning device comprises: an acquiring module configured to acquire a target indoor environment image collected by a robot to be positioned; a dividing module configured to divide the target indoor environment image into a plurality of indoor environment sub-images; a calculating module configured to calculate re-projection errors between each of the indoor environment sub-images and a preset reference image; and a selecting module configured to select a target feature image from the plurality of indoor environment sub-images and the target indoor environment image according to a comparison result. The screening module is configured to obtain a difference between a maximum error and a minimum error in the reprojection errors, compare the difference with a preset difference threshold, and screen a target feature image from the multiple indoor environment sub-images and the target indoor environment image according to a comparison result. The positioning module is configured to perform indoor positioning on the robot to be positioned according to the target feature image.

8. A robot, characterized in that The robot comprises a memory, a processor, and a robot positioning program stored in the memory and executable on the processor, and the robot positioning program is configured to implement the steps of the robot positioning method according to any one of claims 1 to 6.

9. A storage medium, characterized by The storage medium stores a robot positioning program, and the robot positioning program is executable on the processor to implement the steps of the robot positioning method according to any one of claims 1 to 6.

Images