Library robot positioning and mapping method and system, robot and storage medium

Abstract

The invention discloses a library robot positioning and mapping method and system, a robot and a storage medium, and the method comprises the steps: collecting an image in a library, and carrying out the text recognition, and obtaining the text semantic information and position information of a book; performing plane fitting processing on the image, calculating a text semantic plane and an environment plane, and determining a Manhattan coordinate axis under a world coordinate system; performing feature detection on the image, and identifying feature point data; feature point data, text semantic planes and Manhattan coordinate axes in different frames of images are matched; solving pose data of the current frame image by using the matched feature point data; judging whether the current frame image is a key frame image or not, and if yes, constructing a three-dimensional map with text semantic information; and carrying out nonlinear optimization on the map points, the text semantic plane and the pose data. Library information can be fully utilized to improve positioning precision, certain dynamic objects can be compatible, and a semantic map of a book can be generated.

Description

Technical Field

[0001] The present invention relates to a library robot positioning and mapping method, system, robot and storage medium, and belongs to the field of computer vision technology / indoor positioning technology. Background Art

[0002] At present, most library robot positioning methods use RFID tags placed on the edge of the bookshelf for positioning. They mainly update the location information in real time by building a fuzzy reasoning system, presetting the RFID tag path, and dynamically reading the RFID tag information, or use general vision / laser SLAM technology for positioning and mapping. The disadvantage of using the RFID method is that RFID tags need to be manually deployed as prior information before positioning, resulting in high labor costs for using this method, and positioning through RFID is not a 6-degree-of-freedom posture, and the posture information is incomplete; and the disadvantage of using the general SLAM method is that since the method is not aimed at library scenes, the information in the library scene is not fully utilized, such as a large amount of text information, artificial regular scenes, and a large amount of plane information, and it is not optimized for dynamic objects (such as walking people). As a result, the positioning accuracy is not high enough, and the posture tracking is easily lost;

[0003] Most indoor positioning technologies use feature point information for positioning and mapping. They mainly solve the pose through epipolar geometry, and then optimize the pose and map through the constraints of map point reprojection errors. There are also some that optimize the pose by combining point, line, surface and Manhattan world feature constraints. They do not fully utilize the information of the library scene and have a greater impact on dynamic objects. Because the plane is extracted without text, it is impossible to use the rich text information in the library scene, nor can it create a three-dimensional map with text semantics for subsequent use of the map. In addition, this method does not perform special processing on dynamic objects. Summary of the invention

[0004] In view of this, the present invention provides a library robot positioning and mapping method, system, computer equipment and storage medium, which can make full use of library information (book text information, artificial structure information and plane information) to improve positioning accuracy, can be compatible with certain dynamic objects, and can generate book semantic maps.

[0005] The first object of the present invention is to provide a library robot positioning and mapping method

[0006] A second object of the present invention is to provide a library robot positioning and mapping system.

[0007] A third object of the present invention is to provide a computer device.

[0008] A fourth object of the present invention is to provide a storage medium.

[0009] The first object of the present invention can be achieved by adopting the following technical solutions:

[0010] A library robot positioning and mapping method, applied to the robot, the method comprising:

[0011] Collect images in the library and perform text recognition on the images to obtain the text semantic information of the book and the location information of the book in the image;

[0012] Perform plane fitting processing on the image, calculate the text semantic plane and the environment plane, and determine the Manhattan coordinate axis in the world coordinate system based on the environment plane;

[0013] Perform feature detection on the image and identify the feature point data in the image;

[0014] Match the feature point data, text semantic planes and Manhattan coordinate axes in different frame images;

[0015] Use the matched feature point data to solve the pose data of the current frame image;

[0016] Determine whether the current frame image is a key frame image. If so, convert the text semantic information, feature point data, text semantic plane and Manhattan coordinate axis of the current frame image into the world coordinate system according to the posture data to construct a three-dimensional map with text semantic information;

[0017] Perform nonlinear optimization on map points, text semantic planes, and pose data.

[0018] Furthermore, before using the matched feature point data to solve the pose data of the current frame image, the method further includes:

[0019] According to the matching point pairs between two frames of images, a rotation histogram is constructed to eliminate moving objects in the image.

[0020] Furthermore, the step of constructing a rotation histogram based on the matching point pair data between the two frames of images to remove moving objects in the images specifically includes:

[0021] Constructing a rotation histogram according to the matching point pairs between the two frames of images, wherein the rotation histogram reflects the distribution of rotation angles between each matching point pair;

[0022] In the rotation histogram, the matching point pairs in the three groups of rotation angles with the highest occurrence frequency are retained to eliminate moving objects in the image.

[0023] Furthermore, the text recognition is performed on the image to obtain the text semantic information of the book and the location information of the book in the image, which specifically includes:

[0024] Open source optical character recognition technology is used to identify and parse text information on environmental objects to obtain the text semantic information of the book and the location information of the book in the image.

[0025] Furthermore, the image is subjected to plane fitting processing to calculate the text semantic plane and the environment plane, and the Manhattan coordinate axis in the world coordinate system is determined according to the environment plane, specifically including:

[0026] The image is processed by plane fitting using the RANSAC algorithm to calculate the text semantic plane normal vector and the environment plane normal vector. The three most perpendicular environment planes in the environment are found through the angle between the environment plane normal vectors. The Manhattan coordinate axis is formed by these three environment planes.

[0027] Furthermore, matching the feature point data, text semantic planes and Manhattan coordinate axes in different frame images specifically includes:

[0028] Use the optical flow method to calculate the position of each feature point in the previous frame of the library image to obtain the matching point pair data;

[0029] Matching the feature points in the text semantic information with the text semantic plane;

[0030] Use text semantic information to match the relationship between image frames and map text, and use this relationship to remove repeated observations of the same text;

[0031] By calculating the angle between the normal vector of the text semantic plane and the Manhattan coordinate axis, when the angle is close to 0 degree or 180 degrees, it indicates that the text semantic plane matches the Manhattan coordinate axis.

[0032] Furthermore, the nonlinear optimization of the map points, text semantic planes and posture data specifically includes:

[0033] Based on the reprojection error, the distance error from the feature point to the text semantic plane, and the angle error between the text semantic plane and the Manhattan coordinate axis, the pose constraints for the library robot are constructed.

[0034] Combining pose constraints and text semantic information, nonlinear optimization is performed on map points, text semantic planes and poses.

[0035] The second object of the present invention can be achieved by adopting the following technical solutions:

[0036] A library robot positioning and mapping system, applied to a robot, comprising:

[0037] The image acquisition and text recognition module is used to acquire images in the library and perform text recognition on the images to obtain the text semantic information of the book and the location information of the book in the image;

[0038] A plane fitting processing module is used to perform plane fitting processing on the image, calculate the text semantic plane and the environment plane, and determine the Manhattan coordinate axis in the world coordinate system according to the environment plane;

[0039] A feature detection module is used to perform feature detection on an image and identify feature point data in the image;

[0040] A matching module is used to match feature point data, text semantic planes and Manhattan coordinate axes in different frame images;

[0041] A posture solving module is used to solve the posture data of the current frame image using the matched feature point data;

[0042] A three-dimensional map construction module is used to determine whether the current frame image is a key frame image. If so, the text semantic information, feature point data, text semantic plane and Manhattan coordinate axis of the current frame image are converted into the world coordinate system according to the posture data to construct a three-dimensional map with text semantic information;

[0043] The optimization module is used to perform nonlinear optimization on map points, text semantic planes, and pose data.

[0044] The third object of the present invention can be achieved by adopting the following technical solutions:

[0045] A robot comprises a processor and a memory for storing a program executable by the processor, wherein when the processor executes the program stored in the memory, the above-mentioned library robot positioning and mapping method is implemented.

[0046] The fourth object of the present invention can be achieved by adopting the following technical solutions:

[0047] A storage medium stores a program, and when the program is executed by a processor, the above-mentioned library robot positioning and mapping method is implemented.

[0048] The present invention has the following beneficial effects compared with the prior art:

[0049] 1. The present invention can generate a three-dimensional map with the semantics of book text. Since the present invention uses OCR to recognize the text of the scene, the title of the book on the bookshelf and the location of the book can be obtained. Finally, this information data can be restored to the world coordinate system through calculation and optimized posture, becoming a three-dimensional map with the semantics of book text.

[0050] 2. The present invention is compatible with a certain amount of scenes containing dynamic objects (it is more robust in scenes containing certain dynamic objects). Due to the use of the optical flow method and the directional consistency algorithm, it is possible to eliminate moving objects in the image and use the key points of static objects to restore the pose, making the algorithm more robust in dynamic scenes.

[0051] 3. The positioning and mapping accuracy of the present invention is improved. Compared with the general SLAM algorithm, the present invention targets the multi-plane and multi-text artificial scenes of the library, uses the point-to-plane distance, the angle between planes, and the photometric error as constraints, and jointly optimizes the pose and map points. As more information is used, the positioning accuracy is also improved.

[0052] 4. The algorithm of the present invention reduces calculation time. Since the illumination of the library scene is stable, the optical flow method is used instead of the feature point method, which can save the time caused by calculating and matching descriptors in the feature point method.

[0053] 5. The present invention operates automatically without the need for manual deployment of tags. Due to the use of visual SLAM technology, the robot can directly perform its own positioning and mapping through the images obtained by the RGBD camera, so there is no need for manual deployment of tags to achieve positioning, reducing labor costs. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the structures shown in these drawings without paying creative work.

[0055] Figure 1 This is a flow chart of the library robot positioning and mapping method according to Example 1 of the present invention.

[0056] Figure 2 This is a flowchart of matching current frame image information according to Embodiment 1 of the present invention.

[0057] Figure 3 This is a flow chart of removing moving objects from an image according to Embodiment 1 of the present invention.

[0058] Figure 4 This is a structural block diagram of the library robot positioning and mapping system of Example 2 of the present invention. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solutions and advantages of the embodiments of the present invention clearer, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention.

[0060] Embodiment 1:

[0061] like Figure 1 As shown, this embodiment provides a library robot positioning and mapping method, which is applied to the robot and includes the following steps:

[0062] S101, collecting images in a library, and performing text recognition on the images to obtain text semantic information of books and location information of the books in the images.

[0063] This embodiment uses open source optical character recognition (OCR) technology, such as PaddleOCR, to identify and parse text information on environmental objects (book spines, signs and other objects) to obtain the text semantic information of the book (the name of the book or the text information of other objects) and the location information of the book in the image.

[0064] S102, performing plane fitting processing on the image, calculating the text semantic plane and the environment plane, and determining the Manhattan coordinate axis in the world coordinate system according to the environment plane.

[0065] This embodiment uses the RANSAC algorithm to perform plane fitting processing on the image, calculates the text semantic plane normal vector (that is, the plane normal vector corresponding to the text semantic information) and the environment plane normal vector, and finds the three environment planes that are closest to being perpendicular to each other in the environment through the angle between the environment plane normal vectors. The three environment planes form the Manhattan coordinate axis, where the closest to being perpendicular to each other is 87-93 degrees.

[0066] S103: Perform feature detection on the image to identify feature point data in the image.

[0067] This embodiment uses Harris corner points to detect the collected images and identify the corner points in the images as feature point data, which are crucial for subsequent image matching and pose estimation.

[0068] S104: Matching feature point data, text semantic planes, and Manhattan coordinate axes in different frame images.

[0069] like Figure 2As shown, step S104 specifically includes:

[0070] S1041. Use an optical flow method to calculate the position of each feature point in the previous frame of the library image to obtain matching point pair data, wherein the optical flow method is a Lucas-Kanade optical flow method.

[0071] S1042: Matching the feature points in the text semantic information with the text semantic plane.

[0072] S1043. Use text semantic information to match the relationship between image frames and map texts, and use the relationship to remove repeated observations of the same text.

[0073] S1044. Calculate the angle between the normal vector of the text semantic plane and the Manhattan coordinate axis. When the angle is close to 0 degree or 180 degrees, it indicates that the text semantic plane matches the Manhattan coordinate axis.

[0074] After step S104, all the information extracted previously has a matching relationship. Before proceeding to the subsequent steps, this embodiment may further include:

[0075] S105 . Construct a rotation histogram based on the matching point pairs between the two frames of images to eliminate moving objects in the images.

[0076] like Figure 3 As shown, step S105 specifically includes:

[0077] S1051: construct a rotation histogram based on matching point pairs between two image frames.

[0078] This embodiment uses the matching point pairs between the two frames of images obtained in S104 to construct a rotation histogram, which reflects the rotation angle distribution between each matching point pair. There are several groups of rotation angles in the rotation histogram, and each group of rotation angles has several matching point pairs.

[0079] S1052: In the rotation histogram, retain the matching point pairs in the three groups of rotation angles with the highest occurrence frequency to eliminate moving objects in the image.

[0080] This embodiment only retains the matching point pairs in the three groups of rotation angles that appear most frequently in the rotation histogram, because these point pairs are most likely to represent the stable movement of the background rather than being caused by dynamic objects. This method can effectively identify and eliminate those matching point pairs whose movement trends are inconsistent with most of the static objects in the background. These eliminated point pairs are usually associated with moving objects in the image, such as pedestrians or other robots, to avoid causing data interference.

[0081] S106: Use the matched feature point data to solve the pose data of the current frame image.

[0082] This embodiment uses the matched feature point data to calculate the pose data of the current frame image with respect to the world coordinate system through the PNP algorithm.

[0083] S107: Determine whether the current frame image is a key frame image.

[0084] This embodiment determines whether the current frame image is a key frame image, that is, determines whether the current frame image meets the conditions of being a key frame. If so, proceeds to step S108; if not, returns to step S101; wherein, the conditions of the key frame are that a certain time interval is met, the image is not blurred, and the extracted feature points reach a certain number.

[0085] S108. According to the position and posture data, the text semantic information, feature point data, text semantic plane and Manhattan coordinate axis of the current frame image are converted into the world coordinate system to construct a three-dimensional map with text semantic information.

[0086] S109, performing nonlinear optimization on map points, text semantic planes, and position and posture data.

[0087] This embodiment constructs a pose constraint for the library robot based on the reprojection error, the distance error from the feature point to the text semantic plane, and the angle error between the text semantic plane and the Manhattan coordinate axis, which satisfies the following relationship:

[0088] E textplane =nT wc P c -d (1)

[0089]

[0090]

[0091] Among them, E reproj is the reprojection error, E textplane is the distance error from point to text plane, E plane is the angle error between the text plane and the Manhattan plane.

[0092] In equation (1), n ​​is the normal vector of the text plane, T wc is the pose transformation from the camera to the world coordinate system, P c is the key point, and d is the text plane parameter.

[0093] In equation (2), n mw is the normal vector of the Manhattan world plane, R is the rotation matrix from the camera to the world coordinate system, n text is the normal vector of the text plane.

[0094] After constructing the posture constraints, this embodiment combines the posture constraints and text semantic information to perform nonlinear optimization on the map points, text semantic planes and postures.

[0095] It should be noted that although the method operations of the above embodiments are described in a specific order, this does not require or imply that these operations must be performed in this specific order, or that all the operations shown must be performed to achieve the desired results. On the contrary, the steps depicted can be performed in a different order. Additionally or alternatively, some steps can be omitted, multiple steps can be combined into one step, and / or one step can be decomposed into multiple steps.

[0096] Embodiment 2:

[0097] like Figure 4 As shown, this embodiment provides a library robot positioning and mapping system, which includes an image acquisition and text recognition module 401, a plane fitting processing module 402, a feature detection module 403, a matching module 404, a posture solving module 405, a three-dimensional map building module 406 and an optimization module 407. The specific functions of each module are as follows:

[0098] The image acquisition and text recognition module 401 is used to acquire images in the library and perform text recognition on the images to obtain text semantic information of the books and location information of the books in the images;

[0099] The plane fitting processing module 402 is used to perform plane fitting processing on the image, calculate the text semantic plane and the environment plane, and determine the Manhattan coordinate axis in the world coordinate system according to the environment plane;

[0100] The feature detection module 403 is used to perform feature detection on the image and identify feature point data in the image;

[0101] A matching module 404 is used to match feature point data, text semantic planes and Manhattan coordinate axes in different frame images;

[0102] A posture solving module 405 is used to solve the posture data of the current frame image using the matched feature point data;

[0103] The three-dimensional map construction module 406 is used to determine whether the current frame image is a key frame image. If so, the text semantic information, feature point data, text semantic plane and Manhattan coordinate axis of the current frame image are converted into the world coordinate system according to the posture data to construct a three-dimensional map with text semantic information.

[0104] The optimization module 407 is used to perform nonlinear optimization on map points, text semantic planes and posture data.

[0105] It should be noted that the system provided in this embodiment is only illustrated by the division of the above-mentioned functional modules. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure can be divided into different functional modules to complete all or part of the functions described above.

[0106] Embodiment 3:

[0107] This embodiment provides a storage medium, which is a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the library robot positioning and mapping method of the above embodiment 1 is implemented as follows:

[0108] Images are collected in the library, and text recognition is performed on the images to obtain the text semantic information of the book and the location information of the book in the image; plane fitting processing is performed on the image to calculate the text semantic plane and the environment plane, and the Manhattan coordinate axis in the world coordinate system is determined according to the environment plane; feature detection is performed on the image to identify the feature point data in the image; the feature point data, text semantic plane and Manhattan coordinate axis in different frame images are matched; the matched feature point data is used to solve the pose data of the current frame image; it is determined whether the current frame image is a key frame image, and if so, the text semantic information, feature point data, text semantic plane and Manhattan coordinate axis of the current frame image are converted to the world coordinate system according to the pose data to construct a three-dimensional map with text semantic information; nonlinear optimization is performed on the map points, text semantic plane and pose data.

[0109] It should be noted that the computer-readable storage medium of the present embodiment may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0110] In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device, or device. In this embodiment, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable storage medium other than a computer-readable storage medium, which may send, propagate, or transmit a program used by or in combination with an instruction execution system, device, or device. The computer program contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0111] The computer readable storage medium can be written in one or more programming languages ​​or a combination thereof to execute the computer program of the present embodiment, and the programming language includes an object-oriented programming language, such as Java, Python, C++, and also includes a conventional procedural programming language, such as C language or a similar programming language. The program can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (e.g., using an Internet service provider to connect through the Internet).

[0112] In summary, the present invention can make full use of library information (book text information, man-made structure information and plane information) to improve positioning accuracy, is compatible with certain dynamic objects, and can generate a book semantic map.

[0113] The above is only a preferred embodiment of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical solution and invention concept of the present invention within the scope disclosed by the present invention, which shall fall within the protection scope of the present invention.

Claims

1. A library robot positioning and mapping method, applied to the robot, characterized in that: The method comprises: Collect images in the library and perform text recognition on the images to obtain the text semantic information of the book and the location information of the book in the image; Perform plane fitting processing on the image, calculate the text semantic plane and the environment plane, and determine the Manhattan coordinate axis in the world coordinate system based on the environment plane; Perform feature detection on the image and identify the feature point data in the image; Match the feature point data, text semantic planes and Manhattan coordinate axes in different frame images; Use the matched feature point data to solve the pose data of the current frame image; Determine whether the current frame image is a key frame image. If so, convert the text semantic information, feature point data, text semantic plane and Manhattan coordinate axis of the current frame image into the world coordinate system according to the posture data to construct a three-dimensional map with text semantic information; Perform nonlinear optimization on map points, text semantic planes, and pose data.

2. The library robot positioning and mapping method according to claim 1, characterized in that: Before using the matched feature point data to solve the pose data of the current frame image, the method further includes: According to the matching point pairs between two frames of images, a rotation histogram is constructed to eliminate moving objects in the image.

3. The library robot positioning and mapping method according to claim 2, characterized in that: The step of constructing a rotation histogram based on the matching point pair data between the two frames of images to remove moving objects in the images specifically includes: Constructing a rotation histogram according to the matching point pairs between the two frames of images, wherein the rotation histogram reflects the distribution of rotation angles between each matching point pair; In the rotation histogram, the matching point pairs in the three groups of rotation angles with the highest occurrence frequency are retained to eliminate moving objects in the image.

4. The library robot positioning and mapping method according to any one of claims 1 to 3, characterized in that: The performing of text recognition on the image to obtain text semantic information of the book and location information of the book in the image specifically includes: Open source optical character recognition technology is used to identify and parse text information on environmental objects to obtain the text semantic information of the book and the location information of the book in the image.

5. The library robot positioning and mapping method according to any one of claims 1 to 3, characterized in that: The plane fitting process is performed on the image to calculate the text semantic plane and the environment plane, and the Manhattan coordinate axis in the world coordinate system is determined according to the environment plane, specifically including: The image is processed by plane fitting using the RANSAC algorithm to calculate the text semantic plane normal vector and the environment plane normal vector. The three most perpendicular environment planes in the environment are found through the angle between the environment plane normal vectors. The Manhattan coordinate axis is formed by these three environment planes.

6. The library robot positioning and mapping method according to any one of claims 1 to 3, characterized in that: The matching of feature point data, text semantic planes and Manhattan coordinate axes in different frame images specifically includes: Use the optical flow method to calculate the position of each feature point in the previous frame of the library image to obtain the matching point pair data; Matching the feature points in the text semantic information with the text semantic plane; Use text semantic information to match the relationship between image frames and map text, and use this relationship to remove repeated observations of the same text; By calculating the angle between the normal vector of the text semantic plane and the Manhattan coordinate axis, when the angle is close to 0 degree or 180 degrees, it indicates that the text semantic plane matches the Manhattan coordinate axis.

7. The library robot positioning and mapping method according to any one of claims 1 to 3, characterized in that: The nonlinear optimization of map points, text semantic planes and posture data specifically includes: Based on the reprojection error, the distance error from the feature point to the text semantic plane, and the angle error between the text semantic plane and the Manhattan coordinate axis, the pose constraints for the library robot are constructed. Combining pose constraints and text semantic information, nonlinear optimization is performed on map points, text semantic planes and poses.

8. A library robot positioning and mapping system, applied to robots, characterized in that: The system comprises: The image acquisition and text recognition module is used to acquire images in the library and perform text recognition on the images to obtain the text semantic information of the book and the location information of the book in the image; A plane fitting processing module is used to perform plane fitting processing on the image, calculate the text semantic plane and the environment plane, and determine the Manhattan coordinate axis in the world coordinate system according to the environment plane; A feature detection module is used to perform feature detection on an image and identify feature point data in the image; A matching module is used to match feature point data, text semantic planes and Manhattan coordinate axes in different frame images; A posture solving module is used to solve the posture data of the current frame image using the matched feature point data; A three-dimensional map construction module is used to determine whether the current frame image is a key frame image. If so, the text semantic information, feature point data, text semantic plane and Manhattan coordinate axis of the current frame image are converted into the world coordinate system according to the posture data to construct a three-dimensional map with text semantic information; The optimization module is used to perform nonlinear optimization on map points, text semantic planes, and pose data.

9. A robot comprising a processor and a memory for storing a program executable by the processor, characterized in that: When the processor executes the program stored in the memory, the library robot positioning and mapping method described in any one of claims 1 to 7 is implemented.

10. A storage medium storing a program, characterized in that: When the program is executed by a processor, the library robot positioning and mapping method described in any one of claims 1 to 7 is implemented.

Images