A positioning method, device and robot based on point cloud semantic map

Abstract

The present invention provides a positioning method based on a point cloud semantic map. S1: Acquire a real-time image captured by a robot in real time; S2: Use a target detection algorithm to identify the target in the real-time image; S3: Acquire the semantics corresponding to the target from the semantic information table in the semantic map; S4: Reproject the entire point cloud corresponding to the semantics of the target according to the positioning result to obtain a reprojected image; S5: Construct a positioning result optimization cost function based on the reprojected image and the target acquired in real time; S6: Obtain a rough value of the positioning result given by the wheel odometer, and continuously adjust the positioning result value using the Newton-Gauss method until the positioning result optimization cost function reaches a minimum, thereby obtaining the optimal positioning result. The present invention uses an image neural network algorithm to identify the target in the real-time image, which can robustly detect the target. In addition, the present invention uses the point cloud semantics of fixed objects for positioning to avoid interference from dynamic objects.

Description

Technical Field

[0001] The present invention relates to the field of robotics technology, and in particular to a positioning method, device and robot based on point cloud semantic maps. Background Art

[0002] Autonomous mobile robots must be able to navigate their own paths, a prerequisite for which they must be aware of their location. Therefore, positioning technology for autonomous mobile robots has been a hot topic of research in recent years. Currently, the most widely used positioning methods for autonomous mobile robots are laser positioning based on point cloud maps or GPS integration to enhance outdoor positioning robustness.

[0003] However, laser positioning based solely on point cloud maps is prone to reduced positioning accuracy or even position loss in the presence of dynamic objects (such as many cars). In addition, GPS positioning can be integrated in open areas, but this also increases costs and cannot be used in obstructed scenes such as parks.

[0004] The background description provided herein is for the purpose of generally presenting the context of the disclosure. Unless otherwise indicated herein, the materials described in this section are not prior art to the claims of this application and are not admitted to be prior art by inclusion in this section. Summary of the Invention

[0005] In response to the above technical problems in the related art, the present invention proposes a positioning method based on point cloud semantic map, which includes the following steps:

[0006] S1, obtains the real-time image acquired by the robot in real time;

[0007] S2, using a target detection algorithm to identify the target object in the real-time image;

[0008] S3, obtaining the semantics corresponding to the target object from the semantic information table in the semantic map;

[0009] S4, reprojecting all semantic point clouds corresponding to the target object according to the positioning result to obtain a reprojected image;

[0010] S5, constructing a positioning result optimization cost function based on the reprojected image and the target object acquired in real time;

[0011] S6, obtain the rough value of the positioning result [R|t] given by the wheel odometer, and use the Newton-Gauss method to continuously adjust the positioning result [R|t] value until the positioning result optimization cost function Err([R|t]) reaches the minimum, thereby obtaining the optimal positioning result [R|t].

[0012] Specifically, the target detection algorithm in step S2 is YOLO.

[0013] Specifically, the semantic information table records the category, position, and size of the semantics.

[0014] Specifically, the positioning result optimization cost function is:

[0015] Among them, pixel(u Ai ,v Ai ) means getting the pixel value of the i-th point cloud in semantic A, because each point in the point cloud semantics already has a color; pixel(u ai ,v ai ) means obtaining the pixel value of the i-th point cloud in the image target a.

[0016] In a second aspect, another embodiment of the present invention discloses a positioning device based on a point cloud semantic map, which includes the following units:

[0017] A real-time image acquisition unit, used to acquire real-time images acquired by the robot in real time;

[0018] An object detection unit, configured to identify an object in the real-time image using an object detection algorithm;

[0019] A semantic acquisition unit, configured to acquire the semantics corresponding to the target object from a semantic information table in a semantic map;

[0020] A reprojection unit, configured to reproject all semantic point clouds corresponding to the target object according to the positioning result to obtain a reprojected image;

[0021] A positioning result optimization cost function construction unit, configured to construct a positioning result optimization cost function using the reprojected image and the target object acquired in real time;

[0022] The iterative optimization unit is used to obtain the rough value of the positioning result [R|t] given by the wheel odometer, and continuously adjust the positioning result [R|t] value using the Newton-Gauss method until the positioning result optimization cost function Err([R|t]) reaches the minimum, thereby obtaining the optimal positioning result [R|t].

[0023] Specifically, the target detection algorithm in the target detection unit is YOLO.

[0024] Specifically, the semantic information table records the category, position, and size of the semantics.

[0025] Specifically, the positioning result optimization cost function is:

[0026] Among them, pixel(u Ai ,v Ai ) means getting the pixel value of the i-th point cloud in semantic A, because each point in the point cloud semantics already has a color; pixel(u ai ,v ai ) means obtaining the pixel value of the i-th point cloud in the image target a.

[0027] In a third aspect, another embodiment of the present invention discloses a non-volatile memory having instructions stored thereon, which, when executed by a processor, are used to implement a positioning method based on a point cloud semantic map as described above.

[0028] In a fourth aspect, another embodiment of the present invention discloses a robot, which includes a visual sensor, a chassis, and a positioning device based on the point cloud semantic map as described above.

[0029] The present invention uses an image neural network algorithm to identify targets in real-time images and can robustly detect targets. In addition, the present invention uses point cloud semantics for positioning (point cloud semantics are generally some fixed objects) to avoid interference from dynamic objects. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0031] Figure 1 This is a flow chart of a positioning method based on a point cloud semantic map provided by an embodiment of the present invention;

[0032] Figure 2 Schematic diagram of a point cloud map and a point cloud semantic map provided by an embodiment of the present invention;

[0033] Figure 3 Schematic diagram of a positioning device based on a point cloud semantic map provided by an embodiment of the present invention;

[0034] Figure 4 This is a schematic diagram of a positioning device based on a point cloud semantic map provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0035] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention are within the scope of protection of the present invention.

[0036] Example 1

[0037] refer to Figure 1 This embodiment discloses a positioning method based on a point cloud semantic map, which includes the following steps:

[0038] refer to Figure 2 , Figure 2 (a) is a point cloud map, where Figure 2 (a) There is a door on the upper left. Figure 2 (b) is the point cloud semantic map. Figure 2 (b) The corresponding door in the upper left corner is represented by green. The point cloud semantic map is modified by assigning a specific color to each point cloud to make the point cloud map more realistically reflect the actual scene. Secondly, specific targets (i.e., point cloud semantics) are manually labeled on the point cloud map and recorded as a semantic information table. The semantic information table includes the semantic category, the size of the semantic, and all the points included in the semantic.

[0039] The semantics of this embodiment are generally fixed objects, such as doors, buildings, etc., which can avoid interference from dynamic objects.

[0040] The robot of this embodiment is equipped with a laser radar or millimeter-wave radar, and a visual sensor, wherein the visual sensor is used to acquire images in real time, and the laser radar or millimeter-wave radar is used to acquire point cloud data. More specifically, the robot of this embodiment also includes a processing unit and a chassis, wherein the processing unit can receive instructions or control the robot's chassis to move autonomously based on data from the laser radar or millimeter-wave radar. Those skilled in the art will appreciate that the robot of this embodiment also includes a power supply unit for providing power to the robot, and the specific power supply unit can be a lithium-ion battery, etc. More generally, this embodiment also includes other components for realizing the robot's functions, which are not detailed in this embodiment.

[0041] Generally, the robot of this embodiment establishes a semantic map for the walking area, and the semantic map can be a point cloud map using the SLAM algorithm, and specific objects are manually marked with colors and semantic information tables on the point cloud semantic map to form a point cloud semantic map.

[0042] Specifically, the identification of specific objects in the point cloud map can be performed entirely manually, for example, by displaying the point cloud map and manually annotating the specific objects in the point cloud map with semantics. In another embodiment, a deep learning-based object detection algorithm can be used to identify specific objects from the point cloud map and annotate the locations of the specific objects in the point cloud map, thereby assisting in the manual generation of a semantic information table.

[0043] YOLO is an image object detection algorithm proposed in 2016 and published at the Computer Vision and Pattern Recognition (CVPR) conference. YOLO stands for "you only look once," meaning it can identify the category and location of objects in an image with just one glance. In this example, YOLO can be used to perform object recognition on point cloud maps.

[0044] S1, obtains the real-time image acquired by the robot in real time;

[0045] The real-time image in this embodiment is obtained by a visual sensor.

[0046] S2, using a target detection algorithm to identify the target object in the real-time image;

[0047] This example uses YOLO to detect objects on a point cloud map. YOLO outputs the category and location of the detected objects within the image. The real-time image is input into the object detection algorithm (YOLO), which detects objects a, b, and c. This example uses the detection of three objects as an example.

[0048] S3, obtaining the semantics corresponding to the target object from the semantic information table in the semantic map;

[0049] Specifically, in this embodiment, when the robot is started, the pre-established point cloud semantic map will be automatically loaded.

[0050] Specifically, according to the categories of the identified targets a, b, c, the semantic information A, B, C corresponding to the categories of the targets a, b, c are searched from the semantic information table of the semantic map.

[0051] The specific semantic information table records the semantic category, position, size, color, etc.

[0052] S4, reprojecting all semantic point clouds corresponding to the target object according to the positioning result to obtain a reprojected image;

[0053] The positioning result is represented by a [R|t]3X4 matrix, which contains the rotation information R and translation information t.

[0054] Formula (1) is the expression of [R|t], which is the quantity to be determined.

[0055]

[0056] All point clouds of semantics A, B, and C corresponding to the target object are reprojected. Taking semantic A as an example; assuming that semantic A contains N point clouds p = [xyz] T

[0057]

[0058]

[0059]

[0060] The subscript Ai represents the i-th point cloud in the semantic A; the superscripts 1 and 2 represent intermediate variables for differentiation; x , fy, u0, v0 are the internal parameters of the camera, factory inherent parameters, fx, fy are focal lengths, u0, v0 are the main point coordinates (relative to the imaging plane), u Ai , v Ai is the size of the i-th point cloud in semantic A in the image pixel coordinate system.

[0061] S5, constructing a positioning result optimization cost function based on the reprojected image and the target object acquired in real time;

[0062] The variable optimization cost function is constructed by the semantic target (abc) associated with the point cloud semantic map (ABC) and the real-time image:

[0063]

[0064] Among them, pixel(u Ai ,v Ai ) means getting the pixel value of the i-th point cloud in semantic A, because each point in the point cloud semantics already has a color; pixel(u ai ,v ai ) indicates obtaining the pixel value of the i-th point cloud in the image target a;

[0065] S6. Obtain a rough value of the positioning result [R|t] given by the wheel odometer, and continuously adjust the positioning result [R|t] value using the Newton-Gauss method until the positioning result optimization cost function Err([R|t]) reaches a minimum, thereby obtaining the optimal positioning result [R|t];

[0066] The wheel odometer gives a rough value of [R|t], and the Newton-Gauss method is used to continuously adjust the [R|t] value until Err([R|t]) reaches the minimum, thus obtaining the optimal [R|t];

[0067] This embodiment uses an image neural network algorithm to identify targets in real-time images and can robustly detect targets. In addition, this embodiment uses point cloud semantics for positioning (point cloud semantics are generally some fixed objects) to avoid interference from dynamic objects.

[0068] Example 2

[0069] refer to Figure 2 This embodiment discloses a positioning device based on a point cloud semantic map, which includes the following units:

[0070] refer to Figure 2 , Figure 2 (a) is a point cloud map, where Figure 2 (a) There is a door on the upper left. Figure 2 (b) is the point cloud semantic map. Figure 2 (b) The corresponding door in the upper left corner is represented by green. The point cloud semantic map is modified by assigning a specific color to each point cloud to make the point cloud map more realistically reflect the actual scene. Secondly, specific targets (i.e., point cloud semantics) are manually labeled on the point cloud map and recorded as a semantic information table. The semantic information table includes the semantic category, the size of the semantic, and all the points included in the semantic.

[0071] The semantics of this embodiment are generally fixed objects, such as doors, buildings, etc., which can avoid interference from dynamic objects.

[0072] The robot of this embodiment is equipped with a laser radar or millimeter-wave radar, and a visual sensor, wherein the visual sensor is used to acquire images in real time, and the laser radar or millimeter-wave radar is used to acquire point cloud data. More specifically, the robot of this embodiment also includes a processing unit and a chassis, wherein the processing unit can receive instructions or control the robot's chassis to move autonomously based on data from the laser radar or millimeter-wave radar. Those skilled in the art will appreciate that the robot of this embodiment also includes a power supply unit for providing power to the robot, and the specific power supply unit can be a lithium-ion battery, etc. More generally, this embodiment also includes other components for realizing the robot's functions, which are not detailed in this embodiment.

[0073] Generally, the robot of this embodiment establishes a semantic map for the walking area, and the semantic map can be a point cloud map using the SLAM algorithm, and specific objects are manually marked with colors and semantic information tables on the point cloud semantic map to form a point cloud semantic map.

[0074] Specifically, the identification of specific objects in the point cloud map can be performed entirely manually, for example, by displaying the point cloud map and manually annotating the specific objects in the point cloud map with semantics. In another embodiment, a deep learning-based object detection algorithm can be used to identify specific objects from the point cloud map and annotate the locations of the specific objects in the point cloud map, thereby assisting in the manual generation of a semantic information table.

[0075] YOLO is an image object detection algorithm proposed in 2016 and published at the Computer Vision and Pattern Recognition (CVPR) conference. YOLO stands for "you only look once," meaning it can identify the category and location of objects in an image with just one glance. In this example, YOLO can be used to perform object recognition on point cloud maps.

[0076] A real-time image acquisition unit, used to acquire real-time images acquired by the robot in real time;

[0077] The real-time image in this embodiment is obtained by a visual sensor.

[0078] An object detection unit, configured to identify an object in the real-time image using an object detection algorithm;

[0079] This example uses YOLO to detect objects on a point cloud map. YOLO outputs the category and location of the detected objects within the image. The real-time image is input into the object detection algorithm (YOLO), which detects objects a, b, and c. This example uses the detection of three objects as an example.

[0080] A semantic acquisition unit, configured to acquire the semantics corresponding to the target object from a semantic information table in a semantic map;

[0081] Specifically, in this embodiment, when the robot is started, the pre-established point cloud semantic map will be automatically loaded.

[0082] Specifically, according to the categories of the identified targets a, b, c, the semantic information A, B, C corresponding to the categories of the targets a, b, c are searched from the semantic information table of the semantic map.

[0083] The specific semantic information table records the semantic category, position, size, color, etc.

[0084] A reprojection unit, configured to reproject all semantic point clouds corresponding to the target object according to the positioning result to obtain a reprojected image;

[0085] The positioning result is represented by a [R|t]3X4 matrix, which contains the rotation information R and translation information t.

[0086] Formula (1) is the expression of [R|t], which is the quantity to be determined.

[0087]

[0088] All point clouds of semantics A, B, and C corresponding to the target object are reprojected. Taking semantic A as an example; assuming that semantic A contains N point clouds p = [xyz] T

[0089]

[0090]

[0091]

[0092] The subscript Ai represents the i-th point cloud in the semantic A; the superscripts 1 and 2 represent intermediate variables for differentiation; x , fy, u0, v0 are the internal parameters of the camera, factory inherent parameters, fx, fy are focal lengths, u0, v0 are the main point coordinates (relative to the imaging plane), u Ai , v Ai is the size of the i-th point cloud in semantic A in the image pixel coordinate system.

[0093] A positioning result optimization cost function construction unit, configured to construct a positioning result optimization cost function using the reprojected image and the target object acquired in real time;

[0094] The variable optimization cost function is constructed by the semantic target (abc) associated with the point cloud semantic map (ABC) and the real-time image:

[0095]

[0096] Among them, pixel(u Ai ,v Ai ) means getting the pixel value of the i-th point cloud in semantic A, because each point in the point cloud semantics already has a color; pixel(u ai ,v ai ) represents obtaining the pixel value of the i-th point cloud in the image target a;

[0097] The iterative optimization unit is used to obtain the rough value of the positioning result [R|t] given by the wheel odometer, and continuously adjust the positioning result [R|t] value using the Newton-Gauss method until the positioning result optimization cost function Err([R|t]) reaches the minimum, thus obtaining the optimal positioning result [R|t];

[0098] The wheel odometer gives a rough value of [R|t], and the Newton-Gauss method is used to continuously adjust the [R|t] value until Err([R|t]) reaches the minimum, thus obtaining the optimal [R|t];

[0099] This embodiment uses an image neural network algorithm to identify targets in real-time images and can robustly detect targets. In addition, this embodiment uses point cloud semantics for positioning (point cloud semantics are generally some fixed objects) to avoid interference from dynamic objects.

[0100] Example 3

[0101] This embodiment discloses a robot, which includes a visual sensor, a processing unit, a chassis, and a storage unit. The storage unit stores instructions, and when the instructions are executed, it is used to implement the positioning method based on the semantic point cloud map as described in Example 1.

[0102] Specifically, the robot of this embodiment further includes a positioning device based on the semantic point cloud map as described in the second embodiment.

[0103] Example 4

[0104] refer to Figure 4 , Figure 4 is a schematic diagram of the structure of a positioning device based on a point cloud semantic map according to this embodiment. The positioning device 20 based on a point cloud semantic map according to this embodiment includes a processor 21, a memory 22, and a computer program stored in the memory 22 and executable on the processor 21. When the processor 21 executes the computer program, the steps of the aforementioned method embodiments are implemented. Alternatively, when the processor 21 executes the computer program, the functions of the modules / units in the aforementioned apparatus embodiments are implemented.

[0105] Exemplarily, the computer program can be divided into one or more modules / units, which are stored in the memory 22 and executed by the processor 21 to implement the present invention. The one or more modules / units can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the point cloud semantic map-based positioning device 20. For example, the computer program can be divided into the modules described in Example 2. The specific functions of each module can be referred to the operating process of the device described in the above embodiment and will not be repeated here.

[0106] The positioning device 20 based on the point cloud semantic map may include, but is not limited to, a processor 21 and a memory 22. Those skilled in the art will appreciate that the schematic diagram is merely an example of the positioning device 20 based on the point cloud semantic map and does not limit the positioning device 20 based on the point cloud semantic map. The positioning device 20 based on the point cloud semantic map may include more or fewer components than shown in the figure, or may combine certain components or different components. For example, the positioning device 20 based on the point cloud semantic map may also include input and output devices, network access devices, buses, etc.

[0107] The processor 21 can be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor can be a microprocessor or any conventional processor. The processor 21 is the control center of the positioning device 20 based on the point cloud semantic map, and uses various interfaces and lines to connect the various parts of the positioning device 20 based on the point cloud semantic map.

[0108] The memory 22 can be used to store the computer programs and / or modules. The processor 21 implements the various functions of the point cloud semantic map-based positioning device 20 by running or executing the computer programs and / or modules stored in the memory 22 and accessing the data stored in the memory 22. The memory 22 may primarily include a program storage area and a data storage area. The program storage area may store an operating system and at least one application required for a function (such as a sound playback function or an image playback function); the data storage area may store data generated based on the use of the mobile phone (such as audio data, a phone book, etc.). In addition, the memory 22 may include high-speed random access memory and non-volatile memory, such as a hard disk, internal memory, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, at least one disk storage device, a flash memory device, or other volatile solid-state storage device.

[0109] Wherein, if the modules / units integrated in the positioning device 20 based on the point cloud semantic map are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the present invention implements all or part of the processes in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor 21, it can implement the steps of the above-mentioned various method embodiments. Wherein, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium can include: any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practices in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practices, computer-readable media do not include electrical carrier signals and telecommunication signals.

[0110] It should be noted that the device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed across multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the present embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which may be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement the present invention without inventive effort.

[0111] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A positioning method based on point cloud semantic map, comprising the following steps: S1, obtains the real-time image acquired by the robot in real time; S2, using a target detection algorithm to identify the target object in the real-time image; S3, obtaining the semantics corresponding to the target object from the semantic information table in the semantic map; S4, reprojecting all semantic point clouds corresponding to the target object according to the positioning result to obtain a reprojected image; specifically: Positioning results The matrix representation contains the rotation information R and translation information t; Matrix elements are the elements of the rotation matrix R, corresponding to the three-dimensional rotation parameters; and is the element of the translation vector t, corresponding to the three-dimensional translation parameter; formula (1) is The expression is the quantity to be determined; （1） All point clouds of semantics A, B, and C corresponding to the target object are reprojected. Taking semantics A as an example; assuming that semantics A contains N point clouds , where x, y, and z are the coordinates of the point cloud in the three-dimensional world coordinate system; ； ； ； in It is the coordinate of the point cloud in the camera coordinate system after rotation and translation; is the normalized image plane coordinate; the subscript Ai represents the i-th point cloud in the semantic A; the superscripts 1 and 2 represent intermediate variables for distinction; among them, It is the internal parameter of the camera, the factory inherent parameter, fx, fy is the focal length, is the coordinate of the principal point of the imaging plane, is the size of the i-th point cloud in semantic A in the image pixel coordinate system; S5, constructing a positioning result optimization cost function based on the reprojected image and the target object acquired in real time, specifically: ； in Indicates obtaining the pixel value of the i-th point cloud in semantic A, because each point in the point cloud semantics already has a color; Indicates obtaining the pixel value of the i-th point cloud in the image target a; Indicates obtaining the pixel value of the i-th point cloud in semantic B, because each point in the point cloud semantics already has a color; Indicates obtaining the pixel value of the i-th point cloud in the image target b; Indicates obtaining the pixel value of the i-th point cloud in semantic C, because each point in the point cloud semantics already has a color; Indicates obtaining the pixel value of the i-th point cloud in the image object c; N is the total number of point clouds contained in semantic A, m is the total number of point clouds contained in semantic B, and K is the total number of point clouds contained in semantic C; S6, obtain the positioning result given by the wheel odometer The rough value of the positioning result is continuously adjusted using the Newton-Gauss method. value until the positioning result optimizes the cost function Reach the minimum and get the best positioning result .

2. The positioning method according to claim 1, wherein the target detection algorithm in step S2 is YOLO.

3. The positioning method according to claim 1, wherein the semantic information table records the category, position, and size of the semantics.

4. A positioning device corresponding to the positioning method based on point cloud semantic map according to any one of claims 1 to 3, characterized in that: Includes the following units: A real-time image acquisition unit, used to acquire real-time images acquired by the robot in real time; An object detection unit, configured to identify an object in the real-time image using an object detection algorithm; A semantic acquisition unit, configured to acquire the semantics corresponding to the target object from a semantic information table in a semantic map; A reprojection unit, configured to reproject all semantic point clouds corresponding to the target object according to the positioning result to obtain a reprojected image; A positioning result optimization cost function construction unit, configured to construct a positioning result optimization cost function using the reprojected image and the target object acquired in real time; Iterative optimization unit, used to obtain the positioning results given by the wheel odometer The rough value of the positioning result is continuously adjusted using the Newton-Gauss method. value until the positioning result optimizes the cost function Reach the minimum and get the best positioning result .

5. The positioning device according to claim 4, wherein the target detection algorithm in the target detection unit is YOLO. The positioning device according to claim 4 , wherein the semantic information table records the category, position, and size of the semantics.

7. A non-volatile memory having instructions stored thereon, wherein when the instructions are executed by a processor, the instructions are used to implement a positioning method based on a point cloud semantic map as described in any one of claims 1 to 3.

8. A robot comprising a visual sensor, a chassis, and a positioning device based on a point cloud semantic map as claimed in any one of claims 4 to 6.

Images