Robot positioning method and device, electronic equipment and program product

Abstract

The invention is suitable for the technical field of robots, and provides a robot positioning method and device, electronic equipment and a program product. Wherein the robot is provided with a visual sensor and a laser sensor; the robot positioning method comprises the following steps: acquiring an electronic map; determining image information of semantic elements based on image data provided by the visual sensor; matching the image information of the semantic elements in the electronic map to obtain a positioning standard of the robot; determining a first positioning result of the robot based on the point cloud data provided by the laser sensor; and based on the positioning standard, correcting the first positioning result to obtain a target positioning result of the robot. According to the embodiment of the invention, the accuracy of robot positioning can be improved.

Description

Technical Field

[0001] The present application belongs to the field of robotics technology, and in particular relates to a robot positioning method, device, electronic equipment and program product. Background Art

[0002] Existing robot positioning solutions primarily rely on sensors such as 3D LiDAR, Inertial Measurement Units (IMUs), and odometry (Odom). However, 3D LiDAR cannot provide semantic information, resulting in a large amount of information waste and insufficient positioning accuracy. Summary of the Invention

[0003] The embodiments of the present application provide a robot positioning method, device, electronic device and program product, which can improve positioning accuracy.

[0004] A first aspect of an embodiment of the present application provides a robot positioning method, wherein the robot is provided with a visual sensor and a laser sensor; the robot positioning method includes: obtaining an electronic map; determining image information of semantic elements based on image data provided by the visual sensor; matching the image information of the semantic elements in the electronic map to obtain a positioning standard for the robot; determining a first positioning result of the robot based on point cloud data provided by the laser sensor; and correcting the first positioning result based on the positioning standard to obtain a target positioning result of the robot.

[0005] In some embodiments of the first aspect, matching the image information of the semantic element in the electronic map to obtain the positioning standard of the robot includes: when the semantic element is a lane line, matching the lane line in the electronic map to obtain the positioning standard of the robot in the first coordinate system direction; when the semantic element is a ground marker, matching the ground marker in the electronic map to obtain the positioning standard of the robot in the second coordinate system direction, wherein the second coordinate system direction is perpendicular to the first coordinate system direction; when the semantic element is a parking space, matching the parking space in the electronic map to obtain the range positioning standard of the robot.

[0006] In some embodiments of the first aspect, determining the first positioning result of the robot based on the point cloud data provided by the laser sensor includes: matching the point cloud data with the electronic map to obtain an initial laser positioning result; determining a second positioning result based on the image data; using the second positioning result as a predicted value and the initial laser positioning result as an observed value, and correcting the second positioning result to obtain the first positioning result.

[0007] In some embodiments of the first aspect, the robot is provided with an inertial navigation odometer; determining the second positioning result based on the image data includes: extracting feature points based on the image data, matching feature points between two adjacent frames of image data, and determining the visual initial positioning result based on the successfully matched feature point pairs; obtaining a third positioning result of the inertial navigation odometer; using the third positioning result as a predicted value and the visual initial positioning result as an observed value, and correcting the third positioning result to obtain the second positioning result.

[0008] In some embodiments of the first aspect, the inertial odometer includes an odometer and an inertial measurement unit; obtaining the fourth positioning result of the inertial odometer includes: calculating the current position of the robot based on the odometer data of the odometer; determining the current posture based on the posture information of the inertial measurement unit; and obtaining the third positioning result based on the current position and the current posture.

[0009] In some embodiments of the first aspect, the positioning method of the robot further includes: if the laser sensor fails, the second positioning result is used as the target positioning result; if the visual sensor fails, the third positioning result is used as the predicted value, the initial laser positioning result is used as the observed value, the third positioning result is corrected to obtain a fourth positioning result, and the fourth positioning result is used as the target positioning result; if both the visual sensor and the laser sensor fail, the third positioning result is used as the target positioning result.

[0010] In some implementations of the first aspect, if the image information of the semantic element fails to be matched in the electronic map, the robot positioning method further includes: using the first positioning result as a target positioning result.

[0011] A second aspect of an embodiment of the present application provides a positioning device for a robot, wherein the robot is provided with a visual sensor and a laser sensor; the positioning device of the robot includes: an acquisition unit for acquiring an electronic map; a visual analysis unit for determining image information of a semantic element based on image data provided by the visual sensor; a positioning standard analysis unit for matching the image information of the semantic element in the electronic map to obtain the positioning standard of the robot; a positioning unit for determining a first positioning result of the robot based on point cloud data provided by the laser sensor; and a positioning correction unit for correcting the first positioning result based on the positioning standard to obtain a target positioning result of the robot.

[0012] A third aspect of an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the steps of the above-mentioned robot positioning method when executing the computer program.

[0013] A fourth aspect of an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above-mentioned robot positioning method are implemented.

[0014] A fifth aspect of the embodiments of the present application provides a computer program product, which, when the computer program is run, enables the above-mentioned robot positioning method to be executed.

[0015] In an embodiment of the present application, based on the image data provided by the visual sensor, the image information of the semantic element is determined, and the image information of the semantic element is matched in the electronic map to obtain the positioning standard of the robot. Subsequently, based on the point cloud data provided by the laser sensor, the first positioning result of the robot is determined, and based on the positioning standard, the first positioning result is corrected to obtain the target positioning result of the robot. The electronic map and the visual sensor can be used to provide a reference for semantic information for the positioning data of the laser sensor, thereby improving the accuracy of the robot positioning. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 This is a schematic diagram of the implementation flow of a robot positioning method provided in an embodiment of the present application;

[0018] Figure 2 is a schematic diagram of an electronic map provided in an embodiment of the present application;

[0019] Figure 3 This is a schematic diagram of a specific implementation process of positioning correction provided by an embodiment of the present application;

[0020] Figure 4 This is a schematic structural diagram of a robot positioning device provided in an embodiment of the present application;

[0021] Figure 5 It is a structural diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of this application more clear, the present application is further described in detail below with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without making any creative work are protected by this application.

[0023] Existing robot positioning solutions primarily rely on sensors such as 3D LiDAR, inertial measurement units, and odometry. However, 3D LiDAR cannot provide semantic information, resulting in a large amount of information waste and insufficient positioning accuracy.

[0024] In view of this, the present application proposes a robot positioning method that can use electronic maps and visual sensors to provide semantic information reference for the positioning data of laser sensors, thereby improving the accuracy of robot positioning.

[0025] In order to illustrate the technical solution of the present application, specific embodiments are provided below.

[0026] Figure 1 A schematic diagram of the implementation flow of a robot positioning method provided in an embodiment of the present application is shown, and the method can be applied to electronic devices.

[0027] Among them, the above-mentioned electronic device can be a computer, tablet computer, mobile phone, scheduling server and other control devices used to schedule robots, or it can be a robot that requires positioning and navigation, and this application does not impose any restrictions on this.

[0028] In an embodiment of the present application, the robot to be positioned may be provided with a visual sensor and a laser sensor. The visual sensor may refer to a visual odometry (VO), which is a sensor used to estimate the robot's posture change by detecting feature points in continuous frame images and calculating relative motion. The laser sensor may refer to a laser odometry (LO), which is a sensor used to calculate the relative motion of the robot in the environment through continuous laser scanning data.

[0029] In some embodiments of the present application, the robot may also be configured with an inertial navigation odometer, which may include an inertial measurement unit and an odometer.

[0030] In other embodiments of the present application, the robot may also be configured with other types of sensors, which is not limited in the present application.

[0031] Specifically, the above robot positioning method may include the following steps S101 to S105.

[0032] Step S101: Obtain an electronic map.

[0033] The electronic map records the map information of the area where the robot is located. Figure 2 The electronic map provided by the present application is shown, which can record map information of semantic elements such as lane lines, ground signs, parking spaces, etc. The so-called map information includes but is not limited to the map coordinates, names, and categories of semantic elements in the electronic map.

[0034] This application does not restrict the method or type of electronic map acquisition. Regarding acquisition, the electronic map can be a map pre-scanned by a robot or downloaded from the Internet. Regarding map type, the electronic map can be a raster map, a vector map, or the like.

[0035] Step S102: determining image information of semantic elements based on image data provided by the visual sensor.

[0036] In the embodiments of the present application, semantic elements represent graphic symbols or objects with specific meanings that appear in the robot's environment. Based on image data provided by a visual sensor, semantic elements can be extracted through object detection, and their image information can be determined. Object detection can be implemented using deep learning algorithms, contour extraction algorithms, and other methods. The image information determined for a semantic element may include, but is not limited to, its image coordinates, name, and classification.

[0037] Step S103 : Matching the image information of the semantic element in the electronic map to obtain the positioning standard of the robot.

[0038] Among them, the positioning standard represents the reasonable range of the robot.

[0039] In an embodiment of the present application, the image information of a semantic element can reflect the positional relationship between the semantic element and a visual sensor, and the visual sensor is provided on the robot, so it can reflect the positional relationship between the semantic element and the robot. By matching the image information of the semantic element in an electronic map, the semantic element appearing in the image data can be associated with the semantic element appearing in the electronic device, completing the pairing of the same semantic element. Based on the map coordinates of the semantic element in the electronic map and the positional relationship between the semantic element and the robot, the approximate position of the robot can be obtained, thereby obtaining a positioning standard.

[0040] Step S104: determining a first positioning result of the robot based on the point cloud data provided by the laser sensor.

[0041] The first positioning result is based on the data provided by the laser sensor. Specifically, based on the point cloud data collected by the laser sensor, the relative motion of the current frame relative to the previous frame can be determined through continuous frame detection. The current pose is determined by combining this relative motion with the accumulated pose of the previous frame, resulting in the robot's first positioning result.

[0042] Step S105: Based on the positioning standard, the first positioning result is corrected to obtain the target positioning result of the robot.

[0043] Among them, the target positioning result refers to the final positioning result of the robot, which can be used for navigation, information interaction and other functions.

[0044] Since the positioning standard represents the reasonable range of the robot, the first positioning result can be corrected based on the positioning standard, so that the first positioning result falls within the reasonable range represented by the positioning standard, and the target positioning result of the robot is obtained, avoiding the problem of positioning error caused by the first positioning result deviating from the reasonable range.

[0045] In an embodiment of the present application, based on the image data provided by the visual sensor, the image information of the semantic element is determined, and the image information of the semantic element is matched in the electronic map to obtain the positioning standard of the robot. Subsequently, based on the point cloud data provided by the laser sensor, the first positioning result of the robot is determined, and based on the positioning standard, the first positioning result is corrected to obtain the target positioning result of the robot. The electronic map and the visual sensor can be used to provide a reference for semantic information for the positioning data of the laser sensor, thereby improving the accuracy of the robot positioning.

[0046] In some embodiments of the present application, determining a first positioning result of the robot based on point cloud data provided by a laser sensor may include: matching the point cloud data with an electronic map to obtain an initial laser positioning result, determining a second positioning result based on the image data, using the second positioning result as a predicted value and the initial laser positioning result as an observed value, and correcting the second positioning result to obtain a first positioning result.

[0047] Specifically, by matching the point cloud data with the electronic map, the approximate position of the point cloud collected by the laser sensor in the electronic map can be determined, and the initial laser positioning result can be determined by referring to the relative motion between the previous and next frames.

[0048] Among them, the specific process of determining the initial laser positioning result may include: point cloud preprocessing, feature extraction, inter-frame matching, and pose estimation. Point cloud preprocessing refers to filtering noise points through voxel downsampling (Voxel Grid) to reduce the amount of calculation. Feature extraction refers to extracting geometric feature points in the laser point cloud, such as edge points and plane points. Inter-frame matching refers to matching the current frame point cloud with the previous frame point cloud through the ICP (Iterative Closest Point) or NDT (Normal Distributions Transform) method. Pose estimation refers to calculating the motion changes of the robot between two frames and accumulating them to obtain the current pose.

[0049] Preferably, when the robot is equipped with an inertial navigation odometer, the constraints of the inertial navigation odometer can be combined for optimization to reduce the cumulative error.

[0050] The second positioning result is a result obtained by positioning based on the data provided by the visual sensor. Specifically, the second positioning result can be determined by image recognition, front and back frame matching, etc., which is not limited in this application.

[0051] At this point, the second positioning result is used as the predicted value and the initial laser positioning result as the observed value. The second positioning result is corrected to obtain the first positioning result. Specifically, the pose in the second positioning result is used as the predicted value and the pose in the initial laser positioning result as the observed value. The observation residual between the observed and predicted values ​​is calculated, and the predicted value is then adjusted based on the Kalman gain to complete the pose correction and obtain the first positioning result.

[0052] In some embodiments of the present application, the robot is provided with an inertial navigation odometer; determining the second positioning result based on the image data may include: extracting feature points based on the image data, matching feature points between two adjacent frames of image data, and determining the visual initial positioning result based on the successfully matched feature point pairs; obtaining the third positioning result of the inertial navigation odometer; using the third positioning result as a predicted value and the visual initial positioning result as an observed value, and correcting the third positioning result to obtain a second positioning result.

[0053] Specifically, the process of determining the initial visual positioning result can specifically include: feature extraction, feature matching, and pose estimation. Feature extraction refers to extracting feature points from two adjacent frames of image data. The feature points can be scale-invariant features (SIFT), accelerated robust features (SURF), or ORB (Oriented FAST and Rotated Brief) features. Feature matching refers to matching the same feature points between two adjacent frames of image data to form feature point pairs. Pose estimation refers to calculating the relative motion of the camera using PnP (Perspective-n-Point) or optical flow method, and accumulating the current pose.

[0054] Similarly, the constraints of the inertial navigation odometry can be combined, or the cumulative error can be optimized based on the bundle adjustment (BA) algorithm.

[0055] The third positioning result is based on the data provided by the inertial odometry. Using the third positioning result as the predicted value and the initial visual positioning result as the observed value, the third positioning result is corrected to obtain the second positioning result. Specifically, the pose in the third positioning result is used as the predicted value, and the pose in the initial visual positioning result is used as the observed value. The observation residual between the observed and predicted values ​​is calculated, and the predicted value is then adjusted based on the Kalman gain to complete the pose correction and obtain the second positioning result.

[0056] In some embodiments of the present application, the inertial navigation odometry may include an odometer and an inertial measurement unit. In this case, obtaining the fourth positioning result from the inertial navigation odometry may include: calculating the robot's current position based on odometer data from the odometer; determining the current posture based on posture information from the inertial measurement unit; and obtaining the third positioning result based on the current position and current posture.

[0057] Specifically, since the odometer and the inertial measurement unit each have errors, this application adopts sensor fusion technology to combine the advantages of the two to improve positioning accuracy. The inertial measurement unit can provide high-frequency attitude information (yaw angle, roll angle, pitch angle). The odometer can provide odometer data (driving distance information). Based on the odometer data of the odometer, the incremental displacement can be calculated to predict the current position of the robot. The initial attitude is corrected using the attitude information (angular velocity) of the inertial measurement unit to obtain the current attitude. By fusing the current position calculated by the odometer and the current attitude provided by the inertial measurement unit, a third positioning result can be obtained. Preferably, the noise effect can be corrected based on Kalman Filter (KF) or Extended Kalman Filter (EKF).

[0058] Figure 3 The following is a schematic diagram of the specific implementation process of determining the first positioning result of the present application. First, the third positioning result determined based on the inertial navigation odometer is input into the visual module as a predicted value. The visual module can determine the visual initial positioning result based on the image data. The visual initial positioning result is used as the observation value, and the third positioning result is corrected to obtain the second positioning result. The second positioning result is input into the laser module as the predicted value. The laser module extracts the point cloud and matches it with the electronic map to obtain the laser initial positioning result as the observation value, and determines the second positioning result based on the image data. The laser initial positioning result is used as the observation value and the second positioning result is corrected to obtain the first positioning result.

[0059] Visual sensors can provide high-precision motion estimation and are suitable for environments with rich textures; laser odometry is less affected by lighting and can still work stably in low-texture environments (such as basements); inertial odometry can provide attitude information even when the tires slip, which can reduce the impact of errors; based on three different sensors, it can provide accurate positioning results in different scenarios.

[0060] After obtaining the first positioning result, the image information of the semantic element can be matched in the electronic map to obtain the positioning standard of the robot and perform correction.

[0061] Specifically, the image information of the semantic element is matched in the electronic map to obtain the robot's positioning criteria, which may include one or more of the following:

[0062] 1. When the semantic element is a lane line, the lane line is matched in the electronic map to obtain the positioning standard of the robot in the direction of the first coordinate system;

[0063] 2. When the semantic element is a ground marker, the ground marker is matched in the electronic map to obtain the positioning standard of the robot in the direction of the second coordinate system;

[0064] 3. When the semantic element is a parking space, the parking space is matched in the electronic map to obtain the range positioning standard of the robot.

[0065] The second coordinate system direction is perpendicular to the first coordinate system direction, the first coordinate system direction may be horizontal, and the second coordinate system direction may be vertical.

[0066] Please continue to refer to Figure 3 Matching lane lines with the global lane lines in the electronic map provides horizontal positioning standards; matching ground signs with the global ground signs in the electronic map provides vertical positioning standards; matching parking spaces with the global parking spaces in the electronic map provides range positioning standards. Subsequently, based on the positioning standards, if the first positioning result falls within the reasonable range represented by the positioning standards, the first positioning result is used as the target positioning result. If the first positioning result does not fall within the reasonable range represented by the positioning standards, the first positioning result can be corrected to obtain the target positioning result.

[0067] For example, if the first positioning result is (9,18,0), the horizontal positioning standard is 9.2, the vertical positioning standard is 17.9, and the range positioning standard is [(0,0), (0,20), (20,0), (20,20)], then the first positioning result will be corrected to (9.2,17.9,0), so the target positioning result is (9.2,17.9,0).

[0068] Of course, in some embodiments of the present application, if the difference between the first positioning result and the positioning standard exceeds the difference threshold, it means that there is a significant difference in the positioning result at this time, and the robot may have a fault. At this time, an abnormal prompt or abnormal processing can be performed.

[0069] The above-mentioned faults may specifically include sensor abnormality or semantic element matching failure.

[0070] Specifically, in some embodiments of the present application, the robot positioning method may further include: if the laser sensor fails, using the second positioning result as the target positioning result.

[0071] If the visual sensor fails, the third positioning result is used as the predicted value, the initial laser positioning result is used as the observed value, the third positioning result is corrected to obtain the fourth positioning result, and the fourth positioning result is used as the target positioning result.

[0072] If both the visual sensor and the laser sensor fail, the third positioning result will be used as the target positioning result.

[0073] Failure can refer to a fault in the sensor itself (e.g., abnormal current or voltage) or an abnormality in the sensor's positioning calculation process (e.g., inability to complete feature point matching). If the laser sensor fails, the system can be relegated to a fusion positioning method using an inertial navigation odometer and a visual sensor. If the visual sensor fails, the system can be relegated to a fusion positioning method using an inertial navigation odometer and a laser sensor. If both the visual sensor and the laser sensor fail, the system can be relegated to an inertial navigation odometer.

[0074] In other embodiments of the present application, if the image information of the semantic element fails to match the electronic map, the robot positioning method may further include: using the first positioning result as the target positioning result. In other words, the robot is reduced to a fusion positioning method using the inertial odometry, visual sensor, and laser sensor, eliminating the need to correct the positioning result.

[0075] In this way, the robot can obtain more accurate positioning results in different environments and scenarios.

[0076] It should be noted that, for the sake of simplicity of description, the aforementioned method embodiments are all expressed as a series of action combinations. However, those skilled in the art should be aware that this application is not limited to the described order of actions, because according to this application, certain steps can be performed in other orders.

[0077] like Figure 4 Shown is a schematic structural diagram of a robot positioning device 400 provided in an embodiment of the present application, wherein the robot positioning device 400 is configured on an electronic device.

[0078] Specifically, the robot positioning device 400 may include:

[0079] An acquisition unit 401 is used to acquire an electronic map;

[0080] A visual analysis unit 402 is configured to determine image information of semantic elements based on image data provided by the visual sensor;

[0081] A positioning standard analysis unit 403 is configured to match the image information of the semantic element in the electronic map to obtain a positioning standard for the robot;

[0082] a positioning unit 404, configured to determine a first positioning result of the robot based on the point cloud data provided by the laser sensor;

[0083] The positioning correction unit 405 is used to correct the first positioning result based on the positioning standard to obtain the target positioning result of the robot.

[0084] In some embodiments of the present application, the positioning standard analysis unit 403 can be specifically used to: when the semantic element is a lane line, match the lane line in the electronic map to obtain the positioning standard of the robot in the first coordinate system direction; when the semantic element is a ground mark, match the ground mark in the electronic map to obtain the positioning standard of the robot in the second coordinate system direction, and the second coordinate system direction is perpendicular to the first coordinate system direction; when the semantic element is a parking space, match the parking space in the electronic map to obtain the range positioning standard of the robot.

[0085] In some embodiments of the present application, the positioning unit 404 can be specifically used to: match the point cloud data with the electronic map to obtain an initial laser positioning result; determine a second positioning result based on the image data; use the second positioning result as a predicted value and the initial laser positioning result as an observed value to correct the second positioning result to obtain the first positioning result.

[0086] In some embodiments of the present application, the positioning unit 404 can be specifically used to: extract feature points based on the image data, perform feature point matching between two adjacent frames of image data, and determine the visual initial positioning result based on the successfully matched feature point pairs; obtain the third positioning result of the inertial navigation odometer; use the third positioning result as a predicted value and the visual initial positioning result as an observed value, correct the third positioning result, and obtain the second positioning result.

[0087] In some embodiments of the present application, the positioning unit 404 can be specifically used to: calculate the current position of the robot based on the odometer data of the odometer; determine the current posture based on the posture information of the inertial measurement unit; and obtain the third positioning result based on the current position and the current posture.

[0088] In some embodiments of the present application, the positioning unit 404 can be specifically used to: if the laser sensor fails, use the second positioning result as the target positioning result; if the visual sensor fails, use the third positioning result as the predicted value, and the initial laser positioning result as the observed value, correct the third positioning result to obtain a fourth positioning result, and use the fourth positioning result as the target positioning result; if both the visual sensor and the laser sensor fail, use the third positioning result as the target positioning result.

[0089] In some embodiments of the present application, the positioning unit 404 may be specifically configured to: if the image information of the semantic element fails to be matched in the electronic map, use the first positioning result as the target positioning result.

[0090] It should be noted that for the convenience and simplicity of description, the specific working process of the positioning device 400 of the above robot can be referred to Figures 1 to 3 The corresponding process of the method will not be described in detail here.

[0091] like Figure 5 , which is a schematic diagram of an electronic device provided in an embodiment of the present application. Specifically, the electronic device 5 may include: a processor 50, a memory 51, and a computer program 52 stored in the memory 51 and executable on the processor 50, such as a robot positioning program. When the processor 50 executes the computer program 52, the steps of the above-mentioned robot positioning method embodiments are implemented, such as Figure 1 Alternatively, when the processor 50 executes the computer program 52, the functions of the modules / units in the above-mentioned device embodiments are realized, for example, Figure 4 The functions of the acquisition unit 401, the visual analysis unit 402, the positioning standard analysis unit 403, the positioning unit 404 and the positioning correction unit 405 are shown.

[0092] The computer program may be divided into one or more modules / units, which are stored in the memory 51 and executed by the processor 50 to complete the present application. The one or more modules / units may be a series of computer program instruction segments capable of completing specific functions, and the instruction segments are used to describe the execution process of the computer program in the electronic device.

[0093] For example, the computer program can be divided into: an acquisition unit, a visual analysis unit, a positioning standard analysis unit, a positioning unit, and a positioning correction unit. The specific functions of each unit are as follows: the acquisition unit is used to acquire an electronic map; the visual analysis unit is used to determine the image information of the semantic element based on the image data provided by the visual sensor; the positioning standard analysis unit is used to match the image information of the semantic element with the electronic map to obtain the positioning standard of the robot; the positioning unit is used to determine the first positioning result of the robot based on the point cloud data provided by the laser sensor; the positioning correction unit is used to correct the first positioning result based on the positioning standard to obtain the target positioning result of the robot.

[0094] The electronic device may include, but is not limited to, a processor 50 and a memory 51. Those skilled in the art will understand that Figure 5 These are merely examples of electronic devices and do not constitute a limitation of the electronic device. The electronic device may include more or fewer components than shown in the figure, or a combination of certain components, or different components. For example, the electronic device may also include input and output devices, network access devices, buses, etc.

[0095] The processor 50 may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), off-the-shelf programmable gate arrays or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor.

[0096] The memory 51 may be an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device. The memory 51 may also be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device. Furthermore, the memory 51 may include both an internal storage unit of the electronic device and an external storage device. The memory 51 is used to store the computer program and other programs and data required by the electronic device. The memory 51 may also be used to temporarily store data that has been output or is about to be output.

[0097] It should be noted that, for the convenience and brevity of description, the structure of the above electronic device can also refer to the specific description of the structure in the method embodiment, which will not be repeated here.

[0098] Those skilled in the art can clearly understand that, for the convenience and brevity of description, only the division of the above-mentioned functional units and modules is used as an example for illustration. In actual applications, the above-mentioned functions can be distributed and completed by different functional units and modules as needed, that is, the internal structure of the device can be divided into different functional units or modules to complete all or part of the functions described above. The functional units and modules in the embodiment can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. In addition, the specific names of the functional units and modules are only for the convenience of distinguishing each other, and are not used to limit the scope of protection of this application. The specific working process of the units and modules in the above-mentioned system can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0099] In the above embodiments, the description of each embodiment has its own focus. For parts that are not described or recorded in detail in a certain embodiment, reference can be made to the relevant description of other embodiments.

[0100] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0101] In the embodiments provided in the present application, it should be understood that the disclosed devices / electronic devices and methods can be implemented in other ways. For example, the device / electronic device embodiments described above are merely schematic. For example, the division of the modules or units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0102] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0103] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.

[0104] If the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the process in the above-mentioned embodiment method, and can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium, and when the computer program is executed by the processor, it can implement the steps of the above-mentioned various method embodiments. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electric carrier signal, telecommunication signal and software distribution medium. It should be noted that the content contained in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, computer-readable media do not include electric carrier signals and telecommunication signals.

[0105] The above-described embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the various embodiments of the present application, and should all be included in the scope of protection of the present application.

Claims

1. A robot positioning method, characterized in that: The robot is provided with a visual sensor and a laser sensor; The positioning method of the robot includes: Get electronic maps; Determining image information of semantic elements based on image data provided by the visual sensor; Matching the image information of the semantic element in the electronic map to obtain a positioning standard for the robot; Determining a first positioning result of the robot based on the point cloud data provided by the laser sensor; Based on the positioning standard, the first positioning result is corrected to obtain a target positioning result of the robot.

2. The robot positioning method according to claim 1, wherein: The matching of the image information of the semantic element in the electronic map to obtain the positioning standard of the robot includes: When the semantic element is a lane line, matching the lane line in the electronic map to obtain a positioning standard for the robot in the direction of the first coordinate system; When the semantic element is a ground marker, the ground marker is matched in the electronic map to obtain a positioning standard of the robot in a second coordinate system direction, where the second coordinate system direction is perpendicular to the first coordinate system direction; When the semantic element is a parking space, the parking space is matched in the electronic map to obtain a range positioning standard for the robot.

3. The robot positioning method according to claim 1, wherein: Determining a first positioning result of the robot based on the point cloud data provided by the laser sensor includes: Matching the point cloud data with the electronic map to obtain an initial laser positioning result; determining a second positioning result based on the image data; The second positioning result is used as a predicted value and the initial laser positioning result is used as an observed value, and the second positioning result is corrected to obtain the first positioning result.

4. The robot positioning method according to claim 3, wherein: The robot is provided with an inertial navigation odometer; and determining a second positioning result based on the image data comprises: Extracting feature points based on the image data, performing feature point matching between two adjacent frames of image data, and determining an initial visual positioning result based on successfully matched feature point pairs; Obtaining a third positioning result of the inertial navigation odometer; The third positioning result is used as a predicted value, and the initial visual positioning result is used as an observed value, and the third positioning result is corrected to obtain the second positioning result.

5. The robot positioning method according to claim 4, characterized in that: The inertial navigation odometer includes an odometer and an inertial measurement unit; and obtaining a fourth positioning result of the inertial navigation odometer includes: Calculating a current position of the robot based on odometer data of the odometer; determining a current posture based on the posture information of the inertial measurement unit; The third positioning result is obtained based on the current position and the current posture.

6. The robot positioning method according to claim 5, characterized in that: The robot positioning method further includes: If the laser sensor fails, the second positioning result is used as the target positioning result; If the visual sensor fails, the third positioning result is used as a predicted value, the initial laser positioning result is used as an observed value, the third positioning result is corrected to obtain a fourth positioning result, and the fourth positioning result is used as the target positioning result; If both the visual sensor and the laser sensor fail, the third positioning result is used as the target positioning result.

7. The robot positioning method according to any one of claims 1 to 6, characterized in that: If the image information of the semantic element fails to match in the electronic map, the robot positioning method further includes: The first positioning result is used as the target positioning result.

8. A robot positioning device, characterized in that: The robot is provided with a visual sensor and a laser sensor; the positioning device of the robot includes: An acquisition unit, used for acquiring an electronic map; a visual analysis unit, configured to determine image information of semantic elements based on image data provided by the visual sensor; a positioning standard analysis unit, configured to match the image information of the semantic element in the electronic map to obtain a positioning standard for the robot; a positioning unit, configured to determine a first positioning result of the robot based on the point cloud data provided by the laser sensor; A positioning correction unit is used to correct the first positioning result based on the positioning standard to obtain a target positioning result of the robot.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the steps of the robot positioning method according to any one of claims 1 to 7 are implemented.

10. A computer program product, characterized in that The invention comprises a computer program, which enables the robot positioning method according to any one of claims 1 to 7 to be executed when the computer program is run.

Images