Positioning method, device, mobile robot and computer-readable medium

Abstract

Embodiments of the present application disclose a positioning method, apparatus, mobile robot, and computer-readable medium. Embodiments of the method include: determining first pose information of a laser positioning identifier based on data collected by a laser sensor; determining second pose information of a visual positioning identifier based on an image showing the visual positioning identifier collected by an image sensor, the first pose information, and the positional relationship between the laser positioning identifier and the visual positioning identifier; and determining target pose information of a combined identifier including the laser positioning identifier and the visual positioning identifier based on the first pose information and the second pose information. This implementation improves the positioning accuracy.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of computer technology, and specifically to a positioning method, apparatus, mobile robot, and computer-readable medium. Background Art

[0002] With the development of SLAM (Simultaneous Localization And Mapping) technology, high-precision positioning is required in more and more scenarios.

[0003] In the prior art, the traveling direction and distance of a mobile robot are usually determined by positioning a single type of positioning identifier (such as a laser positioning identifier or a visual positioning identifier). This positioning method has a large error, resulting in low positioning accuracy. Summary of the Invention

[0004] The embodiments of the present application propose a positioning method, apparatus, mobile robot, and computer-readable medium to solve the technical problem of low positioning accuracy in the prior art.

[0005] In a first aspect, the embodiments of the present application provide a positioning method, which includes: determining the first pose information of a laser positioning identifier based on the data collected by a laser sensor; determining the second pose information of the visual positioning identifier based on the image showing the visual positioning identifier collected by an image sensor, the first pose information, and the positional relationship between the laser positioning identifier and the visual positioning identifier; determining the target pose information of a combined identifier including the laser positioning identifier and the visual positioning identifier based on the first pose information and the second pose information.

[0006] In a second aspect, the embodiments of the present application provide a mobile robot, including: a laser sensor; an image sensor; one or more processors; a storage device storing one or more programs, which when executed by the one or more processors, cause the one or more processors to implement the method described in the first aspect.

[0007] In a third aspect, the embodiments of the present application provide a computer-readable medium storing a computer program, which when executed by a processor, implements the method described in the first aspect.

[0008] In a fourth aspect, the embodiments of the present application provide a computer program product, including a computer program or instruction, characterized in that when the computer program or instruction is executed by a processor, it implements the method described in the first aspect.

[0009] The positioning method, device, mobile robot, and computer-readable medium provided by the embodiments of the present application first determine the first pose information of the laser positioning identifier based on the data collected by the laser sensor; then determine the second pose information of the visual positioning identifier based on the image showing the visual positioning identifier collected by the image sensor, the first pose information, and the positional relationship between the laser positioning identifier and the visual positioning identifier; finally, determine the target pose information of the combined identifier including the laser positioning identifier and the visual positioning identifier based on the first pose information and the second pose information, so as to be able to perform the positioning of the visual positioning identifier by combining the laser positioning identifier and the visual positioning identifier simultaneously, eliminating the errors in the positioning of a single positioning identifier (such as the errors caused by laser distortion, low image resolution, etc.), and improving the positioning accuracy. Description of the Drawings

[0010] Other features, objects, and advantages of the present application will become more apparent by reading the detailed description of the non-limiting embodiments with reference to the following drawings:

[0011] Figure 1 is a flowchart of an embodiment of the positioning method according to the present application;

[0012] Figure 2 is a schematic diagram of a positional relationship between the laser positioning identifier and the visual positioning identifier of the positioning method according to the present application;

[0013] Figure 3 is a schematic diagram of another positional relationship between the laser positioning identifier and the visual positioning identifier of the positioning method according to the present application;

[0014] Figure 4 is a flowchart of another embodiment of the positioning method according to the present application;

[0015] Figure 5 is a schematic structural diagram of an embodiment of the positioning device according to the present application;

[0016] Figure 6 is a schematic structural diagram of a computer system of an electronic device for implementing the embodiments of the present application. Detailed Embodiments

[0017] The present application will be further described in detail below with reference to the drawings and embodiments. It can be understood that the specific embodiments described herein are only used to explain the related invention and are not intended to limit the invention. Additionally, it should be noted that for the sake of description, only parts related to the relevant invention are shown in the drawings.

[0018] It should be noted that, without conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the drawings and embodiments.

[0019] With the development of intelligent technologies such as the Internet of Things (IoT), artificial intelligence (AI), and big data, the demand for using these intelligent technologies to transform and upgrade the traditional logistics industry has become even stronger, and intelligent logistics (Intelligent Logistics System) has become a research hotspot in the logistics field. Intelligent logistics utilizes AI, big data, and various IoT devices and technologies such as information sensors, radio frequency identification (RFID) technology, and global positioning system (GPS), and is widely applied to basic activity links such as the transportation, warehousing, distribution, packaging, loading and unloading, and information services of materials, realizing intelligent analysis and decision-making, automated operation, and high-efficiency optimization management in the process of material management. IoT technologies include sensing devices, RFID technology, laser infrared scanning, infrared induction identification, etc. The IoT can effectively connect the materials in logistics with the network, can monitor the materials in real time, and can also sense environmental data such as the humidity and temperature of the warehouse to ensure the storage environment of the materials. Through big data technology, all data in logistics can be sensed, collected, uploaded to the data layer of the information platform, and operations such as filtering, mining, and analyzing the data are performed, and finally accurate data support is provided for business processes (such as transportation, warehousing, storage, picking, packaging, sorting, outbound, inventory, distribution, etc.). The application directions of AI in logistics can be roughly divided into two types: 1) using intelligent devices empowered by AI technology, such as driverless trucks, automated guided vehicles (AGVs), autonomous mobile robots (AMRs), forklifts, shuttle cars, stacker cranes, driverless delivery vehicles, drones, service robots, robotic arms, intelligent terminals, etc., to replace some manual labor; 2) improving the efficiency of manual labor through software systems driven by technologies or algorithms such as computer vision, machine learning, and operations research optimization, such as transportation equipment management systems, warehouse management, equipment scheduling systems, order allocation systems, etc. With the research and progress of intelligent logistics, this technology has been applied in many fields, such as retail and e-commerce, electronic products, tobacco, medicine, industrial manufacturing, footwear, textiles, food, etc.

[0020] In the field of intelligent logistics, the positioning of mobile robots is one of the key technologies. This application provides a positioning method that is beneficial to improving the positioning accuracy.

[0021] Please refer to Figure 1 , which shows the flow 100 of an embodiment of the positioning method according to this application. This positioning method includes the following steps:

[0022] Step 101, based on the data collected by the laser sensor, determine the first pose information of the laser positioning identifier.

[0023] In this embodiment, the execution subject of the positioning method may be an electronic device such as a mobile robot. The above-mentioned mobile robot may include, but is not limited to, devices capable of automatic positioning and movement such as AGV (Automated Guided Vehicle), AMR (Autonomous Mobile Robots), and shuttle vehicles.

[0024] In the scenario involved in this embodiment, laser positioning marks may be set. The laser positioning marks may be reflective markers with strong reflective characteristics, such as reflective strips, reflective plates, etc. Among them, according to the number of reflective markers, the laser positioning marks may also include, but are not limited to, at least one of the following: single reflective marker, double reflective marker. The single reflective marker may be a piece of reflective strip or a piece of reflective plate, etc. The double reflective marker may refer to two reflective strips or reflective plates placed in parallel, etc.

[0025] In this embodiment, the above-mentioned execution subject may be equipped with a laser sensor. The laser sensor may refer to a sensor that uses laser technology for measurement, such as a laser scanner, a lidar, etc. The data collected by the laser sensor may be laser data. The reflection intensity and distribution of laser points can be determined through the laser data. The above-mentioned execution subject may collect laser data in real time through the laser sensor, and may determine the position and pose of the laser positioning mark based on the reflection intensity and distribution of the laser points in the laser data collected by the laser sensor, so as to obtain the first pose information of the laser positioning mark.

[0026] Among them, the first pose information may be used to represent the pose of the laser positioning mark relative to the above-mentioned execution subject. The reference coordinate system of the first pose information may be a robot coordinate system, such as a three-dimensional rectangular coordinate system (including the x-axis, y-axis, and z-axis) established with the position where the mobile robot is located as the origin and based on the current running direction of the mobile robot. The first pose information may include, but is not limited to, information such as coordinates and yaw angle. It should be noted that the above-mentioned yaw angle is the angle of rotation around the z-axis. The above-mentioned coordinates may be the coordinates of a certain specified point in the laser positioning mark, and the specified point may be used as the positioning point. As an example, if the laser positioning mark is a single reflective marker, the specified point may be the center point of the single reflective marker. As another example, if the laser positioning mark is a double reflective marker, the specified point may be the midpoint of the connection line of the center points of the two reflective markers. In addition, other points may be specified according to needs, not limited to the examples listed above.

[0027] It should be noted that if the robot coordinate system is different from the laser sensor coordinate system, the pose information of the laser positioning mark in the laser sensor coordinate system can be first determined based on the above steps, and then the pose information can be converted into the pose information in the robot coordinate system by means of coordinate transformation, so as to obtain the first pose information.

[0028] It can be understood that due to the characteristics of the laser sensor and the material of the laser positioning mark, laser distortion is likely to occur. In addition, when the laser positioning mark is far from the laser sensor, due to the limited resolution of the laser sensor, the positioning accuracy is likely to be reduced. In addition, some laser sensors can only detect two-dimensional information and lack three-dimensional information. Therefore, only using the laser sensor to locate the laser positioning mark usually has low accuracy. Therefore, in the embodiment of the present application, the visual positioning mark is further located by the image sensor, and the positioning is carried out by combining the laser positioning result and the visual positioning result to make up for the defects of laser positioning, so as to improve the positioning accuracy.

[0029] Step 102: Determine the second pose information of the visual positioning mark based on the image showing the visual positioning mark collected by the image sensor, the first pose information, and the position relationship between the laser positioning mark and the visual positioning mark.

[0030] In this embodiment, the above execution entity can be equipped with an image sensor, such as a camera, a webcam, etc. The image sensor can use the photoelectric conversion function of the optoelectronic device to convert the optical image on the photosensitive surface into an electrical signal proportional to the optical image, so as to realize image acquisition.

[0031] In the scenario involved in this embodiment, a visual positioning mark can also be set. The visual positioning mark can be various easily recognizable markers, such as two-dimensional codes, markers in a common visual reference library (such as apriltag), or patterns with specific shapes, etc. The position relationship between the laser positioning mark and the visual positioning mark can be obtained in advance when the mark is set and stored in the above execution entity in advance. The above position relationship can be represented by the relative coordinates of a specified point in the visual positioning mark and a specified point in the laser positioning mark. Among them, the specified point can be used as a positioning point, and the specified point of the visual positioning mark can also be a pre-specified point such as the midpoint, which is not specifically limited here.

[0032] As an example, if the laser positioning mark is a single reflective marker, the position relationship between the laser positioning mark and the visual positioning mark can be referred to Figure 2 . As Figure 2 shown, the center of the laser positioning mark can be used as the origin to establish a rectangular coordinate system, and the coordinates of the center of the visual positioning mark in this coordinate system can be determined. The coordinates (such as Figure 2x1 and y1 in [reference] can be used to represent the positional relationship between the laser positioning marker and the visual positioning marker.

[0033] As another example, if the laser positioning marker is a double retroreflective marker, the positional relationship between the laser positioning marker and the visual positioning marker can be referred to Figure 3 . As Figure 2 shown, the midpoint of the line connecting the centers of the two retroreflective markers in the laser positioning marker can be used as the origin to establish a rectangular coordinate system, and the coordinates of the center of the visual positioning marker in this coordinate system can be determined. These coordinates (such as Figure 2 x1 and y1 in [reference]) can be used to represent the positional relationship between the laser positioning marker and the visual positioning marker.

[0034] In this embodiment, the above-mentioned execution entity can collect images in real time through an image sensor, and determine the second pose information of the visual positioning marker based on the image showing the visual positioning marker collected by the image sensor, the first pose information, and the positional relationship between the laser positioning marker and the visual positioning marker. Among them, the second pose information can be used to represent the pose of the visual positioning marker relative to the above-mentioned execution entity. The reference coordinate system of the second pose information can also be the robot coordinate system. The second pose information may include, but is not limited to, information such as coordinates, yaw angle, roll angle, and pitch angle. The above-mentioned roll angle is the angle of rotation around the x-axis, and the above-mentioned pitch angle is the angle of rotation around the y-axis. The above-mentioned coordinates can be the coordinates of a specified point in the visual positioning marker.

[0035] As an example, the above-mentioned execution entity can first determine the position of the visual positioning marker in the robot coordinate system through coordinate calculation based on the first pose information of the laser positioning marker in the robot coordinate system and the positional relationship between the laser positioning marker and the visual positioning marker. Then, the position of the visual positioning marker in the robot coordinate system is converted to the position in the image sensor coordinate system through coordinate system conversion, and further converted to the position of the visual positioning marker in the image collected by the image sensor. After that, using the position of the visual positioning marker in the image as a reference, the pose of the visual positioning marker in the image is visually recognized through the Github open-source algorithm or by calling the functions in the opencv library to obtain the pose information of the visual positioning marker in the image sensor coordinate system. Finally, the pose information is converted to the pose information in the robot coordinate system through coordinate system conversion to obtain the second pose information.

[0036] As another example, the above-mentioned execution entity may first determine the position of the visual positioning identifier in the laser coordinate system through coordinate calculation based on the pose information of the laser positioning identifier in the laser sensor coordinate system and the positional relationship between the laser positioning identifier and the visual positioning identifier. Then, through coordinate transformation, the position of the visual positioning identifier in the laser coordinate system is transformed into the position in the image sensor coordinate system, and further transformed into the position of the visual positioning identifier in the image captured by the image sensor. After that, taking the position of the visual positioning identifier in the image as a reference, the pose of the visual positioning identifier in the image is visually recognized in the same way as in the above example to obtain the pose information of the visual positioning identifier in the image sensor coordinate system. Finally, through coordinate transformation, the pose information is transformed into the pose information in the robot coordinate system, and the second pose information can be obtained.

[0037] Step 103: Based on the first pose information and the second pose information, determine the target pose information of the combined identifier including the laser positioning identifier and the visual positioning identifier.

[0038] In this embodiment, the above-mentioned execution entity may use the combination of the laser positioning identifier and the visual positioning identifier as the combined identifier, and determine the target pose information of the combined identifier based on the first pose information and the second pose information. Among them, since the positioning points of the laser positioning identifier and the visual positioning identifier are different (for example, the positioning point of the laser positioning identifier is located at the center of the laser positioning identifier, and the positioning point of the visual positioning identifier is located at the center of the laser positioning identifier. Because the laser positioning identifier and the visual positioning identifier are arranged at different positions, their positioning points are different), in order to fuse the pose information of the two, it is necessary to first select a point as the positioning point of the combined identifier, and transform the first pose information and the second pose information into the pose information of this point, so as to combine them to obtain the target pose information.

[0039] It should be noted that when determining the positioning point of the combined identifier, the positioning point of any one of the laser positioning identifier and the visual positioning identifier can be used as the positioning point, or another point can be selected as the positioning point, which is not specifically limited here. When combining the pose information, methods such as weighted summation can be used. This is not specifically limited here.

[0040] In some alternative implementation manners of this embodiment, before combining the pose information, the above-mentioned execution entity may first determine the positioning point of the combined identifier based on the type of the laser positioning identifier. As an example, if the laser positioning identifier is a single reflective marker, the center point of the laser positioning identifier may be used as the positioning point of the combined identifier. As another example, if the laser positioning identifier is a double reflective marker, the midpoint of the line connecting the center points of the two reflective markers may be used as the positioning point of the combined identifier. After determining the positioning point of the combined identifier, the first pose information and the second pose information are respectively converted into the pose information of this positioning point to obtain the third pose information and the fourth pose information. Specifically, the above-mentioned execution entity may determine the third pose information of the combined identifier based on the first pose information and the positional relationship (which can be represented by coordinates) between the laser positioning identifier and the positioning point. Similarly, the fourth pose information of the combined identifier can be determined based on the second pose information and the positional relationship (which can be represented by coordinates) between the visual positioning identifier and the positioning point. The above-mentioned third pose information is the pose information of the positioning point of the combined identifier obtained based on the first pose information. The above-mentioned fourth pose information is the pose information of the positioning point of the combined identifier obtained based on the second pose information. The coordinate information in the third pose information and the fourth pose information can be directly calculated based on the coordinate translation method. The angle information in the third pose information can be the angle information in the first pose information, and the angle information in the fourth pose information can be the angle information in the second pose information. After obtaining the third pose information and the fourth pose information, the above-mentioned execution entity may combine the two by means of weighted summation, etc., to obtain the target pose information of the combined identifier.

[0041] In some alternative implementation manners of this embodiment, the third pose information may include a first coordinate and a first yaw angle, and the fourth pose information includes a second coordinate, a second yaw angle, a roll angle, and a pitch angle. The above-mentioned execution entity may first determine the target coordinate of the visual positioning identifier based on the first coordinate and the second coordinate. Then, the first yaw angle and the second yaw angle are weighted and summed to obtain the target yaw angle. Finally, based on the target coordinate, the target yaw angle, the roll angle, and the pitch angle, the target pose information of the visual positioning identifier can be determined. Among them, the target pose information is the set of the above-mentioned coordinates and angles, that is, the target pose information may include the target coordinate, the target yaw angle, the roll angle, and the pitch angle.

[0042] Among them, when determining the target coordinates based on the first coordinate and the second coordinate, the distance threshold can be determined first based on the resolution of the laser sensor. Since the laser distortion degree is low and the measurement is more accurate when the distance is short, if the distance from the laser positioning mark to the laser sensor is less than the distance threshold, the first coordinate can be used as the target coordinate, so as to ensure that the obtained pose information has high accuracy. When the distance is large, the accuracy is reduced due to laser distortion. At this time, combining visual positioning can make up for the deficiency of laser positioning. Therefore, if the distance from the laser positioning mark to the laser sensor is greater than or equal to the distance threshold, the first coordinate and the second coordinate can be weighted and summed to obtain the target coordinate, thereby improving the positioning accuracy.

[0043] The method provided by the above embodiments of the present application first determines the first pose information of the laser positioning mark based on the data collected by the laser sensor; then determines the second pose information of the visual positioning mark based on the image collected by the image sensor, the first pose information, and the positional relationship between the laser positioning mark and the visual positioning mark; finally, determines the target pose information of the combined mark including the laser positioning mark and the visual positioning mark based on the first pose information and the second pose information, so as to be able to perform the positioning of the visual positioning mark by combining the laser positioning mark and the visual positioning mark at the same time, eliminating the errors in the positioning of a single positioning mark (such as the errors caused by laser distortion, low image resolution, etc.), and improving the positioning accuracy. In addition, it can prevent the misdetection of the positioning mark when using a single positioning method, and improve the robustness of pose detection.

[0044] In some optional embodiments, in step 101, the above execution subject can determine the first pose information of the laser positioning mark through the following sub-steps S11 to sub-step S12:

[0045] Sub-step S11, select laser points with a reflection intensity greater than the intensity threshold and a target number (which can be denoted as N) of adjacent points in sequence from the data collected by the laser sensor to obtain a point cloud.

[0046] Among them, the target number N can be preset or determined in real time. For example, it can be determined based on the resolution of the laser sensor and the size of the laser positioning mark. Here, len can represent the laser length, len R represents the length of the laser positioning mark, and θ R represents the resolution of the laser, then the target number N can be determined according to the following formula:

[0047]

[0048] Here, an index number can be set for each laser point in sequence, such as "index:1", "index:2", "index:3", etc. The above-mentioned execution entity can filter out N laser points with adjacent reflection intensities greater than the intensity threshold and record the index number of the starting laser point among the above laser points, such as "index:51". The above laser points can form a point cloud corresponding to the laser positioning identifier. For example, if the starting laser point is "index:51" and N is 5, the point cloud includes laser points with index numbers from "index:51" to "index:55".

[0049] It should be noted that if the laser positioning identifier is multiple reflective markers (such as double reflective markers), the point cloud corresponding to each reflective marker can be determined respectively. Further, if there are multiple groups of laser positioning identifiers in the scene and each group of laser positioning identifiers is a double reflective marker. At this time, the above-mentioned execution entity can first determine the point cloud corresponding to each reflective marker and the center of each point cloud in the above manner. Then, the distance between the centers of the point clouds is determined pairwise. If the distance between the centers of two point clouds is close to the distance between the two reflective markers in the double reflective marker (such as the difference is less than a preset value), it can be determined that these two point clouds are the point clouds corresponding to this group of double reflective markers.

[0050] Sub-step S12, based on the point cloud, position the laser positioning identifier to obtain the first pose information. Here, the attitude of the laser positioning identifier (such as the yaw angle) can be determined based on the distribution of laser points in the point cloud, and the position of the laser positioning identifier (such as coordinates) can be determined based on the position of the center point of the point cloud, so as to obtain the first pose information including the above yaw angle and the above coordinates.

[0051] In some alternative implementation manners, the above-mentioned execution entity can first fit a straight line based on the point cloud. Then, based on the angle of the straight line, determine the yaw angle of the laser positioning identifier. After that, based on the intersection points of the boundary of the point cloud and the straight line, determine the coordinates of the center point of the laser positioning identifier. Finally, based on the yaw angle and the coordinates of the center point, obtain the first pose information. The first pose information can include the yaw angle and the coordinates of the center point.

[0052] Among them, when fitting a straight line to determine the yaw angle, the above-mentioned execution entity can take a number of laser points on both sides of the laser positioning mark (for example, if the laser points with index numbers "index:51" to "index:55" are included in the point cloud, 5 laser points from "index:45" to "index:50" can be taken on the left side of the laser positioning mark, and 5 laser points from "index:56" to "index:60" can be taken on the right side of the laser positioning mark) for straight line fitting, and then expand to a certain distance (such as 10 cm) on both sides, thereby obtaining the fitted straight line. The angle of rotation of this straight line relative to the z-axis in the robot coordinate system is the yaw angle of the laser positioning mark.

[0053] Among them, when determining the coordinates of the center point of the laser positioning mark based on the intersection points of the boundary of the point cloud and this straight line, if the laser positioning mark is a single reflective marker, the above-mentioned execution entity can first determine the intersection points of the boundaries on the left and right sides of the laser positioning mark and this straight line respectively, obtaining two coordinate values, such as (x1, y1) and (x2, y2). Then, the coordinates (x, y) of the center point can be determined according to the following formula: (x, y) = (x1 + x2, y1 + y2) / 2. If the laser positioning mark is a double reflective marker, the above-mentioned execution entity can first determine the intersection points of the boundaries on the front, back, left, and right sides of the laser positioning mark and this straight line respectively, obtaining four coordinate values, such as (x1, y1), (x2, y2), (x3, y3), and (x4, y4). Then, the coordinates (x, y) of the center point can be determined according to the following formula: (x, y) = (x1 + x2 + x3 + x4, y1 + y2 + y3 + y4) / 4.

[0054] By selecting laser points with the number of sequentially adjacent targets based on the reflection intensity to obtain the point cloud, and the number of targets can be determined by the resolution of the laser sensor and the size of the laser positioning mark, the point cloud corresponding to the laser positioning mark can be accurately screened, thereby improving the accuracy of the first pose information.

[0055] In some optional embodiments, since the data collected by the laser sensor and the images collected by the image sensor may not be synchronized in time, after obtaining the first pose information, the first pose information can also be updated based on the time difference between the data collected by the laser sensor and the images collected by the image sensor and the current running speed of the above-mentioned execution entity, so that the updated first pose information can be directly applied to the images collected by the image sensor, thereby achieving data synchronization.

[0056] Specifically, after obtaining the first pose information (which can be denoted as After that, the above-mentioned execution entity can determine the pose change amount of the laser positioning identifier (which can be denoted as ) based on the current running speed of the mobile robot (which can include the linear velocity and the angular velocity, and can be denoted as vel), the first time when the data is collected by the laser sensor (which can be denoted as t1), and the second time when the image is collected by the image sensor (which can be denoted as t2), that is After that, the first pose information can be updated based on the pose change amount, that is

[0057] It should be noted that when the frequency of the images collected by the image sensor is less than the frequency of the laser data collected by the laser sensor, the images collected by the image sensor can be further interpolated to reduce the frequency gap.

[0058] By updating the first pose information through the above implementation method, the first pose information acting on the images collected by the image sensor can be synchronized with the time of the images collected by the image sensor, thereby improving the accuracy of the first pose information and further improving the accuracy of pose detection.

[0059] Further referring to Figure 4 , it shows the flow 400 of another embodiment of the positioning method. The flow 400 of this positioning method includes the following steps:

[0060] Step 401, based on the data collected by the laser sensor, determine the first pose information of the laser positioning identifier.

[0061] In this embodiment, the execution entity of the positioning method can be an electronic device such as a mobile robot. Step 401 can refer to Figure 1 Step 101 of the corresponding embodiment, which will not be elaborated here.

[0062] In some optional implementation manners of this embodiment, the above-mentioned laser positioning identifier includes at least one of the following types: single reflective marker, double reflective marker.

[0063] In some optional implementation manners of this embodiment, the first pose information can be determined through the following steps: select laser points with a reflection intensity greater than the intensity threshold and the target number of adjacent ones in sequence from the data collected by the laser sensor to obtain a point cloud; position the above-mentioned laser positioning identifier based on the above-mentioned point cloud to obtain the first pose information.

[0064] In some alternative implementation manners of this embodiment, the positioning of the above laser positioning identifier based on the above point cloud to obtain the first pose information may include: fitting a straight line based on the above point cloud; determining the yaw angle of the above laser positioning identifier based on the angle of the above straight line; determining the coordinates of the center point of the above laser positioning identifier based on the intersection point of the boundary of the above point cloud and the above straight line; and obtaining the first pose information based on the above yaw angle and the coordinates of the above center point.

[0065] In some alternative implementation manners of this embodiment, after obtaining the first pose information, the above execution entity may further determine the pose change amount of the above laser positioning identifier based on the current running speed of the above mobile robot, the first time when the above laser sensor collects the above data, and the second time when the above image sensor collects the above image; and update the above first pose information based on the above pose change amount. By updating the first pose information in the above implementation manner, the first pose information acting on the image collected by the image sensor can be synchronized with the time of the image collected by the image sensor, thereby improving the accuracy of the first pose information, and further improving the accuracy of pose detection.

[0066] It can be understood that due to the characteristics of the laser sensor and the material of the laser positioning mark, the situation of laser distortion is likely to occur. In addition, when the laser positioning mark is far from the laser sensor, due to the limited resolution of the laser sensor, the positioning accuracy is likely to be reduced. In addition, some laser sensors can only detect two-dimensional information and lack three-dimensional information. Therefore, only using the laser sensor to position the laser positioning mark usually has a low accuracy. Therefore, in the embodiment of the present application, the visual positioning identifier is further positioned by the image sensor, and the positioning is performed by combining the laser positioning result and the visual positioning result to make up for the defects of laser positioning, thereby improving the positioning accuracy.

[0067] Step 402: Determine the target area of the visual positioning identifier in the image collected by the image sensor based on the first pose information, the position relationship between the laser positioning identifier and the visual positioning identifier, the parameter information of the image sensor, and the parameter information of the visual positioning identifier.

[0068] In this embodiment, after the above-mentioned execution entity determines the first pose information of the laser positioning identifier, it may determine the target area of the visual positioning identifier in the image collected by the image sensor based on the first pose information, the positional relationship between the laser positioning identifier and the visual positioning identifier, the parameter information of the image sensor, and the parameter information of the visual positioning identifier. Among them, the parameter information of the image sensor may include internal parameters and external parameters. The internal parameters may include, but are not limited to, parameters such as the resolution of the image sensor. The external parameters may include, but are not limited to, at least one of the following: the positional relationship between the image sensor and the mobile robot, and the positional relationship between the image sensor and the laser sensor. The above-mentioned parameter information of the visual positioning identifier may include, but is not limited to, the size of the visual positioning identifier. The target area in the image collected by the image sensor may be the area where the visual positioning identifier is located, that is, the target area may include the visual positioning identifier.

[0069] In some alternative implementation manners of this embodiment, the above-mentioned visual positioning identifier includes, but is not limited to, a two-dimensional code.

[0070] In some alternative implementation manners of this embodiment, the target area of the visual positioning identifier in the image collected by the image sensor may be determined according to the following sub-steps S21 to sub-step S23:

[0071] Sub-step S21: Based on the first pose information, the positional relationship between the laser positioning identifier and the visual positioning identifier, and the parameter information of the image sensor, determine the position of the visual positioning identifier in the image sensor coordinate system. The parameter information used here may be external parameters, specifically including the positional relationship between the image sensor and the mobile robot. Specifically, first, based on the first pose information of the laser positioning identifier in the robot coordinate system and the positional relationship between the laser positioning identifier and the visual positioning identifier, the position of the visual positioning identifier in the robot coordinate system may be determined by means of coordinate calculation. Then, based on the positional relationship between the image sensor and the mobile robot, the conversion relationship between the image sensor coordinate system and the robot coordinate system may be determined, so that the position of the visual positioning identifier in the robot coordinate system is converted into the position in the image sensor coordinate system through coordinate system conversion, and further converted into the position of the visual positioning identifier in the image collected by the image sensor. Among them, the position may be represented by coordinates.

[0072] Sub-step S22: Based on the position of the visual positioning identifier in the image sensor coordinate system (which may be denoted as coordinates (x, y)), the size of the visual positioning identifier (the length may be denoted as len1, and the width may be denoted as len2), and the resolution of the image sensor (which may be denoted as R), determine the initial area of the visual positioning identifier in the image. The initial area may be represented by the coordinate values of four vertices, such as:

[0073]

[0074] Sub-step S23: Based on the preset error information, expand the initial region to obtain the target region in the image. The error information may include, but is not limited to, extraction error and fixed error. The extraction error may refer to the error caused by inaccurate acquisition of the QR code position, and the fixed error may refer to the error caused by the external parameters. Each error in the error information can be preset based on tests. The error can be characterized by the number of pixel points. The above-mentioned execution entity can expand the pixel points of the above-mentioned number of pixel points outward around the initial region to obtain the target region.

[0075] Determine the position of the visual positioning identifier in the image sensor coordinate system through the first pose information, the positional relationship between the laser positioning identifier and the visual positioning identifier, and the parameter information of the image sensor. Then, based on this position, determine the initial region of the visual positioning identifier in the image. Compared with directly recognizing the image and other methods, the recognition efficiency can be improved, thereby accelerating the acquisition of the pose information of the visual positioning identifier. In addition, by determining the initial region of the visual positioning identifier in the image and expanding the initial region to obtain the target region, it is possible to avoid situations such as missed detection caused by incomplete visual positioning identifiers in the initial region, and improve the success rate of positioning the visual positioning identifier.

[0076] Step 403: Perform intensity normalization on the pixel values in the target region to obtain the target region image.

[0077] In this embodiment, after determining the target region, the above-mentioned execution entity can perform intensity normalization on the pixel values in the target region to obtain the target region image. Among them, intensity normalization can be used to enhance the contrast of the image. Specifically, the average pixel value in the target region can be calculated first. Then, based on the comparison between the pixel value of each pixel point and the average pixel value, determine the pixel value scaling factor of each pixel point (that is, a coefficient used to multiply the pixel value). Then, based on this pixel value scaling factor, update the pixel value of each pixel point, so as to obtain the target region image with enhanced contrast.

[0078] As an example, if the average pixel value is 2, two thresholds can be set, such as the first threshold is 1 and the second threshold is 3. The pixel value scaling factors can include two, namely the first pixel value scaling factor (such as 0.5) and the second pixel value scaling factor (such as 2). If a certain pixel value (such as 1) is less than or equal to the first threshold (that is, 1), it can be multiplied by the first pixel value scaling factor (that is, 0.5) to obtain the updated pixel value 0.5. If another pixel value (such as 5) is greater than or equal to the second threshold (that is, 3), it can be multiplied by the second pixel value scaling factor (that is, 2) to obtain the updated pixel value 10. Thus, smaller pixel values can be further reduced, and larger pixel values can be further increased, achieving the effect of enhancing the contrast. The false extraction rate of the visual positioning identifier is reduced.

[0079] Step 404: Locate the visual positioning identifier based on the target area image to obtain the second pose information of the visual positioning identifier.

[0080] In this embodiment, the above-mentioned execution entity can locate the visual positioning identifier based on the target area image to obtain the second pose information of the visual positioning identifier. Here, through the Github open-source algorithm or by calling the functions in the opencv library, the pose of the visual positioning identifier in the image is visually recognized to obtain the pose information of the visual positioning identifier in the image sensor coordinate system. For example, the target area image can be used as the input, and parameters such as the size of the visual positioning identifier are input at the same time, and the pose information of the visual positioning identifier in the input image can be obtained. By means of coordinate system conversion, this pose information is converted into the pose information in the image sensor coordinate system and further converted into the pose information in the robot coordinate system, and then the second pose information can be obtained.

[0081] It can be understood that when the resolution of the image sensor is low, directly recognizing the visual positioning identifier from the original image is likely to result in a low recognition accuracy due to the small size of the visual positioning identifier in the image. When the resolution of the image sensor is large, directly recognizing the visual positioning identifier from the original image will result in a long time consumption due to the large amount of calculation. In this embodiment, by cropping the target area image of the target area and locating the visual positioning identifier based on the target area image, compared with locating the visual positioning identifier based on the original image, the visual positioning identifier can be detected in a small area, greatly improving the detection rate and success rate of the visual positioning identifier.

[0082] Step 405: Determine the target pose information of the combined identifier including the laser positioning identifier and the visual positioning identifier based on the first pose information and the second pose information.

[0083] Step 405 in this embodiment can refer to Step 103 in the above-mentioned embodiment, which will not be elaborated here.

[0084] In some optional implementation manners of this embodiment, the above-mentioned determining the target pose information of the combined identifier including the above-mentioned laser positioning identifier and the above-mentioned visual positioning identifier based on the above-mentioned first pose information and the above-mentioned second pose information may include: determining the positioning point of the combined identifier including the above-mentioned laser positioning identifier and the above-mentioned visual positioning identifier based on the type of the above-mentioned laser positioning identifier; determining the third pose information of the combined identifier based on the above-mentioned first pose information and the position relationship between the above-mentioned laser positioning identifier and the above-mentioned positioning point; determining the fourth pose information of the combined identifier based on the above-mentioned second pose information and the position relationship between the above-mentioned visual positioning identifier and the above-mentioned positioning point; determining the target pose information of the combined identifier based on the above-mentioned third pose information and the above-mentioned fourth pose information.

[0085] In some alternative implementation manners of this embodiment, the above third pose information includes a first coordinate and a first yaw angle, and the above fourth pose information includes a second coordinate, a second yaw angle, a roll angle, and a pitch angle; based on the above third pose information and the above fourth pose information, determining target pose information including a combined identifier of the above laser positioning identifier and the vision positioning identifier includes: determining a target coordinate of the vision positioning identifier based on the above first coordinate and the above second coordinate; performing weighted summation on the above first yaw angle and the above second yaw angle to obtain a target yaw angle; and determining the target pose information of the vision positioning identifier based on the above target coordinate, the above target yaw angle, the above roll angle, and the above pitch angle.

[0086] In some alternative implementation manners of this embodiment, when determining the target coordinate based on the first coordinate and the second coordinate, a distance threshold may be first determined based on the resolution of the laser sensor. Since the laser distortion degree is low and the measurement is more accurate when the distance is short, if the distance from the laser positioning identifier to the laser sensor is less than the distance threshold, the first coordinate may be used as the target coordinate, so as to ensure that the obtained pose information has high accuracy. When the distance is large, the accuracy is reduced due to laser distortion. At this time, combining vision positioning can make up for the deficiency of laser positioning. Therefore, if the distance from the laser positioning identifier to the laser sensor is greater than or equal to the distance threshold, weighted summation may be performed on the first coordinate and the second coordinate to obtain the target coordinate, thereby improving the positioning accuracy.

[0087] From Figure 4 it can be seen that compared with the corresponding embodiment of Figure 1 , the process 400 of the positioning method in this embodiment involves steps of performing intensity normalization on pixel values in a target area to obtain a target area image and positioning a vision positioning identifier based on the target area image to obtain second pose information of the vision positioning identifier. Thus, smaller pixel values in the target area can be further reduced, and larger pixel values can be further increased, achieving the effect of enhancing contrast. The false extraction rate of the vision positioning identifier is reduced. At the same time, compared with positioning the vision positioning identifier based on the original image, the vision positioning identifier can be detected in a small area, greatly improving the detection rate and success rate of the vision positioning identifier.

[0088] Further referring to Figure 5 , as an implementation of the methods shown in the above figures, an embodiment of a positioning device is provided in this application. This device embodiment corresponds to the method embodiment shown in Figure 1 , and this device can be specifically applied to various electronic devices.

[0089] As Figure 5As shown in the figure, the positioning device 500 of this embodiment includes: a laser positioning unit 501, configured to determine the first pose information of the laser positioning identifier based on the data showing the visual positioning identifier collected by the laser sensor; a visual positioning unit 502, configured to determine the second pose information of the visual positioning identifier based on the image collected by the image sensor, the first pose information, and the position relationship between the laser positioning identifier and the visual positioning identifier; and a result fusion unit 503, configured to determine the target pose information of the combined identifier including the laser positioning identifier and the visual positioning identifier based on the first pose information and the second pose information.

[0090] In some optional implementation manners of this embodiment, the laser positioning unit 501 is further configured to select laser points with a reflection intensity greater than the intensity threshold and a target number of adjacent points in sequence from the data collected by the laser sensor to obtain a point cloud; and perform positioning on the laser positioning identifier based on the point cloud to obtain the first pose information.

[0091] In some optional implementation manners of this embodiment, the laser positioning unit 501 is further configured to fit a straight line based on the point cloud; determine the yaw angle of the laser positioning identifier based on the angle of the straight line; determine the coordinates of the center point of the laser positioning identifier based on the intersection point of the boundary of the point cloud and the straight line; and obtain the first pose information based on the yaw angle and the coordinates of the center point.

[0092] In some optional implementation manners of this embodiment, the method further includes a synchronization unit, configured to, after obtaining the first pose information, further include: determining the pose change amount of the laser positioning identifier based on the current running speed of the mobile robot, the first time when the laser sensor collects the data, and the second time when the image sensor collects the image; and updating the first pose information based on the pose change amount.

[0093] In some optional implementation manners of this embodiment, the visual positioning unit 502 is further configured to determine the target area of the visual positioning identifier in the image collected by the image sensor based on the first pose information, the position relationship between the laser positioning identifier and the visual positioning identifier, the parameter information of the image sensor, and the parameter information of the visual positioning identifier; perform intensity normalization on the pixel values in the target area to obtain a target area image; and perform positioning on the visual positioning identifier based on the target area image to obtain the second pose information of the visual positioning identifier.

[0094] In some alternative implementation manners of this embodiment, the above-mentioned visual positioning unit 502 is further configured to determine the average pixel value within the target area; determine the pixel value scaling factor of each pixel point based on the comparison between the pixel value of each pixel point within the target area and the above-mentioned average pixel value; update the pixel value of each pixel point based on the pixel value scaling factor of each pixel point to obtain the target area image with enhanced contrast.

[0095] In some alternative implementation manners of this embodiment, the above-mentioned visual positioning unit 502 is further configured to determine the position of the above-mentioned visual positioning identifier in the image sensor coordinate system based on the above-mentioned first pose information, the positional relationship between the above-mentioned laser positioning identifier and the visual positioning identifier, and the parameter information of the above-mentioned image sensor; determine the initial area of the above-mentioned visual positioning identifier in the above-mentioned image based on the position of the above-mentioned visual positioning identifier in the image sensor coordinate system, the size of the above-mentioned visual positioning identifier, and the resolution of the above-mentioned image sensor; expand the above-mentioned initial area based on the preset error information to obtain the target area in the above-mentioned image.

[0096] In some alternative implementation manners of this embodiment, the above-mentioned result fusion unit 503 is further configured to determine the positioning point of the combined identifier including the above-mentioned laser positioning identifier and the visual positioning identifier based on the type of the above-mentioned laser positioning identifier; determine the third pose information of the above-mentioned combined identifier based on the above-mentioned first pose information and the positional relationship between the above-mentioned laser positioning identifier and the above-mentioned positioning point; determine the fourth pose information of the above-mentioned combined identifier based on the above-mentioned second pose information and the positional relationship between the above-mentioned visual positioning identifier and the above-mentioned positioning point; determine the target pose information of the above-mentioned combined identifier based on the above-mentioned third pose information and the above-mentioned fourth pose information.

[0097] In some alternative implementation manners of this embodiment, the above-mentioned third pose information includes a first coordinate and a first yaw angle, and the above-mentioned fourth pose information includes a second coordinate, a second yaw angle, a roll angle, and a pitch angle; the above-mentioned result fusion unit 503 is further configured to determine the target coordinate of the above-mentioned visual positioning identifier based on the above-mentioned first coordinate and the above-mentioned second coordinate; perform weighted summation on the above-mentioned first yaw angle and the above-mentioned second yaw angle to obtain the target yaw angle; determine the target pose information of the above-mentioned visual positioning identifier based on the above-mentioned target coordinate, the above-mentioned target yaw angle, the above-mentioned roll angle, and the above-mentioned pitch angle.

[0098] In some alternative implementation manners of this embodiment, the above result fusion unit 503 is further configured to determine a distance threshold based on the resolution of the above laser sensor; if the distance from the above laser positioning identifier to the above laser sensor is less than the above distance threshold, use the above first coordinate as the target coordinate; if the distance from the above laser positioning identifier to the above laser sensor is greater than or equal to the above distance threshold, perform weighted summation on the above first coordinate and the above second coordinate to obtain the target coordinate.

[0099] In some alternative implementation manners of this embodiment, the above laser positioning identifier includes at least one of the following types: single retroreflective marker, double retroreflective marker; the above visual positioning identifier includes a QR code.

[0100] The device provided in the above embodiment of this application first determines the first pose information of the laser positioning identifier based on the data collected by the laser sensor; then determines the second pose information of the visual positioning identifier based on the image collected by the image sensor, the first pose information, and the positional relationship between the laser positioning identifier and the visual positioning identifier; finally determines the target pose information of the combined identifier including the laser positioning identifier and the visual positioning identifier based on the first pose information and the second pose information, so as to be able to perform positioning of the visual positioning identifier by combining the laser positioning identifier and the visual positioning identifier at the same time, eliminating the errors in positioning a single positioning identifier (such as the errors caused by laser distortion, low image resolution, etc.), and improving the positioning accuracy.

[0101] Next, refer to Figure 6 , which shows a schematic structural diagram of an electronic device for implementing some embodiments of this application. Figure 6 The electronic device shown is only an example and should not impose any restrictions on the functions and usage scope of the embodiments of this application.

[0102] As Figure 6 shown, the electronic device 600 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which can perform various appropriate actions and processes according to the program stored in the read-only memory (ROM) 602 or the program loaded from the storage device 608 into the random access memory (RAM) 603. In the RAM 603, various programs and data required for the operation of the electronic device 600 are also stored. The processing device 601, the ROM 602, and the RAM 603 are connected to each other through a bus 604. The input / output (I / O) interface 605 is also connected to the bus 604.

[0103] Typically, the following devices can be connected to the I / O interface 605: input devices 606 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; output devices 607 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; storage devices 608 including, for example, a disk, a hard disk, etc.; and a communication device 609. The communication device 609 can allow the electronic device 600 to communicate with other devices wirelessly or wiredly to exchange data. Although Figure 6 the electronic device 600 with various devices is shown, it should be understood that it is not required to implement or have all the shown devices. Instead, more or fewer devices can be implemented or had. Figure 6 Each block shown in may represent one device or, as needed, multiple devices.

[0104] An embodiment of the present application also provides a computer program product, including a computer program which, when executed by a processor, implements the above-mentioned positioning method.

[0105] In particular, according to some embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of the present application include a computer program product which includes a computer program carried on a computer-readable medium, and the computer program contains program codes for executing the methods shown in the flowcharts. In such some embodiments, the computer program can be downloaded and installed from the network through the communication device 609, or installed from the storage device 608, or installed from the ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the methods of some embodiments of the present application are executed.

[0106] An embodiment of the present application also provides a computer-readable medium, on which a computer program is stored, and the program, when executed by a processor, implements the above-mentioned positioning method.

[0107] It should be noted that the computer-readable medium described in some embodiments of the present application may be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present application, the computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of the present application, the computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer-readable signal medium may also be any computer-readable medium other than the computer-readable storage medium, and the computer-readable signal medium may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0108] In some embodiments, the client and the server may communicate using any currently known or future-developed network protocol, such as HTTP (HyperText Transfer Protocol), and may be interconnected with digital data communication in any form or medium (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), the Internet (e.g., the Internet), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0109] The above computer-readable medium may be included in the above electronic device; or may exist separately without being assembled into the electronic device. The above computer-readable medium carries one or more programs, and when the one or more programs are executed by the electronic device, the electronic device is caused to: determine the first pose information of the laser positioning identifier based on the data collected by the laser sensor; determine the second pose information of the visual positioning identifier based on the image showing the visual positioning identifier collected by the image sensor, the first pose information, and the positional relationship between the laser positioning identifier and the visual positioning identifier; and determine the target pose information of the combined identifier including the laser positioning identifier and the visual positioning identifier based on the first pose information and the second pose information.

[0110] Computer program code for performing the operations of some embodiments of the present application may be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages such as Java, Smalltalk, C++; and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, executed as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network connection, or may be connected to an external computer (for example, by using an Internet service provider to connect through the Internet). The above networks include local area networks (LANs) or wide area networks (WANs).

[0111] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a portion of code that contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and the combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0112] The units described in some embodiments of the present application can be implemented in software or in hardware. The described units can also be provided in a processor. For example, it can be described as: a processor includes a first determination unit, a second determination unit, a selection unit, and a third determination unit. Among them, the names of these units do not constitute a limitation on the unit itself in some cases.

[0113] The functions described above herein can be performed, at least in part, by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that can be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), Systems on Chip (SOCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0114] The above description is only some preferred embodiments of the present application and an explanation of the applied technical principles. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present application is not limited to the technical solutions formed by the specific combination of the above technical features, and should also cover other technical solutions formed by any combination of the above technical features or their equivalent features without departing from the above inventive concept. For example, the technical solutions formed by mutually replacing the above features with the (but not limited to) technical features having similar functions disclosed in the embodiments of the present application.

Claims

1. A positioning method, characterized in that, Including: Determine the first pose information of the laser positioning identifier based on the data collected by the laser sensor; Determine the second pose information of the visual positioning identifier based on the image showing the visual positioning identifier collected by the image sensor, the first pose information, and the positional relationship between the laser positioning identifier and the visual positioning identifier; Determine the target pose information of the combined identifier including the laser positioning identifier and the visual positioning identifier based on the first pose information and the second pose information.

2. The method according to claim 1, characterized in that, The determining the first pose information of the laser positioning identifier based on the data collected by the laser sensor includes: Select, from the data collected by the laser sensor, laser points with a reflection intensity greater than the intensity threshold and a target number of adjacent points in sequence to obtain a point cloud; Locate the laser positioning identifier based on the point cloud to obtain the first pose information.

3. The method according to claim 2, characterized in that, The locating the laser positioning identifier based on the point cloud to obtain the first pose information includes: Fit a straight line based on the point cloud; Determine the yaw angle of the laser positioning identifier based on the angle of the straight line; Determine the coordinates of the center point of the laser positioning identifier based on the intersection points of the boundary of the point cloud and the straight line; Obtain the first pose information based on the yaw angle and the coordinates of the center point.

4. The method according to any one of claims 1 - 3, characterized in that, After obtaining the first pose information, the method further includes: Determine the pose change amount of the laser positioning identifier based on the current running speed of the mobile robot, the first time when the laser sensor collects the data, and the second time when the image sensor collects the image; Update the first pose information based on the pose change amount.

5. The method according to any one of claims 1 - 3, characterized in that, The determining the second pose information of the visual positioning identifier based on the image showing the visual positioning identifier collected by the image sensor, the first pose information, and the positional relationship between the laser positioning identifier and the visual positioning identifier includes: Determine the target area of the visual positioning identifier in the image collected by the image sensor based on the first pose information, the positional relationship between the laser positioning identifier and the visual positioning identifier, the parameter information of the image sensor, and the parameter information of the visual positioning identifier; Perform intensity normalization on the pixel values in the target area to obtain a target area image; Locate the visual positioning identifier based on the target area image to obtain the second pose information of the visual positioning identifier.

6. The method according to claim 5, characterized in that, The performing intensity normalization on the pixel values in the target area to obtain a target area image includes: Determine the average pixel value in the target area; Determine the pixel value scaling factor of each pixel point based on the comparison between the pixel value of each pixel point in the target area and the average pixel value; Update the pixel values of each pixel point based on the pixel value scaling factor of each pixel point to obtain a target area image with enhanced contrast.

7. The method according to claim 6, characterized in that, The determining the target area of the visual positioning identifier in the image collected by the image sensor based on the first pose information, the positional relationship between the laser positioning identifier and the visual positioning identifier, the parameter information of the image sensor, and the parameter information of the visual positioning identifier includes: Determine the position of the visual positioning identifier in the image sensor coordinate system based on the first pose information, the positional relationship between the laser positioning identifier and the visual positioning identifier, and the parameter information of the image sensor; Determine the initial region of the visual positioning identifier in the image based on the position of the visual positioning identifier in the image sensor coordinate system, the size of the visual positioning identifier, and the resolution of the image sensor; Enlarge the initial region based on preset error information to obtain the target region in the image.

8. The method according to any one of claims 1 - 3, 6 - 7, characterized in that, The determining the target pose information of the combined identifier including the laser positioning identifier and the visual positioning identifier based on the first pose information and the second pose information includes: Determine the positioning point of the combined identifier including the laser positioning identifier and the visual positioning identifier based on the type of the laser positioning identifier; Determine the third pose information of the combined identifier based on the first pose information and the positional relationship between the laser positioning identifier and the positioning point; Determine the fourth pose information of the combined identifier based on the second pose information and the positional relationship between the visual positioning identifier and the positioning point; Determine the target pose information of the combined identifier based on the third pose information and the fourth pose information.

9. The method according to claim 8, wherein The third pose information includes a first coordinate and a first yaw angle, and the fourth pose information includes a second coordinate, a second yaw angle, a roll angle, and a pitch angle; The determining the target pose information of the combined identifier including the laser positioning identifier and the visual positioning identifier based on the third pose information and the fourth pose information includes: Determine the target coordinate of the visual positioning identifier based on the first coordinate and the second coordinate; Perform a weighted sum of the first yaw angle and the second yaw angle to obtain the target yaw angle; Determine the target pose information of the visual positioning identifier based on the target coordinate, the target yaw angle, the roll angle, and the pitch angle.

10. The method according to claim 9, wherein The determining the target coordinate of the visual positioning identifier based on the first coordinate and the second coordinate includes: Determine a distance threshold based on the resolution of the laser sensor; If the distance from the laser positioning identifier to the laser sensor is less than the distance threshold, use the first coordinate as the target coordinate; If the distance from the laser positioning identifier to the laser sensor is greater than or equal to the distance threshold, perform a weighted sum of the first coordinate and the second coordinate to obtain the target coordinate.

11. The method according to any one of claims 1 - 3, 6 - 7, 9 - 10, wherein The laser positioning identifier includes at least one of the following types: single retroreflective marker, double retroreflective marker; the visual positioning identifier includes a QR code.

12. A mobile robot, wherein Includes: Laser sensor; Image sensor; One or more processors; A storage device having stored thereon one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1-10.

13. A computer - readable medium having a computer program stored thereon, wherein When the program is executed by the processor, it implements the method according to any one of claims 1-10.

14. A computer program product comprising a computer program or instructions, wherein When the computer program or instruction is executed by a processor, it implements the method described in any one of claims 1-10.

Images