Initial positioning method and device, electronic equipment and computer readable storage medium

Abstract

The invention provides an initial positioning method and device, electronic equipment and a computer readable storage medium, multiple particles are constructed, each particle has pose information, and marker information monitored in real time is obtained based on monitoring data of a sensor. Converting marker information monitored in real time into marker information under a map coordinate system based on the pose information of each particle, matching the marker information under the map coordinate system corresponding to each particle with markers in a pre-constructed marker map, and determining a target particle according to a matching result; and obtaining an initial pose according to the pose information of the target particle. According to the scheme, pose initialization is realized by monitoring the marker and matching the marker with the marker in the pre-constructed map, the implementation complexity is reduced, the real-time application requirement can be met, and the method can be applied to various application scenes.

Description

Technical Field

[0001] This invention relates to the field of navigation and positioning technology, and more specifically, to an initial positioning method, apparatus, electronic device, and computer-readable storage medium. Background Technology

[0002] In many fields requiring localization, such as autonomous driving and robotics, Simultaneous Localization and Mapping (SLAM) technology is used to determine one's position and orientation in a world coordinate system or a map coordinate system. This serves as a foundational service for downstream sensing, planning, and control. SLAM technology requires a series of processes based on the initial pose to perform pose prediction. Therefore, determining the initial pose is crucial; inaccurate initial pose determination will prevent subsequent localization functions from working properly.

[0003] Currently, pose initialization primarily utilizes information output from the Global Positioning System (GPS) or integrated navigation systems. However, in many scenarios, GPS signals may be unavailable, making it difficult to obtain the initial pose using GPS signals or integrated navigation information. Alternatively, some methods estimate the initial pose by matching scene descriptors, such as scancontext, with keyframes from the map. However, this method only utilizes single-frame information and is prone to failing to find similar keyframes for similar scenes, and also struggles with finding similar keyframes for non-keyframe poses, impacting subsequent localization processes. Another approach involves hard matching using a global map search, but this method is time-consuming and unsuitable for real-time requirements. Summary of the Invention

[0004] The objectives of this invention include, for example, providing an initial positioning method, apparatus, electronic device, and computer-readable storage medium that can meet the needs of real-time applications and can be applied in a variety of application scenarios.

[0005] The embodiments of the present invention can be implemented as follows:

[0006] In a first aspect, the present invention provides an initial positioning method, the method comprising:

[0007] Construct multiple particles, each of which has pose information;

[0008] Real-time monitoring information of markers is obtained based on sensor monitoring data;

[0009] Based on the pose information of each particle, the real-time monitored marker information is converted into marker information in the map coordinate system;

[0010] The marker information in the map coordinate system corresponding to each particle is matched with the markers in the pre-constructed marker map. The target particle is determined based on the matching result, and the initial pose is obtained based on the pose information of the target particle.

[0011] Based on the initial positioning method provided in the first aspect, pose initialization is achieved by monitoring landmarks and matching them with landmarks in a pre-built map, which reduces the implementation complexity, meets the needs of real-time applications, and can be applied to a variety of application scenarios.

[0012] In an optional implementation, the step of obtaining real-time monitored marker information based on sensor monitoring data includes:

[0013] Obtain a point cloud obtained from sensor monitoring, wherein the point cloud includes multiple points;

[0014] A three-dimensional mesh is constructed within the three-dimensional space of the point cloud, and the three-dimensional mesh comprises multiple meshes;

[0015] The number of points within each grid is counted, and the real-time monitored marker information is determined based on the number of points within each grid.

[0016] By collecting point clouds and obtaining landmark information based on the distribution of points in the point clouds, accurate landmark information can be obtained.

[0017] In an optional implementation, the step of counting the number of points within each grid and determining the real-time monitored marker information based on the number of points within each grid includes:

[0018] The grid is traversed horizontally to count the number of points in each traversed grid.

[0019] If the number of points in the traversed grid is greater than or equal to the first preset number, the grid is marked as occupied.

[0020] In the same vertical direction, if there are more than two preset number of consecutive grids marked as occupied, then the points within those consecutive grids are determined to be points of the marker.

[0021] By statistically analyzing the distribution of grid midpoints in both the horizontal and vertical directions, markers are determined, ensuring that the solution can be implemented conveniently and quickly.

[0022] In an optional implementation, the step of constructing a three-dimensional mesh within the three-dimensional space of the point cloud includes:

[0023] Based on the depth information of each point included in the point cloud, points whose depth information meets the preset requirements are selected, and points whose depth information does not meet the preset requirements are deleted.

[0024] A 3D mesh is constructed within the 3D space of the point cloud formed by the selected points.

[0025] By filtering out points in the point cloud, the influence of irrelevant interference points can be avoided, and the workload can be reduced.

[0026] In an optional implementation, the step of obtaining real-time monitored marker information based on sensor monitoring data includes:

[0027] Obtain real-time images and point clouds from sensor monitoring, wherein the point cloud comprises multiple points;

[0028] The real-time image is imported into the target detection model, and the bounding boxes of the targets present in the real-time image are output.

[0029] Based on the relationship between the real-time image and the point cloud, target points in the point cloud corresponding to the target bounding box positions in the real-time image are obtained, and each target point is determined as a marker point.

[0030] By performing bounding box detection on real-time images and combining the relationship between point clouds and real-time images, points in the point cloud that are represented as landmarks can be accurately identified.

[0031] In an optional implementation, the step of obtaining the target point in the point cloud corresponding to the target bounding box position in the real-time image based on the relationship between the real-time image and the point cloud includes:

[0032] Project each point in the point cloud onto the real-time image to obtain the projection point corresponding to each point in the real-time image;

[0033] Determine the target projection point within the target bounding box of the real-time image;

[0034] Obtain the corresponding target point in the point cloud for each of the target projection points.

[0035] By using projection, the conversion from three-dimensional points to two-dimensional points can be effectively achieved.

[0036] In an optional implementation, the real-time image is obtained through camera monitoring, and the point cloud is obtained through lidar monitoring;

[0037] The step of projecting each point in the point cloud onto the real-time image includes:

[0038] Obtain the calibration parameters of the camera and the three-dimensional coordinate information of each point;

[0039] The coordinate information of each point in the camera coordinate system is calculated based on the calibration parameters and the three-dimensional coordinate information of each point.

[0040] The projection point of each point in the real-time image is determined based on the coordinate information of each point in the camera coordinate system.

[0041] The camera's calibration parameters effectively enable the conversion between 3D coordinate information and coordinate information in the camera coordinate system.

[0042] In an optional implementation, the step of matching the marker information in the map coordinate system corresponding to each particle with the markers in the pre-constructed marker map includes:

[0043] For each particle, obtain the corresponding markers in the pre-constructed marker map under the map coordinate system of the particle, and form a marker group;

[0044] Calculate the matching degree between two markers in each of the marker groups;

[0045] The matching degree between the marker information in the map coordinate system corresponding to the particle and the marker in the marker map is obtained based on the matching degree corresponding to each marker group.

[0046] The overall matching degree between the marker information in the map coordinate system corresponding to the particle and the marker in the marker map is obtained by statistically analyzing the matching degree between each pair of corresponding markers.

[0047] In an optional implementation, the step of determining the target particle based on the matching result includes:

[0048] The particle with the highest matching degree among the plurality of particles is obtained, and it is detected whether the highest matching degree is greater than or equal to a preset threshold.

[0049] If the highest matching degree is greater than or equal to the preset threshold, then the particle is determined to be the target particle;

[0050] If the highest matching degree is less than the preset threshold, the pose information of each particle is updated, and the matching degree between the updated marker information in the map coordinate system corresponding to each particle and the marker in the marker map is calculated until the highest matching degree is greater than or equal to the preset threshold, at which point the particle corresponding to the highest matching degree is determined as the target particle.

[0051] By updating the particle pose information, target particles that meet the requirements can be found, so as to obtain a highly accurate initial pose.

[0052] In an optional implementation, the step of updating the pose information of each of the particles includes:

[0053] Obtain the angular velocity and linear velocity monitored by the sensors;

[0054] Based on the current pose information of each particle, the angular velocity, and the linear velocity, the updated pose information of the particle is calculated.

[0055] By monitoring angular velocity and linear velocity in real time, pose information is monitored, ensuring that the updated pose information matches the actual scene.

[0056] In an optional implementation, the step of updating the pose information of each of the particles includes:

[0057] Filter out particles with a matching degree greater than or equal to the set threshold;

[0058] For each selected particle, multiple duplicate particles are obtained based on the particle, and the pose information of each duplicate particle is randomly adjusted.

[0059] The pose information of each selected particle and each of the copied particles is updated.

[0060] By randomly perturbing high-quality particles before updating their pose information, it is possible to find target particles of even higher quality.

[0061] In an optional implementation, the landmark map includes multiple types of landmarks, and the method further includes a step of pre-constructing the landmark map, which includes:

[0062] Construct sub-maps corresponding to various types of landmarks;

[0063] The markers in each sub-map are filtered so that each sub-map contains only one marker at every preset range.

[0064] The landmark map is obtained by merging multiple sub-maps.

[0065] By constructing multiple sub-maps and merging them into a landmark map, and filtering the landmarks in the sub-maps, matching difficulties caused by excessively high landmark density can be avoided.

[0066] In an optional implementation, the method further includes a step of pre-constructing the landmark map, which includes:

[0067] Obtain keyframe point clouds, keyframe images, and keyframe poses corresponding to multiple consecutive keyframes.

[0068] For each of the key frames, candidate markers are determined based on the key frame point cloud, key frame image, and key frame pose corresponding to the key frame.

[0069] Check if a marker identical to the candidate marker already exists. If it does, delete the candidate marker; otherwise, place the candidate marker in the marker map.

[0070] By comparing the identified candidate landmarks with landmarks in the existing landmark map, duplicate landmark settings can be avoided.

[0071] In a second aspect, the present invention provides an initial positioning device, the device comprising:

[0072] A construction module is used to construct multiple particles, each of which has pose information;

[0073] The acquisition module is used to obtain real-time monitoring information of markers based on sensor monitoring data;

[0074] The conversion module is used to convert the real-time monitored marker information into marker information in the map coordinate system based on the pose information of each particle.

[0075] The matching module is used to match the marker information in the map coordinate system corresponding to each particle with the markers in the pre-constructed marker map, determine the target particle based on the matching result, and obtain the initial pose based on the pose information of the target particle.

[0076] The apparatus provided in the second aspect can perform the methods described in any implementation of the first aspect, which will not be elaborated here. Furthermore, the technical effects of the apparatus described in the second aspect can be referenced to the technical effects of the methods described in any implementation of the first aspect, which will not be elaborated here.

[0077] Thirdly, the present invention provides an electronic device including one or more storage media and one or more processors communicating with the storage media, wherein the one or more storage media store machine-executable instructions executable by the processor, and when the electronic device is running, the processor executes the machine-executable instructions to perform the method described in any of the foregoing embodiments.

[0078] The technical effects of the electronic device described in the third aspect can be referred to the technical effects of the method described in any of the implementation methods in the first aspect, and will not be repeated here.

[0079] Fourthly, the present invention provides a computer-readable storage medium storing machine-executable instructions that, when executed by a processor, implement the method described in any of the foregoing embodiments.

[0080] The technical effects of the computer-readable storage medium described in the fourth aspect can be referred to the technical effects of the method described in any implementation of the first aspect, and will not be repeated here. Attached Figure Description

[0081] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly introduced below. It should be understood that the following drawings only show some embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0082] Figure 1 A flowchart of the initial positioning method provided by the present invention;

[0083] Figure 2 This is a schematic diagram of multiple particles generated in this invention;

[0084] Figure 3 This is another schematic diagram of the multiple particles generated in this invention;

[0085] Figure 4 Figure 1 A flowchart of the sub-steps included in S2;

[0086] Figure 5 for Figure 4 A flowchart of the sub-steps included in S22;

[0087] Figure 6 for Figure 4 A flowchart of the sub-steps included in S23;

[0088] Figure 7 for Figure 1 Another flowchart of the sub-steps included in S2;

[0089] Figure 8 for Figure 7 A flowchart of the sub-steps included in S23;

[0090] Figure 9 A flowchart of the marker map construction method in the initial positioning method provided by the present invention;

[0091] Figure 10 This is a schematic diagram of the complete point cloud acquired by the sensor in this invention;

[0092] Figure 11 This is a schematic diagram of the point cloud in the landmark map constructed in this invention;

[0093] Figure 12 for Figure 1 A flowchart of the sub-steps included in S4;

[0094] Figure 13 for Figure 1 Another flowchart of the sub-steps included in S4;

[0095] Figure 14 A functional block diagram of the initial positioning device provided by the present invention;

[0096] Figure 15 This is a structural block diagram of the electronic device provided by the present invention. Detailed Implementation

[0097] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0098] Therefore, the following detailed description of the embodiments of the invention provided in the accompanying drawings is not intended to limit the scope of the claimed invention, but merely to illustrate selected embodiments of the invention. All other embodiments obtained by those skilled in the art based on the embodiments of the invention without inventive effort are within the scope of protection of the invention.

[0099] It should be noted that similar labels and letters in the following figures indicate similar items. Therefore, once an item is defined in one figure, it does not need to be further defined and explained in subsequent figures.

[0100] In the description of this invention, it should be noted that the terms "first" and "second" are used only for distinguishing descriptions and should not be construed as indicating or implying relative importance.

[0101] It should be noted that, where there is no conflict, the features in the embodiments of the present invention can be combined with each other.

[0102] Please see Figure 1 The above is a flowchart of an initial positioning method provided in an embodiment of the present invention. This initial positioning method can be executed by an initial positioning device, which can be implemented by software and / or hardware and can be configured in an electronic device, such as a microprocessor or microcontroller. The detailed steps of this initial positioning method are described below.

[0103] S1, construct multiple particles, each with pose information.

[0104] S2 obtains real-time monitoring information of markers based on sensor monitoring data.

[0105] S3 converts the real-time monitored marker information into marker information in the map coordinate system based on the pose information of each particle.

[0106] S4. Match the marker information in the map coordinate system corresponding to each particle with the markers in the pre-constructed marker map, determine the target particle based on the matching result, and obtain the initial pose based on the pose information of the target particle.

[0107] In this embodiment, the constructed multiple particles can be understood as multiple pose information of a set device, such as a vehicle, subway, or robot. The term "device" will be used collectively in the following description. The pose information of the multiple particles can be generated within a certain range according to certain rules. For example, the multiple particles can be uniformly distributed within a certain range (e.g., ...). Figure 2 (as shown), or it can be according to a Gaussian distribution (such as...). Figure 3 (as shown in the image) etc.

[0108] After the device is started, pose initialization is required to determine the initial pose for subsequent positioning. The initial pose of the subway can be calculated based on the pose information of multiple particles.

[0109] The equipment is equipped with multiple sensors, including inertial navigation components, millimeter-wave radar, lidar, and cameras. After the equipment starts up and during its subsequent movement, each sensor monitors data at a specific sampling frequency to collect relevant data.

[0110] The data collected by each sensor includes data on numerous objects around the device. To facilitate comparison using stable and distinctive objects as targets, this embodiment can obtain real-time monitoring information of landmarks based on the sensor's monitoring data. Landmarks include objects commonly found on roads, such as poles, traffic lights, and road signs.

[0111] Since the marker information collected by sensors is in the coordinate system of each sensor—for example, marker information collected by LiDAR is in the LiDAR coordinate system, while marker information collected by cameras is in the camera coordinate system—it is necessary to uniformly convert the marker information monitored in real time by each sensor into information in the map coordinate system.

[0112] When converting the real-time marker information monitored by various sensors into information in a map coordinate system, the device's own pose information will affect the conversion process. That is, the conversion result will be different depending on the device's pose information. Therefore, for each particle, the real-time monitored marker information is converted into marker information in a map coordinate system based on the particle's pose information.

[0113] Furthermore, in this embodiment, a landmark map is pre-established. This landmark map is constructed based on landmarks within a global scope; that is, the landmark map contains information about landmarks within a global scope. The landmark information in the landmark map is information in a map coordinate system.

[0114] Based on this, the marker information in the map coordinate system corresponding to each particle is matched with the markers in the marker map, and the target particle is determined based on the matching results. The higher the matching degree between the marker information in the map coordinate system corresponding to a particle and the markers in the marker map, the more accurate the pose information corresponding to that particle. Therefore, based on the matching results, the particle with the highest accuracy can be determined as the target particle. The pose information of this target particle can then be used as the device's current initial pose. This initial pose can be provided to the backend relocalization system for relocalization.

[0115] The initial positioning method provided in this embodiment uses multiple sensors to monitor landmarks and match them with landmarks in a pre-built map to achieve pose initialization, reducing implementation complexity, meeting real-time application requirements, and applicable to various application scenarios.

[0116] In this embodiment, sensors include inertial navigation components, millimeter-wave radar, lidar, cameras, etc., and markers include poles, traffic lights, road signs, etc. Different types of markers can be determined based on data monitored by one or more sensors. For example, marker information can be obtained based on point cloud data monitored by lidar and / or image data monitored by the camera.

[0117] Please see Figure 4 In one possible implementation, real-time monitoring information of markers can be obtained based on sensor monitoring data in the following way.

[0118] S21A, obtains the point cloud obtained by sensor monitoring, the point cloud includes multiple points.

[0119] S22A constructs a 3D mesh within the 3D space of the point cloud, and the 3D mesh consists of multiple meshes.

[0120] S23A: Count the number of points in each grid and determine the real-time monitored marker information based on the number of points in each grid.

[0121] In this embodiment, a point cloud can be obtained using LiDAR. This point cloud represents the points of objects within a local area near the device's location, including objects such as poles, the ground, and buildings. The obtained point cloud comprises multiple points, each a three-dimensional point in three-dimensional space. A three-dimensional mesh is constructed based on the three-dimensional space containing the point cloud. This three-dimensional mesh can be understood as a cube that encloses all the points in the point cloud. This three-dimensional mesh is divided into multiple grids, each of which can be understood as a smaller cube within the larger cube. Thus, each point in the point cloud will fall into one of the grids in the three-dimensional mesh. Each grid may contain one or more points, or it may contain no points.

[0122] In this embodiment, considering that the distribution of points in areas too far from the sensor in the actual collected point cloud is relatively sparse, while points in areas too close to the sensor may not be able to detect the required landmarks (e.g., landmarks are required to be above a certain height) due to field of view limitations, please refer to [link to relevant documentation]. Figure 5 In this embodiment, the construction of a three-dimensional mesh can be achieved in the following way:

[0123] S221A: Based on the depth information of each point in the point cloud, filter out points whose depth information meets the preset requirements, and delete points whose depth information does not meet the preset requirements.

[0124] S222A constructs a 3D mesh within the 3D space of the point cloud formed by the selected points.

[0125] In this embodiment, each point in the point cloud has three-dimensional coordinate information. Based on the three-dimensional coordinate information of each point, the distance between each point and the sensor can be obtained; this distance is the depth information. Points with depth values ​​greater than a first preset depth value and points with depth values ​​less than a second preset depth value are deleted to retain points within a certain range. Then, a three-dimensional mesh is constructed based on the three-dimensional space where the points within this certain range are located. The outer contour of the three-dimensional mesh can just enclose the points within this certain range.

[0126] In this embodiment, the location of a marker can be determined based on the distribution of points in the point cloud across various grids, thereby determining the marker information. For details, please refer to [link to relevant documentation]. Figure 6 The real-time monitored marker information can be determined based on the number of points in each grid in the following way.

[0127] S231A traverses each grid in the horizontal direction to count the number of points in the traversed grid.

[0128] S232, if the number of points in the traversed grid is greater than or equal to the first preset number, mark the grid as occupied.

[0129] S233A, in the same vertical direction, if there are more than two preset number of consecutive grids marked as occupied, then the points in the consecutive grids are determined to be the points of the marker.

[0130] In this embodiment, each point in the point cloud detected by the lidar is located in the lidar coordinate system, and each point has three-dimensional coordinate values. The lidar coordinate system includes the X-axis and Y-axis on the horizontal plane, and the Z-axis perpendicular to the horizontal plane, with the Z-axis direction being vertical.

[0131] In a three-dimensional mesh, the mesh can be understood as multiple horizontal mesh layers stacked vertically, with each mesh layer containing multiple meshes. In this embodiment, for each horizontal mesh layer, each mesh in the layer can be traversed. If the number of points in a mesh is greater than or equal to a first preset number, such as 5, then the mesh is determined to be in an occupied state and marked as occupied.

[0132] Based on this, in the vertical direction, if there are consecutive grids marked as occupied, and the number of grids is greater than or equal to a second preset number, then the location of these consecutive grids can be determined as the location of a marker, such as the location of a rod-shaped object. The points within these consecutive grids are the points constituting the marker. In this way, the position information of the marker in the lidar coordinate system can be obtained.

[0133] The second preset number can be obtained based on the set minimum height value and the resolution res of the 3D mesh. The resolution res of the 3D mesh can determine the number of meshes in the 3D mesh, for example, it can be 0.4.

[0134] In this embodiment, given a fixed resolution res of the 3D mesh, the dimensions of the constructed 3D mesh in each direction can be represented by the following formula:

[0135] Dimx = ceil(Xmax - Xmin) / res + 1

[0136] Dimy = ceil(Ymax - Ymin) / res + 1

[0137] Dimz = ceil(Zmax - Zmin) / res + 1

[0138] Where Xmax, Xmin, Ymax, Ymin, Zmax, and Zmin represent the coordinate values ​​of the grids with the maximum and minimum coordinate values ​​in the X-axis, Y-axis, and Z-axis directions, respectively.

[0139] When determining the grid location of each point P in the point cloud, it can be determined based on the 3D coordinates and resolution of point P in the following way:

[0140] Idx=round((point.x-Xmin) / res)

[0141] Idy=round((point.y-Ymin) / res)

[0142] Idx=round((point.z-Zmin) / res)

[0143] Where point.x, point.y, and point.z represent the coordinates of point P on the X-axis, Y-axis, and Z-axis, respectively.

[0144] The above describes a method for obtaining marker information based on point cloud data collected by lidar, which can obtain information about markers such as rod-shaped objects.

[0145] In another implementation of this embodiment, data monitored by LiDAR and cameras can be combined to obtain marker information, such as information on traffic lights. Please refer to [reference needed]. Figure 7 In this implementation, real-time monitoring information of markers can be obtained based on sensor monitoring data in the following ways.

[0146] S21B acquires real-time images and point clouds obtained from sensor monitoring, the point cloud comprising multiple points.

[0147] S22B imports real-time images into the object detection model and outputs the bounding boxes of objects present in the real-time images.

[0148] S23B, based on the relationship between the real-time image and the point cloud, obtains the target points in the point cloud corresponding to the target bounding box positions in the real-time image, and determines each target point as a marker point.

[0149] In this embodiment, the real-time image is obtained from camera monitoring, and the point cloud data is obtained from LiDAR monitoring. Importing the real-time image into the target detection model allows us to obtain the bounding boxes present in the real-time image. The location of the bounding box can be understood as the location of a landmark in the real-time image, such as a traffic light. The bounding box can be labeled (u, v, w, h), where u and v represent the coordinates of the bounding box in the camera coordinate system, and w and h represent the width and height of the bounding box, respectively.

[0150] The object detection model can be a YOLO model. The YOLO model can predict the location and category of objects in an image through a single forward propagation, and can be used for real-time detection of objects such as traffic lights. In this embodiment, a general YOLO model can be retrained using samples containing markers such as traffic lights to obtain the object detection model.

[0151] The location of the target bounding box output by the target detection model is the location of the marker.

[0152] Since both the lidar and the camera are mounted on equipment such as vehicles, there is a certain relationship between the real-time images and point clouds collected by the two. Therefore, after determining the target box in the real-time image, the point in the point cloud corresponding to the location of the target box, i.e. the target point, can be determined. The three-dimensional coordinate information of the marker can be obtained based on the three-dimensional coordinate information of the target point.

[0153] Please see Figure 8 In this embodiment, when obtaining the target point in the point cloud corresponding to the target bounding box position in the real-time image based on the relationship between the real-time image and the point cloud, it can be achieved in the following way:

[0154] S231B projects each point in the point cloud onto a real-time image to obtain the corresponding projection point of each point in the real-time image.

[0155] S232B, determine the target projection point within the target bounding box of the real-time image.

[0156] S233B, obtain the corresponding target points in the point cloud for each target projection point.

[0157] In this embodiment, projecting points from the point cloud onto the real-time image essentially involves converting the three-dimensional coordinate information of the points into coordinate information in the camera coordinate system. In this embodiment, camera calibration parameters and the three-dimensional coordinate information of each point can be obtained. Based on the calibration parameters and the three-dimensional coordinate information of each point, the coordinate information of each point in the camera coordinate system is calculated. Based on the coordinate information of each point in the camera coordinate system, the projection point of each point in the real-time image is determined.

[0158] The camera's calibration parameters include intrinsic and extrinsic parameters. Intrinsic parameters include focal length, principal point, radial distortion, and tangential distortion coefficients. Extrinsic parameters include rotation matrix and translation vector. In this embodiment, based on the camera's intrinsic and extrinsic parameters, and the three-dimensional coordinate information of each point, the coordinate information of the point in the camera coordinate system can be calculated in the following way:

[0159] Coordinate information in the camera coordinate system = intrinsic parameters × extrinsic parameters × 3D coordinate information

[0160] In real-time images, the location information of the target bounding box is the coordinate information in the camera coordinate system, and the target bounding box can define a region. Thus, based on the coordinate information of each projection point, it is possible to determine which projection points are within the target bounding box, and the points in the point cloud corresponding to the projection points within the target bounding box are the target points.

[0161] For each target bounding box, there may be multiple points corresponding to that target bounding box. Therefore, the mean value can be calculated based on the three-dimensional coordinate information of these multiple points, and the calculated mean value can be used as the three-dimensional coordinate information of points such as traffic lights.

[0162] The above describes the process of determining marker information based on real-time monitored data. The obtained marker information is the three-dimensional coordinate information of the marker. It is necessary to convert this marker information into marker information in a map coordinate system using the pose information of each particle. In this embodiment, the marker information is converted by multiplying the particle pose information by the marker information. The conversion formula is as follows:

[0163] Lmap = Particle_pose * Llidar

[0164] Here, Lmap represents the position information of the marker in the map coordinate system, Particle_pose represents the pose information of the particle, and Llidar represents the three-dimensional coordinate information of the marker.

[0165] After obtaining the landmark information in the map coordinate system, it can be matched with the landmarks in the pre-built landmark map. The landmark map is built offline, and the process of pre-building the landmark map is described below.

[0166] Please see Figure 9 In this embodiment, the landmark map can be constructed in the following way:

[0167] S01, obtain the keyframe point cloud, keyframe image and keyframe pose corresponding to multiple consecutive keyframes respectively.

[0168] S02, for each keyframe, determine candidate markers based on the keyframe point cloud, keyframe image and keyframe pose corresponding to the keyframe.

[0169] S03, check if a marker identical to the candidate marker already exists. If it exists, proceed to S04; otherwise, proceed to S05.

[0170] S04, delete the candidate markers.

[0171] S05, Place the candidate markers in the marker map.

[0172] In this embodiment, the image frames acquired by each sensor are consecutive multi-frame images, including keyframes and non-keyframes. Keyframes are specific frames selected for map building and localization. Keyframes generally contain rich feature information describing the environment, which is helpful for subsequent matching and localization processes. Keyframes are typically selected based on certain time intervals and the degree of scene change; for example, a significant scene change may trigger the selection of a new keyframe. Non-keyframes are ordinary frames not selected for map building and localization. Non-keyframes contain less information than keyframes. Non-keyframes are closely linked to keyframes in time and are mainly used for tracking known feature points and predicting vehicle movement, not for map building.

[0173] In this embodiment, multiple keyframes are obtained, and the keyframe point cloud, keyframe image, and keyframe pose corresponding to each keyframe are also obtained. The keyframe point cloud, keyframe image, and keyframe pose are corresponding in timestamp.

[0174] Candidate landmarks can be determined based on keyframe point clouds, keyframe images, and keyframe poses. Since the landmark map is constructed based on information from consecutive keyframes, the same landmark may be detected in consecutive keyframes. Therefore, landmarks detected based on each keyframe are considered as candidate landmarks. By searching for existing landmarks in the landmark map, if a landmark identical to a candidate landmark already exists in the map, the candidate landmark does not need to be added; otherwise, it is added to the landmark map.

[0175] When searching for a marker in the map that is the same as a candidate marker, the system can search for a marker with the same or similar coordinates based on the candidate marker's coordinates in the map coordinate system. If such a marker exists, it is determined that the marker is the same as the candidate marker.

[0176] In this embodiment, the detection of candidate markers based on each keyframe can be carried out in the same way as the real-time monitoring marker information obtained based on sensor monitoring data in the real-time detection stage, and will not be described again here.

[0177] The marker information detected using the above method includes the marker's three-dimensional coordinates, such as the three-dimensional coordinates of poles and traffic lights. Therefore, it is also necessary to convert the marker's three-dimensional coordinates into coordinates in the map coordinate system of the marker map.

[0178] In this embodiment, based on each keyframe, after obtaining the three-dimensional coordinate information of the marker based on the keyframe, the coordinate information in the corresponding map coordinate system is obtained by multiplying the keyframe pose corresponding to the keyframe by the three-dimensional coordinate information.

[0179] As can be seen from the above, there are many types of landmarks. When building a landmark map in advance, sub-maps corresponding to various types of landmarks can be built. The landmarks in each sub-map are filtered so that each sub-map has only one landmark every preset range. Then, multiple sub-maps are merged to obtain the landmark map.

[0180] In this embodiment, corresponding sub-maps are constructed for each type of landmark. For example, a landmark map can be constructed for poles, and a traffic light map can be constructed for traffic lights. When constructing sub-maps for each type of landmark, candidate landmarks can be determined based on the keyframe point cloud, keyframe image, and keyframe pose corresponding to each keyframe, as described above, and the map is constructed based on the candidate landmarks in the same way. This embodiment will not elaborate on this further.

[0181] To avoid overly dense markers in the marker map affecting matching, markers in each sub-map can be filtered to reduce their density. When merging multiple sub-maps to obtain a marker map, different types of markers can be distinguished and marked within the marker map. Figure 10 A schematic diagram of the complete point cloud acquired by the sensor is shown. Figure 11 A schematic diagram of the point cloud in the constructed landmark map is shown.

[0182] In this embodiment, the marker map is saved after being constructed offline. During the real-time monitoring phase, the marker information in the map coordinate system corresponding to each particle is matched with the markers in the pre-constructed marker map to determine the target particle. Please refer to [reference needed]. Figure 12 This step can be achieved in the following way:

[0183] S41, for each particle, obtain the corresponding markers in the pre-constructed marker map under the map coordinate system of the particle, and form a marker group.

[0184] S42, calculate the matching degree between two markers in each marker group.

[0185] S43, based on the matching degree corresponding to each group of markers, obtain the matching degree between the marker information in the map coordinate system corresponding to the particle and the markers in the marker map.

[0186] In this embodiment, the corresponding marker information in the map coordinate system is obtained based on the pose information of each particle. The real-time monitored marker information has coordinate information in the map coordinate system. Each marker in the marker map has coordinate information in the map coordinate system. Therefore, the corresponding markers can be determined to form a marker group based on coordinate information comparison.

[0187] Two markers in a marker group may not have completely identical coordinate information. In this embodiment, the matching degree between the two markers can be calculated using a binary Gaussian probability calculation method. In this method, the mean can be obtained based on the average coordinate information of the markers in the marker map, and the variance can be set to 0.2.

[0188] In this way, the matching degree of each group of markers can be calculated, and the mean or cumulative value of the matching degrees of multiple groups of markers can be calculated as the matching degree between the marker information in the map coordinate system corresponding to the particle and the markers in the marker map.

[0189] In this way, the target particle can be determined based on the matching degree of each particle. Please refer to the following: Figure 13 In this embodiment, this step can be implemented in the following way:

[0190] S44: Obtain the particle with the highest matching degree among multiple particles, and check whether the highest matching degree is greater than or equal to a preset threshold. If the highest matching degree is greater than or equal to the preset threshold, execute the following S45. If the highest matching degree is less than the preset threshold, execute the following S46.

[0191] S45, the particle is identified as the target particle.

[0192] S46, update the pose information of each particle, and calculate the matching degree between the marker information in the map coordinate system corresponding to each particle after the update and the marker in the marker map, until the highest matching degree is greater than or equal to the preset threshold, and then determine the particle corresponding to the highest matching degree as the target particle.

[0193] In this embodiment, each particle has pose information, which can be understood as setting initial pose information for each particle. Based on this initial pose information, the pose information of the particles can be iteratively updated, and can be updated multiple times based on real-time monitoring information.

[0194] Therefore, in each round, if the highest matching degree found is greater than or equal to the preset threshold, the corresponding particle can be identified as the target particle. If the highest matching degree is less than the preset threshold, the particle's state needs to be updated.

[0195] In this embodiment, the inertial navigation system on the vehicle or other equipment can monitor real-time angular velocity information, and the millimeter-wave radar can monitor real-time linear velocity. When updating the pose information of each particle, the angular velocity and linear velocity monitored by the sensors can be obtained. Based on the current pose information, angular velocity, and linear velocity of each particle, the updated pose information of the particle is calculated.

[0196] The pose information of each particle includes position information and attitude information. The position information includes x-coordinate, y-coordinate, and z-coordinate values, while the attitude information includes yaw angle, pitch angle, and roll angle. The z-coordinate, pitch angle, and roll angle are fixed by default. The x-coordinate, y-coordinate, and yaw angles can be calculated using the following methods.

[0197] When the difference between the angular velocity and 0 is within a certain range, the calculation formula is as follows:

[0198]

[0199]

[0200] yaw t =yaw t-1

[0201] When the difference between the angular velocity and 0 exceeds the above-mentioned certain range, the calculation formula is as follows:

[0202]

[0203]

[0204]

[0205] In the above formula, x t and x t-1 Represents the x-coordinate and y-coordinate of the particle in the two iterations. t and y t-1 v represents the y-coordinate value of the particle in two iterations. linear Indicates linear velocity. yaw represents the time interval between two iterations. t and yaw t-1 This represents the particle's yaw angle between two iterations. It represents angular velocity.

[0206] In this embodiment, considering the errors in the sensor monitoring data, a certain Gaussian perturbation can be added to the pose information after the updated pose information of each particle is calculated, thereby improving the robustness of the result.

[0207] In this embodiment, the pose information of each particle is updated in the above manner, and then the marker information in the map coordinate system corresponding to each particle is recalculated based on the updated pose information. Then, it is matched with the markers in the marker map until a particle with the highest matching degree greater than or equal to a preset threshold is found.

[0208] In this embodiment, considering that the matching degree can reflect the accuracy of the particle pose information, when updating the pose information of each particle, particles with a matching degree greater than or equal to a set threshold can be selected. For each selected particle, multiple duplicate particles are obtained based on the particle, and the pose information of each duplicate particle is randomly adjusted. Then, the pose information of each selected particle and each duplicate particle is updated.

[0209] In this way, random perturbations can be applied to particles with good quality, and on this basis, particles with highly accurate pose information can be found.

[0210] Through the above process, after each update, the marker information in the map coordinate system corresponding to the particle can be matched with the marker map to find the target particle, and then the pose information of the target particle can be obtained as the initial pose to be provided to the backend relocalization system for relocalization.

[0211] Based on the same inventive concept, please refer to Figure 14 This diagram illustrates the functional modules of the initial positioning device provided in an embodiment of the present invention. This embodiment can divide the initial positioning device into functional modules according to the above method embodiment. For example, each function can be divided into its own functional modules, or two or more functions can be integrated into one processing module. The integrated module can be implemented in hardware or as a software functional module.

[0212] It should be noted that the module division in this embodiment of the invention is illustrative and represents only one logical functional division. In actual implementation, other division methods may be used. For example, in the case of dividing each functional module according to its corresponding function, Figure 14 The initial positioning device shown is only a schematic diagram. The initial positioning device may include a construction module, an acquisition module, a conversion module, and a matching module. The functions of each module of the initial positioning device will be described in detail below.

[0213] A construction module is used to construct multiple particles, each of which has pose information;

[0214] It is understood that this building module can be used to execute the above S11. For details on the implementation of this building module, please refer to the above content related to S11.

[0215] The acquisition module is used to obtain real-time monitoring information of markers based on sensor monitoring data;

[0216] It is understood that this acquisition module can be used to execute the above S12. For details on the implementation of this acquisition module, please refer to the above content related to S12.

[0217] The conversion module is used to convert the real-time monitored marker information into marker information in the map coordinate system based on the pose information of each particle.

[0218] It is understood that this conversion module can be used to execute the above S13. For details on the implementation of this conversion module, please refer to the above content related to S13.

[0219] The matching module is used to match the marker information in the map coordinate system corresponding to each particle with the markers in the pre-constructed marker map, determine the target particle based on the matching result, and obtain the initial pose based on the pose information of the target particle.

[0220] It is understood that this matching module can be used to execute the above S14. For details on the implementation of this matching module, please refer to the above content related to S14.

[0221] In one possible implementation, the above-mentioned obtaining module can be used for:

[0222] Obtain a point cloud obtained from sensor monitoring, wherein the point cloud includes multiple points;

[0223] A three-dimensional mesh is constructed within the three-dimensional space of the point cloud, and the three-dimensional mesh comprises multiple meshes;

[0224] The number of points within each grid is counted, and the real-time monitored marker information is determined based on the number of points within each grid.

[0225] In one possible implementation, the aforementioned obtaining module can specifically be used for:

[0226] The grid is traversed horizontally to count the number of points in each traversed grid.

[0227] If the number of points in the traversed grid is greater than or equal to the first preset number, the grid is marked as occupied.

[0228] In the same vertical direction, if there are more than two preset number of consecutive grids marked as occupied, then the points within those consecutive grids are determined to be points of the marker.

[0229] In one possible implementation, the aforementioned obtaining module can specifically be used for:

[0230] Based on the depth information of each point included in the point cloud, points whose depth information meets the preset requirements are selected, and points whose depth information does not meet the preset requirements are deleted.

[0231] A 3D mesh is constructed within the 3D space of the point cloud formed by the selected points.

[0232] In one possible implementation, the above-mentioned obtaining module can be used for:

[0233] Obtain real-time images and point clouds from sensor monitoring, wherein the point cloud comprises multiple points;

[0234] The real-time image is imported into the target detection model, and the target bounding boxes present in the real-time image are output.

[0235] Based on the relationship between the real-time image and the point cloud, target points in the point cloud corresponding to the target bounding box positions in the real-time image are obtained, and each target point is determined as a marker point.

[0236] In one possible implementation, the aforementioned obtaining module can specifically be used for:

[0237] Project each point in the point cloud onto the real-time image to obtain the projection point corresponding to each point in the real-time image;

[0238] Determine the target projection point within the target bounding box of the real-time image;

[0239] Obtain the corresponding target point in the point cloud for each of the target projection points.

[0240] In one possible implementation, the real-time image is obtained through camera monitoring, and the point cloud is obtained through lidar monitoring; the aforementioned acquisition module can specifically be used for:

[0241] The step of projecting each point in the point cloud onto the real-time image includes:

[0242] Obtain the calibration parameters of the camera and the three-dimensional coordinate information of each point;

[0243] The coordinate information of each point in the camera coordinate system is calculated based on the calibration parameters and the three-dimensional coordinate information of each point.

[0244] The projection point of each point in the real-time image is determined based on the coordinate information of each point in the camera coordinate system.

[0245] In one possible implementation, the matching module described above can be used for:

[0246] For each particle, obtain the corresponding markers in the pre-constructed marker map under the map coordinate system of the particle, and form a marker group;

[0247] Calculate the matching degree between two markers in each of the marker groups;

[0248] The matching degree between the marker information in the map coordinate system corresponding to the particle and the marker in the marker map is obtained based on the matching degree corresponding to each marker group.

[0249] In one possible implementation, the matching module described above can be used for:

[0250] The particle with the highest matching degree among the plurality of particles is obtained, and it is detected whether the highest matching degree is greater than or equal to a preset threshold.

[0251] If the highest matching degree is greater than or equal to the preset threshold, then the particle is determined to be the target particle;

[0252] If the highest matching degree is less than the preset threshold, the pose information of each particle is updated, and the matching degree between the updated marker information in the map coordinate system corresponding to each particle and the marker in the marker map is calculated until the highest matching degree is greater than or equal to the preset threshold, at which point the particle corresponding to the highest matching degree is determined as the target particle.

[0253] In one possible implementation, the matching module described above can be used for:

[0254] Obtain the angular velocity and linear velocity monitored by the sensors;

[0255] Based on the current pose information of each particle, the angular velocity, and the linear velocity, the updated pose information of the particle is calculated.

[0256] In one possible implementation, the matching module described above can be used for:

[0257] Filter out particles with a matching degree greater than or equal to the set threshold;

[0258] For each selected particle, multiple duplicate particles are obtained based on the particle, and the pose information of each duplicate particle is randomly adjusted.

[0259] The pose information of each selected particle and each of the copied particles is updated.

[0260] In one possible implementation, the landmarks in the landmark map include multiple types, and the initial positioning device further includes a setup module, which can be used for:

[0261] Construct sub-maps corresponding to various types of landmarks;

[0262] The markers in each sub-map are filtered so that each sub-map contains only one marker at every preset range.

[0263] The landmark map is obtained by merging multiple sub-maps.

[0264] In one possible implementation, the establishment module can be used to:

[0265] Obtain keyframe point clouds, keyframe images, and keyframe poses corresponding to multiple consecutive keyframes.

[0266] For each of the key frames, candidate markers are determined based on the key frame point cloud, key frame image, and key frame pose corresponding to the key frame.

[0267] Check if a marker identical to the candidate marker already exists. If it does, delete the candidate marker; otherwise, place the candidate marker in the marker map.

[0268] Please see Figure 15 This is a structural block diagram of an electronic device provided in an embodiment of the present invention. The electronic device can be a computer device, a microprocessor, a microcontroller, etc., and includes a memory, a processor, and a communication module. The memory, processor, and communication module are electrically connected directly or indirectly to each other to achieve data transmission or interaction. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0269] The memory is used to store programs or data. The memory can be, but is not limited to, Random Access Memory (RAM), Read Only Memory (ROM), Programmable Read-Only Memory (PROM), Erasable Programmable Read-Only Memory (EPROM), Electrically Erasable Programmable Read-Only Memory (EEPROM), etc.

[0270] The processor is used to read / write data or programs stored in the memory and to execute the initial positioning method provided in any embodiment of the present invention.

[0271] The communication module is used to establish communication connections between electronic devices and other communication terminals via a network, and to send and receive data via the network.

[0272] It should be understood that, Figure 15 The structure shown is only a schematic diagram of an electronic device; the electronic device may also include components that are larger than those shown. Figure 15 The more or fewer components shown, or having the same Figure 15 The different configurations shown.

[0273] Furthermore, embodiments of the present invention also provide a computer-readable storage medium storing machine-executable instructions, which, when executed, implement the initial positioning method provided in the above embodiments.

[0274] Specifically, the computer-readable storage medium can be a general-purpose storage medium, such as a removable disk or hard disk. When the computer program on the computer-readable storage medium is run, it can execute the aforementioned initial positioning method. The processes involved in the execution of the executable instructions on the computer-readable storage medium can be referred to the relevant descriptions in the above method embodiments, and will not be detailed here.

[0275] This invention provides a computer program product comprising a computer program (also referred to as code or instructions) that, when executed, causes a computer to perform the method described in any possible implementation of the method embodiments of this invention.

[0276] This invention provides a chip system comprising a processor and a memory, the memory being used to store one or more programs; when the processor executes the one or more programs, it implements the method described in any possible implementation of the method embodiments of this invention.

[0277] In summary, the initial positioning method, apparatus, electronic device, and computer-readable storage medium provided in this invention construct multiple particles, each with pose information, and obtain real-time monitored landmark information based on sensor monitoring data. Then, based on the pose information of each particle, the real-time monitored landmark information is converted into landmark information in a map coordinate system. The landmark information in the map coordinate system corresponding to each particle is matched with landmarks in a pre-constructed landmark map. The target particle is determined based on the matching result, and the initial pose is obtained based on the pose information of the target particle. This solution achieves pose initialization by monitoring landmarks and matching them with landmarks in a pre-constructed map, reducing implementation complexity, meeting real-time application requirements, and applicable to various application scenarios.

[0278] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the technical scope disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. An initial positioning method, characterized in that, The method includes: Construct multiple particles, each of which has pose information; Real-time monitoring information of markers is obtained based on sensor monitoring data; Based on the pose information of each particle, the real-time monitored marker information is converted into marker information in the map coordinate system; The marker information in the map coordinate system corresponding to each particle is matched with the markers in the pre-constructed marker map. The target particle is determined based on the matching result, and the initial pose is obtained based on the pose information of the target particle.

2. The initial positioning method according to claim 1, characterized in that, The step of obtaining real-time monitored marker information based on sensor monitoring data includes: Obtain a point cloud obtained from sensor monitoring, wherein the point cloud includes multiple points; A three-dimensional mesh is constructed within the three-dimensional space of the point cloud, and the three-dimensional mesh comprises multiple meshes; The number of points within each grid is counted, and the real-time monitored marker information is determined based on the number of points within each grid.

3. The initial positioning method according to claim 2, characterized in that, The step of counting the number of points within each grid and determining the real-time monitored marker information based on the number of points within each grid includes: The grid is traversed horizontally to count the number of points in each traversed grid. If the number of points in the traversed grid is greater than or equal to the first preset number, the grid is marked as occupied. In the same vertical direction, if there are more than two preset number of consecutive grids marked as occupied, then the points within those consecutive grids are determined to be points of the marker.

4. The initial positioning method according to claim 2, characterized in that, The step of constructing a three-dimensional mesh within the three-dimensional space of the point cloud includes: Based on the depth information of each point included in the point cloud, points whose depth information meets the preset requirements are selected, and points whose depth information does not meet the preset requirements are deleted. A 3D mesh is constructed within the 3D space of the point cloud formed by the selected points.

5. The initial positioning method according to claim 2, characterized in that, The step of obtaining real-time monitored marker information based on sensor monitoring data includes: Obtain real-time images and point clouds from sensor monitoring, wherein the point cloud comprises multiple points; The real-time image is imported into the target detection model, and the bounding boxes of the targets present in the real-time image are output. Based on the relationship between the real-time image and the point cloud, target points in the point cloud corresponding to the target bounding box positions in the real-time image are obtained, and each target point is determined as a marker point.

6. The initial positioning method according to claim 5, characterized in that, The step of obtaining the target point in the point cloud corresponding to the target bounding box position in the real-time image based on the relationship between the real-time image and the point cloud includes: Project each point in the point cloud onto the real-time image to obtain the projection point corresponding to each point in the real-time image; Determine the target projection point within the target bounding box of the real-time image; Obtain the corresponding target point in the point cloud for each of the target projection points.

7. The initial positioning method according to claim 6, characterized in that, The real-time images are obtained through camera monitoring, and the point clouds are obtained through lidar monitoring. The step of projecting each point in the point cloud onto the real-time image includes: Obtain the calibration parameters of the camera and the three-dimensional coordinate information of each point; The coordinate information of each point in the camera coordinate system is calculated based on the calibration parameters and the three-dimensional coordinate information of each point. The projection point of each point in the real-time image is determined based on the coordinate information of each point in the camera coordinate system.

8. The initial positioning method according to claim 1, characterized in that, The step of matching the marker information in the map coordinate system corresponding to each particle with the markers in the pre-constructed marker map includes: For each particle, obtain the corresponding markers in the pre-constructed marker map under the map coordinate system of the particle, and form a marker group; Calculate the matching degree between two markers in each of the marker groups; The matching degree between the marker information in the map coordinate system corresponding to the particle and the marker in the marker map is obtained based on the matching degree corresponding to each marker group.

9. The initial positioning method according to claim 8, characterized in that, The step of determining the target particle based on the matching result includes: The particle with the highest matching degree among the plurality of particles is obtained, and it is detected whether the highest matching degree is greater than or equal to a preset threshold. If the highest matching degree is greater than or equal to the preset threshold, then the particle is determined to be the target particle; If the highest matching degree is less than the preset threshold, the pose information of each particle is updated, and the matching degree between the updated marker information in the map coordinate system corresponding to each particle and the marker in the marker map is calculated until the highest matching degree is greater than or equal to the preset threshold, at which point the particle corresponding to the highest matching degree is determined as the target particle.

10. The initial positioning method according to claim 9, characterized in that, The step of updating the pose information of each particle includes: Obtain the angular velocity and linear velocity monitored by the sensors; Based on the current pose information of each particle, the angular velocity, and the linear velocity, the updated pose information of the particle is calculated.

11. The initial positioning method according to claim 9, characterized in that, The step of updating the pose information of each particle includes: Filter out particles with a matching degree greater than or equal to the set threshold; For each selected particle, multiple duplicate particles are obtained based on the particle, and the pose information of each duplicate particle is randomly adjusted. The pose information of each selected particle and each of the copied particles is updated.

12. The initial positioning method according to any one of claims 1-11, characterized in that, The landmark map includes multiple types of landmarks, and the method further includes a step of pre-constructing the landmark map, which includes: Construct sub-maps corresponding to various types of landmarks; The markers in each sub-map are filtered so that each sub-map contains only one marker at every preset range. The landmark map is obtained by merging multiple sub-maps.

13. The initial positioning method according to any one of claims 1-11, characterized in that, The method also includes a step of pre-constructing the landmark map, which includes: Obtain keyframe point clouds, keyframe images, and keyframe poses corresponding to multiple consecutive keyframes. For each of the key frames, candidate markers are determined based on the key frame point cloud, key frame image, and key frame pose corresponding to the key frame. Check if a marker identical to the candidate marker already exists. If it does, delete the candidate marker; otherwise, place the candidate marker in the marker map.

14. An initial positioning device, characterized in that, The device includes: A construction module is used to construct multiple particles, each of which has pose information; The acquisition module is used to obtain real-time monitoring information of markers based on sensor monitoring data; The conversion module is used to convert the real-time monitored marker information into marker information in the map coordinate system based on the pose information of each particle. The matching module is used to match the marker information in the map coordinate system corresponding to each particle with the markers in the pre-constructed marker map, determine the target particle based on the matching result, and obtain the initial pose based on the pose information of the target particle.

15. An electronic device, characterized in that, The device includes one or more storage media and one or more processors communicating with the storage media, wherein the one or more storage media store machine-executable instructions executable by the processor, and when the electronic device is running, the processor executes the machine-executable instructions to perform the method according to any one of claims 1-13.

16. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores machine-executable instructions that, when executed by a processor, implement the method described in any one of claims 1-13.

Images