Map construction method, device and autonomous mobile device

Abstract

The present application provides a map construction method, device and autonomous mobile device, which relate to the field of artificial intelligence technology. The method includes: collecting environmental information in the parking lot through a ranging sensor provided on the autonomous mobile device to obtain a current point cloud frame; selecting a current local point cloud from the current point cloud frame; judging whether the posture of the autonomous mobile device has changed based on the current map and the current local point cloud; when it is determined that the posture has changed, using the current point cloud frame as the Nth point cloud frame, and updating the current map based on the Nth point cloud frame. The map construction method provided by the present application can significantly reduce invalid map update operations in parking lots with frequent environmental changes by extracting local information within a preset height range from the environmental information and determining whether the map needs to be updated based on the local information. This can significantly reduce the consumption of computing resources and save computing costs while ensuring high-precision positioning of the autonomous mobile device.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a map construction method, device, and autonomous mobile device. Background Art

[0002] Signal shielding is common in parking lots, making it difficult for autonomous mobile devices (such as vehicles in autonomous driving mode) to locate themselves using absolute positioning methods like the Global Positioning System (GPS). Position estimation methods based on internal sensing sensors like inertial measurement units (IMUs) are prone to cumulative errors over extended positioning processes, leading to severe position drift and failing to meet the positioning accuracy required for functions like autonomous parking.

[0003] In order to improve the positioning accuracy of autonomous mobile devices in environments with signal shielding, related technologies provide a positioning solution using SLAM (Simultaneous Localization and Mapping) technology. In the process of positioning using SLAM technology, in order to meet the required positioning accuracy, the map needs to be updated in real time according to environmental changes. However, the frequent entry and exit of vehicles and people in parking lots causes the environment near autonomous mobile devices to constantly change. In this case, even if the position of the autonomous mobile device itself has not changed, the map will be frequently updated, resulting in excessive consumption of computing resources and high computing costs. Summary of the Invention

[0004] In view of this, embodiments of the present application provide a map construction method, apparatus, autonomous mobile device, electronic device, and computer-readable storage medium, which can improve the positioning accuracy of autonomous mobile devices in parking lots while effectively avoiding excessive consumption of computing resources.

[0005] A first aspect of the present application provides a map construction method, comprising: collecting environmental information in a parking lot through a ranging sensor provided on an autonomous mobile device to obtain a current point cloud frame; selecting a current local point cloud from the current point cloud frame, wherein the current local point cloud is composed of points in the current point cloud frame that are located above a preset static height in the parking lot; judging whether the posture of the autonomous mobile device has changed based on the current map and the current local point cloud, wherein the current map is a map of the parking lot constructed based on the 1st point cloud frame to the N-1th point cloud frame, where N is an integer greater than 1; when it is determined that the posture has changed, using the current point cloud frame as the Nth point cloud frame, and updating the current map based on the Nth point cloud frame.

[0006] In some implementations, the map construction method provided in the first aspect of the present application further includes: when it is determined that the posture has not changed, determining that the current map does not need to be updated.

[0007] In some implementations, the preset static height is equal to 0.5 to 0.9 times the height of the space in the parking lot.

[0008] In some implementations, updating the current map based on the Nth point cloud frame includes: determining that the Nth point cloud frame includes point cloud information corresponding to a specific object, wherein the specific object is a static object of a preset type located in a parking lot; extracting semantic information of the specific object based on the Nth point cloud frame as the Nth semantic information corresponding to the Nth point cloud frame; matching the Nth point cloud frame with the N-1th point cloud frame based on the Nth semantic information and the N-1th semantic information corresponding to the N-1th point cloud frame; and updating the current map based on the matching result.

[0009] In some implementations, the map construction method provided in the first aspect of the present application further includes: determining that the Nth point cloud frame includes point cloud information of a newly added specific object by comparing the Nth semantic information and the N-1th semantic information, wherein the newly added specific object is a specific object that does not appear in the N-1th point cloud frame; and adding the semantic information of the newly added specific object to the position corresponding to the newly added specific object in the current map.

[0010] In some implementations, the current map is a probabilistic grid map, wherein semantic information of the newly added specific object is added to the position corresponding to the newly added specific object in the current map, including: determining a specific grid corresponding to the newly added specific object in the current map; and adding the semantic information of the newly added specific object to the semantic list of the specific grid.

[0011] In some implementations, the map construction method provided in the first aspect of the present application further includes: performing loop detection based on the Nth semantic information and all semantic information in the current map; and updating the current map based on the loop detection result.

[0012] In some implementations, the preset types of static objects include traffic safety signs, guidance signs, and auxiliary facilities.

[0013] The second aspect of the present application provides a map construction device, including: an acquisition module, which is used to collect environmental information in a parking lot through a ranging sensor provided on an autonomous mobile device to obtain a current point cloud frame; a data selection module, which is used to select a current local point cloud from the current point cloud frame, wherein the current local point cloud is composed of points in the current point cloud frame that are located above a preset static height in the parking lot; a judgment module, which is used to judge whether the posture of the autonomous mobile device has changed based on the current map and the current local point cloud, wherein the current map is a map of the parking lot constructed based on the 1st point cloud frame to the N-1th point cloud frame, where N is an integer greater than 1; and a calculation module, which is used to use the current point cloud frame as the Nth point cloud frame when it is determined that the posture has changed, and to update the current map based on the Nth point cloud frame.

[0014] The third aspect of the present application provides an autonomous mobile device, comprising: a ranging sensor for collecting environmental information in a parking lot; and a processor for executing any one of the map construction methods provided in the first aspect of the present application.

[0015] A fourth aspect of the present application provides an electronic device, comprising: a memory for storing computer instructions; and a processor for executing the computer instructions to implement any one of the map construction methods provided in the first aspect of the present application.

[0016] The fifth aspect of the present application provides a computer-readable storage medium storing computer instructions. When the computer instructions are executed by a processor, any one of the map construction methods provided in the first aspect of the present application is implemented.

[0017] The map construction method, apparatus, autonomous mobile device, electronic device, and computer-readable storage medium provided in the present application can significantly reduce ineffective map update operations in parking lot scenarios with frequent environmental changes by extracting local information within a preset height range from environmental information collected by a ranging sensor and determining whether a map update is needed based on the local information. This can thereby significantly reduce the consumption of computing resources and save computing costs while ensuring high-precision positioning of the autonomous mobile device.

[0018] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory and should not be construed as limiting the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to make the purpose, technical solutions and advantages of the embodiments of the present application more clear, the embodiments of the present application will be further described in detail below with reference to the accompanying drawings. It should be understood that the accompanying drawings constitute part of the specification and are used together with the embodiments of the present application to explain the present application and do not constitute a limitation of the present application. Unless otherwise specified, in the drawings, the same symbols and numbers generally represent the same steps or components.

[0020] Figure 1 Shown is a schematic diagram of an exemplary map construction system provided in an embodiment of the present application.

[0021] Figure 2 FIG2 is a flow chart of a map construction method provided in an embodiment of the present application.

[0022] Figure 3 The figure shows a flowchart of an exemplary map construction method provided in an embodiment of the present application.

[0023] Figure 4 Shown is a flowchart of another exemplary map construction method provided in an embodiment of the present application.

[0024] Figure 5 Shown is a schematic diagram of a 2.5D probability grid map containing a semantic list in an exemplary map construction method provided in an embodiment of the present application.

[0025] Figure 6 The figure shows a flowchart of an exemplary map construction method provided in an embodiment of the present application.

[0026] Figure 7 FIG2 is a schematic diagram of a map construction device provided in an embodiment of the present application.

[0027] Figure 8 Shown is a schematic diagram of an autonomous mobile device provided in an embodiment of the present application.

[0028] Figure 9 Shown is a schematic diagram of an exemplary electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0029] To achieve autonomous parking in parking lots, autonomous mobile devices must be able to navigate within narrow and crowded spaces. High-precision positioning is a crucial prerequisite for autonomous navigation. However, indoor parking lots (especially underground ones) often experience severe signal obstruction, making absolute positioning technologies like GPS inoperable. This results in low positioning accuracy for traditional integrated navigation and positioning solutions, making them incapable of meeting the requirements of autonomous navigation.

[0030] To achieve positioning in such scenarios, in some implementations, motion information collected by interoceptive sensors on autonomous mobile devices (such as inertial measurement units (IMUs) and wheel speed encoders) can be used to measure position and posture, thereby achieving positioning through inertial navigation. However, due to the cumulative errors typically associated with inertial navigation, long-term positioning using this method can easily lead to severe position drift, and thus still cannot meet high-precision positioning requirements.

[0031] In this context, SLAM technology has become the preferred tool for solving the problem of positioning accuracy in parking lots. SLAM technology is based on the environmental information collected by the external sensing sensors (cameras, lidar, etc.) on autonomous mobile devices, continuously sensing changes in the surrounding environment, and inferring changes in the position and posture of the autonomous mobile device itself based on changes in the environment, thereby achieving high-precision positioning and mapping. However, there are usually many vehicles and people frequently entering and exiting parking lots, so even if the autonomous mobile device is stationary, the environment around it may be constantly changing. In other words, if the map is updated in real time every time an environmental change is sensed, it may be necessary to frequently perform updates even when the posture of the autonomous mobile device has not changed. Obviously, the map update in this case does not make a substantial contribution to the positioning of the autonomous mobile device, but instead leads to excessive consumption of computing resources.

[0032] In view of this, an embodiment of the present application provides a map construction method that can achieve high-precision positioning of autonomous mobile devices based on SLAM technology in parking lots with signal shielding, while avoiding invalid updates of maps, thereby effectively reducing the consumption of computing resources and lowering positioning costs.

[0033] Exemplary Systems

[0034] Figure 1 FIG2 is a schematic diagram of an exemplary map construction system 100 provided in an embodiment of the present application. The map construction system 100 may include an autonomous mobile device 110 , a processing device 120 , and a storage device 130 .

[0035] The autonomous mobile device 110 may be a vehicle or robot with an autonomous driving mode. In order to realize the function of autonomous movement, the autonomous mobile device 110 needs to be provided with a sensor for sensing the surrounding environment. In an embodiment of the present application, the autonomous mobile device 110 is provided with at least a ranging sensor 111 for collecting environmental information. The ranging sensor 111 may be, for example, a laser radar (e.g., a mechanical rotating 3D laser radar, a solid-state / semi-solid-state 3D laser radar), a binocular vision sensor, or a structured light sensor (e.g., an RGB-D camera), which can obtain 3D point cloud information of the environmental space within its acquisition range.

[0036] Processing device 120 can be communicatively connected to ranging sensor 111 to receive environmental information collected by ranging sensor 111 and perform required processing actions based on the environmental information. For example, processing device 120 can be used to measure the position and posture of autonomous mobile device 110, construct a map of the environment in which autonomous mobile device 110 is located, and communicate with other types of sensors to achieve multi-sensor information fusion.

[0037] The storage device 130 can be communicatively connected to the processing device 120 so that the processing device 120 can obtain corresponding data from the storage device 130 according to the processing action to be performed. The storage device 130 can also be used to receive and store data processed by the processing device 120.

[0038] It should be understood that the processing device 120 can be a processor or server, or it can be an electronic device with processing capabilities, such as a computer, mobile phone, tablet computer, or in-vehicle computer. The processing device 120 can be located in the cloud or locally on the autonomous mobile device 110, as long as it can communicate with the various functional modules on the autonomous mobile device 110. The storage device 130 can be a server device located in the cloud or a local storage medium. The embodiments of this application do not limit the specific implementation of the processing device 120 and the storage device 130.

[0039] Exemplary Methods

[0040] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the embodiments described are only part of the embodiments of the present application, rather than all the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments in the present application are within the scope of protection of this application.

[0041] Figure 2 The figure shows a flow chart of a map construction method provided by an embodiment of the present application. Figure 1 The processing device 120 in the map construction system 100 shown in FIG. Figure 2 As shown, the method may include the following contents.

[0042] S210: Collect environmental information in the parking lot through a ranging sensor provided on the autonomous mobile device to obtain a current point cloud frame.

[0043] Specifically, autonomous mobile devices can be Figure 1 In the autonomous mobile device 110 in the map construction system 100 shown, the ranging sensor may be a ranging sensor 111 provided on the autonomous mobile device 110. The ranging sensor can obtain a point cloud frame containing environmental information by collecting spatial depth information of the surrounding environment.

[0044] When an autonomous mobile device is located in a parking lot, the ranging sensor can collect environmental information within the parking lot at a preset frequency or at specific time points to obtain multiple point cloud frames. In the embodiments of the present application, the point cloud frame obtained by the ranging sensor in the current acquisition is referred to as the current point cloud frame.

[0045] It is understood that the ranging sensor is fixedly mounted on the autonomous mobile device. That is, during the operation of the autonomous mobile device, the position and posture of the ranging sensor relative to the autonomous mobile device do not change. Therefore, when there is a difference between two consecutive point cloud frames collected by the ranging sensor, it indicates that the environment in which the autonomous mobile device is located has changed. This change may be caused by a change in the position and posture of the autonomous mobile device itself, a change in the external environment near the autonomous mobile device (within the acquisition range of the ranging sensor), or a change in both the autonomous mobile device and the external environment.

[0046] S220: Select the current local point cloud from the current point cloud frame.

[0047] In some preferred embodiments, the current local point cloud is composed of points in the current point cloud frame that are located above a preset static height in the parking lot.

[0048] As previously analyzed, parking lots often have numerous vehicles and people frequently entering and exiting, causing the environment near the autonomous mobile device to constantly change. This requires constant map updates to meet positioning requirements, resulting in a significant amount of inefficient computation. However, the inventors discovered that the range of movement of vehicles and people is limited in height. Therefore, the frequent entry and exit of vehicles and people only causes changes to the local environment within the parking lot, while the environmental information in the spaces above this local environment remains unchanged.

[0049] Based on this finding, in an embodiment of the present application, a preset static height can be set according to the height of the general activity range of vehicles and personnel, and whether the current map needs to be updated can be determined based on the changes in depth information of spatial points above the preset static height in the parking lot space.

[0050] Specifically, after acquiring the current point cloud frame through the ranging sensor, points located above a preset static height can be selected from the current point cloud frame to obtain the current local point cloud. In other words, the current local point cloud retains points in the current point cloud frame whose height is greater than the preset static height. When the external environment changes, resulting in a difference between the current point cloud frame and the previous point cloud frame, and the autonomous mobile device does not actually change its position, there may be no difference between the current local point cloud and the local point cloud corresponding to the previous point cloud frame. Therefore, it can be determined that the current map does not need to be updated.

[0051] In some implementations, the current point cloud frame may be subjected to a through-filtering process according to a preset static height to obtain a current local point cloud, and a subsequent map update process may be performed based on the current local point cloud.

[0052] As an example, the preset static height can be equal to 0.5 to 0.9 times the height of the parking space. As another example, since most vehicles are less than 1.6 meters tall and parking spaces are generally higher than 2.2 meters tall, in order to minimize interference from other vehicles while ensuring sufficient spatial information for calculation, the preset static height can be set to any value greater than or equal to 1.6 meters and less than or equal to 2.0 meters.

[0053] In other optional embodiments, the preset static height may include a preset maximum static height and a preset minimum static height. In this case, the current local point cloud in S220 may be composed of points in the current point cloud frame that are above the preset maximum static height in the parking lot and points that are below the preset minimum static height in the parking lot.

[0054] S230: Determine whether the position of the autonomous mobile device has changed based on the current map and the current local point cloud.

[0055] The current map is a map of the parking lot constructed based on the 1st point cloud frame to the N-1th point cloud frame, where N is an integer greater than 1.

[0056] S240: When it is determined that the posture has changed, the current point cloud frame is used as the Nth point cloud frame, and the current map is updated according to the Nth point cloud frame.

[0057] Specifically, the current map can be a map of the parking lot constructed based on a previous point cloud frame. By comparing the current map with the current local point cloud, it can be determined whether the position of the autonomous mobile device has changed. If the position of the autonomous mobile device has changed, the current map can be updated based on the environmental information contained in the current point cloud frame.

[0058] Here, the previous point cloud frame refers to the point cloud frame captured by the ranging sensor in the parking lot before the current point cloud frame. For example, the current point cloud frame may be the Nth point cloud frame, and the previous point cloud frames may include the 1st to the N-1th point cloud frames. Preferably, to conserve computing and storage resources, after confirming that the position and posture of the autonomous mobile device have indeed changed, the current point cloud frame may be determined as the Nth point cloud frame, and the current map may be updated based on this Nth point cloud frame.

[0059] In some implementations, when updating the current map based on the Nth point cloud frame, the map may be updated using all environmental information contained in the Nth point cloud frame, or only using environmental information in the Nth point cloud frame that is located above a preset static altitude (or below a minimum preset static altitude and above a maximum preset static altitude). The specific range of information used may be selected by those skilled in the art based on actual needs and is not limited in this embodiment of the present application.

[0060] It should be understood that in the embodiments of the present application, the first point cloud frame may refer to the first point cloud frame collected by the ranging sensor after entering the current parking lot, the first point cloud frame collected by the ranging sensor after entering the current floor, or the first point cloud frame collected by the ranging sensor after entering any range defined based on actual needs. In other words, the specific definition of the first point cloud frame can be designed by those skilled in the art based on actual needs, and the embodiments of the present application do not limit this.

[0061] The map construction method provided in the embodiment of the present application can significantly reduce invalid map update operations in parking lot scenarios with frequent environmental changes by extracting local information within a preset height range from the environmental information collected by the ranging sensor and determining whether the map needs to be updated based on the local information. This can ensure high-precision positioning of autonomous mobile devices while significantly reducing the consumption of computing resources and saving computing costs.

[0062] Accordingly, in some embodiments, based on the determination result in S230, the map construction method may further include: determining that the current map does not need to be updated when it is determined that the position and posture of the autonomous mobile device have not changed. Specifically, by comparing the current map and the current local point cloud in step S230, if it is determined that the position and posture of the autonomous mobile device have not changed, it may be determined that the current map does not need to be updated.

[0063] Figure 3 The figure shows a flowchart of a map construction method provided by another embodiment of the present application. Figure 1 The processing device 120 in the illustrated map building system 100 executes.

[0064] Typically, the ground height of a parking lot does not vary significantly within a single floor; in other words, the floor of a parking lot can be considered a flat surface. Therefore, when an autonomous mobile device moves within a single floor of a parking lot, its position can be considered to involve changes in only three degrees of freedom (x, y, and yaw). Based on this, in some implementations, a 2D probability grid map of the parking lot can be constructed based on the environmental information collected by ranging sensors, thereby enabling the positioning of the autonomous mobile device.

[0065] Understandably, 2D probabilistic grid maps are a relatively mature solution for indoor structured environments in related technologies. However, during implementation, the inventors discovered that parking lot environments differ significantly from typical indoor structured environments. Directly adopting the 2D probabilistic grid map-based implementation approach in related technologies for map construction can easily lead to scene degradation and significant positioning errors.

[0066] Through detailed analysis, the inventors discovered that the root cause of the aforementioned issues lies in the high degree of scene similarity caused by the large number of repetitive structures in parking lots (for example, identical columns, identical signage, similar elevator doors, and stairway entrances). This characteristic results in the point cloud frames collected by the ranging sensor at different positions and angles potentially containing a large amount of similar environmental information. This, in turn, can easily lead to mismatches during the front-end matching and loop closure detection processes in the SLAM solution, resulting in incorrect pose constraints being constructed during the back-end optimization process, ultimately leading to deviations in the pose estimation results.

[0067] In order to solve this problem, through a lot of investigation work, the inventors realized that there are usually some specific objects in parking lots, such as: traffic safety signs such as lane lines, stop lines, speed bumps, entrance and exit signs, elevator signs, stairwell signs and other guiding signs, as well as parking space number plates, parking lot area number signs, reflectors, anti-collision columns and other auxiliary facilities. These specific objects are generally in a stationary state, their positions are relatively stable, and they are usually set at different positions in the parking lot according to their respective uses. Based on these characteristics, in the embodiments of the present application, the semantic features of these specific objects can be extracted and used in the process of constructing the map (especially in the steps of front-end matching and loop detection), and on the basis of the feature point matching constraint relationship, additional semantic matching constraint relationships can be further constructed, thereby effectively improving the accuracy of the posture constraint relationship and avoiding deviations in the posture estimation results.

[0068] Based on this concept, Figure 2 Based on the embodiments shown, this application Figure 3 The embodiment shown provides a specific implementation method for the above step S240. Figure 3 As shown, step S240 may include the following contents.

[0069] S341: When it is determined that the posture has changed, the current point cloud frame is used as the Nth point cloud frame, and it is determined that the Nth point cloud frame includes point cloud information corresponding to the specific object.

[0070] The specific objects are static objects of a preset type located in the parking lot, for example, static objects of a preset type may include traffic safety signs, guidance signs, and auxiliary facilities.

[0071] Specifically, when these specific objects are within the acquisition range of the ranging sensor, the point cloud frames collected by the ranging sensor will also contain feature points corresponding to these specific objects. Therefore, a preset type of static object can be pre-selected as the specific object, and after obtaining the point cloud frame, it can be determined whether the point cloud frame includes point cloud information corresponding to the specific object based on the spatial characteristics of the specific object. For the Nth point cloud frame, when it is determined that the point cloud frame includes point cloud information corresponding to the specific object, the positional relationship between the autonomous mobile device and the specific object can be calculated based on the external parameters of the ranging sensor, and the location of the specific object can be determined on the map.

[0072] S342: Extracting semantic information of a specific object according to the Nth point cloud frame as Nth semantic information corresponding to the Nth point cloud frame.

[0073] Specifically, the semantic features of a specific object can be obtained through a pre-trained neural network model. For example, the Nth point cloud frame can be input into the neural network model to segment and identify the Nth point cloud frame, thereby obtaining the corresponding semantic information. Here, the Nth point cloud frame may include only one specific object or multiple specific objects, which is not limited in the embodiments of the present application. When the Nth point cloud frame includes multiple specific objects, the set of semantic information corresponding to these specific objects can be used as the Nth semantic information corresponding to the Nth point cloud frame.

[0074] It can be understood that when no specific object appears in a point cloud frame, it can be directly determined that the point cloud frame does not include the specific object and does not have corresponding semantic information.

[0075] S343: Match the Nth point cloud frame with the N-1th point cloud frame according to the Nth semantic information and the N-1th semantic information corresponding to the N-1th point cloud frame.

[0076] While matching feature points of the Nth point cloud frame with the N-1th point cloud frame, semantic information matching can be performed on the Nth point cloud frame with the N-1th point cloud frame based on the Nth semantic information corresponding to the Nth point cloud frame and the N-1th semantic information corresponding to the N-1th point cloud frame to determine whether a new specific object currently appears in the environment near the autonomous mobile device.

[0077] S344: Update the current map according to the matching results.

[0078] By combining the matching results of the semantic information and the matching results of the feature points, the newly added environmental information can be determined, and then the current map can be updated according to the newly added environmental information.

[0079] Optionally, in Figure 4 In an exemplary implementation shown in FIG. Figure 3The map construction method shown may also include the following contents.

[0080] S445: Determine that the Nth point cloud frame includes point cloud information of the newly added specific object by comparing the Nth semantic information with the N-1th semantic information.

[0081] The newly added specific object is a specific object that does not appear in the N-1th point cloud frame.

[0082] S446: Adding semantic information of the newly added specific object to the position corresponding to the newly added specific object in the current map.

[0083] Specifically, when it is determined that a new specific object appears in the Nth point cloud frame, that is, a new specific object is added, the semantic information corresponding to the new specific object can be added to the current map while the current map is updated.

[0084] In some preferred implementations, the current map may be a probabilistic grid map. Accordingly, the above S446 may specifically include: determining a specific grid corresponding to the newly added specific object in the current map; and adding semantic information of the newly added specific object to the semantic list of the specific grid.

[0085] As an example, the current map can be a 2D probability grid map, wherein a semantic information list can be set for each grid to add the semantic information corresponding to the grid (i.e., the semantic information of the specific object located at the corresponding position of the grid). In other words, improvements can be made on the basis of the 2D probability grid map to obtain, for example Figure 5 The 2.5D probability grid map shown contains a semantic list, and the parking lot map is updated and the high-precision positioning of the autonomous mobile device is achieved based on the 2.5D probability grid map.

[0086] Optionally, in another exemplary implementation, Figure 3 or Figure 4 The map construction method shown may further include: performing loop detection based on the Nth semantic information and all semantic information in the current map; and updating the current map based on the loop detection result.

[0087] The map construction method provided in the embodiment of the present application utilizes specific static objects preset in the parking lot scene to obtain semantic information of the specific static objects during the mapping process. It then expands on a general map (e.g., a 2D probability grid map) and stores semantic information such as lane lines, guide signs, parking lines, speed bumps, etc. in the scene in the map, forming a map with semantic information (e.g., a 2.5D probability grid map). By adding matching constraints based on semantic information, the accuracy of the front-end matching and loop detection links in the SLAM process can be improved, thereby effectively avoiding the mismatch problem caused by the presence of a large number of highly similar scenes in the parking lot, and significantly improving the pose estimation accuracy of autonomous mobile devices in the parking lot.

[0088] Figure 6 The figure shows a flowchart of an exemplary map construction method provided in an embodiment of the present application. Figure 6 The method shown is Figure 2 A specific implementation of the embodiment, to avoid repetition, mainly describes the differences here. Figure 6 As shown, the method may include the following contents.

[0089] S610: fusing data collected by multiple sensors on the autonomous mobile device based on an extended Kalman filter algorithm to make a preliminary estimate of the position and posture of the autonomous mobile device.

[0090] In this implementation, the autonomous mobile device can be a vehicle suitable for using SLAM technology, such as an autonomous vehicle or mobile robot. The various sensors can include, for example, wheel speed encoders, inertial measurement units (IMUs), and ranging sensors. The ranging sensors can be mechanical rotary 3D lidars, solid-state / semi-solid-state 3D lidars, or structured light cameras, all of which can provide 3D point cloud information about the surrounding space.

[0091] Specifically, in some embodiments, in order to correctly apply the extended Kalman filter algorithm to perform the update / correction step, before executing S610, the state equation and the observation equation may be established in the following manner.

[0092] First, a coordinate system for the autonomous mobile device can be established. The x-axis points in the direction of travel of the autonomous mobile device, the y-axis points to the left of the autonomous mobile device, and the z-axis points upwards according to the right-hand rule. It should be understood that this embodiment of the present application does not consider special situations such as sideslipping and bumping of the autonomous mobile device, that is, the velocity and acceleration of the autonomous mobile device along the y and z axes are ignored.

[0093] Based on this, the system status can be established as:

[0094] X=[xyz roll pitch yaw vx v roll v pitch v yaw a x ]

[0095] Then the state equation of the system can be established as follows:

[0096] X k+1 =f(X k )+Q k

[0097] Where, X k 、X k+1 They represent the system states at time k and k+1 respectively, Q k represents the process noise at time k, and f(·) represents the nonlinear function.

[0098] If the acceleration / angular velocity constant model is used to predict the system state during the filter prediction phase, the nonlinear function f(·) can be represented by the state transfer matrix A. The diagonal elements of the state transfer matrix A are all 1, and all off-diagonal elements are 0, except for the off-diagonal elements described below.

[0099] A(x,v x )=cos(yaw)*cos(pitch)*Δt

[0100] A(x,a x )=0.5*A(x,v x )*Δt

[0101] A(y,v x )=sin(yaw)*cos(pitch)*Δt

[0102] A(y,a x )=0.5*A(y,v x )*Δt

[0103] A(z,v x )=―sin(pitch)*Δt

[0104] A(z,a x )=0.5*A(z,v x )*Δt

[0105] A(roll,v roll )=Δt

[0106] A(roll,v pitch )=sin(roll)*tan(pitch)*Δt

[0107] A(roll,v yaw)=cos(roll)*tan(pitch)*Δt

[0108] A(pitch,v pitch )=cos(roll)*Δt

[0109] A(pitch,v yaw )=―sin(roll)*Δt

[0110] A(yaw,v pitch )=sin(roll) / cos(pitch)*Δt

[0111] A(yaw,v yaw )=cos(roll) / cos(pitch)*Δt

[0112] A(v x ,a x )=Δt

[0113] The inertial measurement unit can output angular velocity and acceleration values, and the wheel speed encoder can output the speed along the x-axis and the angular velocity around the z-axis. Based on this, the measurement values ​​of each sensor can be directly used as observation values ​​to correct the system state.

[0114] Specifically, the system observation equation is established as follows:

[0115] Y k+1 =h(x k+1 )+R k+1

[0116] Where Y k+1 is the system observation at time k+1, R k+1 is the observation noise at time k+1, and h(·) is the system observation function. It should be understood that in the embodiment of the present application, the measured values ​​are used as observation values, so the matrix of the observation function is represented as the unit matrix.

[0117] Furthermore, considering that the output of the wheel speed encoder is a pulse signal, a simple mathematical model can be used to convert the pulse signal into the linear velocity of the left and right wheels of the autonomous mobile device. In the embodiments of the present application, a two-wheel differential model as shown below can be used to convert the linear velocity of the left and right wheels into the corresponding position and posture of the autonomous mobile device.

[0118] v x =(v r +v l ) / 2

[0119] ω z =(v r -v l ) / Δd

[0120] Where, v x represents the speed of the autonomous mobile device along the x-axis, ω z represents the angular velocity of the autonomous mobile device around the z-axis, v r represents the linear velocity of the right wheel of the autonomous mobile device, v l represents the linear velocity of the left wheel of the autonomous mobile device, and Δd represents the axle length between the left and right wheels of the autonomous mobile device.

[0121] After the state equation and observation equation of the system are established according to the above steps, S610 can be executed, that is, the update / correction step in the extended Kalman filter algorithm is used to obtain the initial pose estimation of the autonomous mobile device.

[0122] S620: Preprocess the acquired point cloud data.

[0123] Among them, preprocessing can include operations such as distortion removal, time alignment, point cloud downsampling and coordinate system conversion.

[0124] a. Distortion removal

[0125] During the process of collecting a frame of environmental information by the ranging sensor, the autonomous mobile device inevitably moves, causing the relative position between the autonomous mobile device and the spatial environment (such as obstacles) to change, which will increase the measurement error of the ranging sensor. To reduce the measurement error caused by the movement of the autonomous mobile device, the point cloud can be distorted before using it for calculation.

[0126] Specifically, based on the timestamp of each point in the point cloud frame and the pose of the autonomous mobile device at each time point provided by the pose fusion module, the pose of the autonomous mobile device at each point in the point cloud frame is calculated using a linear interpolation algorithm. For details, please refer to the following formula:

[0127] T k =T t *(1―scale)+T t+1 *scale

[0128] Where, T i represents the pose of the autonomous mobile device output by the pose fusion module at time i, and scale represents the proportion of time t to k in the period from t to t+1, that is,

[0129] b. Time alignment

[0130] Furthermore, point cloud frames at different times can be aligned to the same time. For details, please refer to the following formula:

[0131]

[0132] Where p i Indicates the i-th point in the current point cloud frame, T i represents the pose of the autonomous mobile device obtained by interpolation of the i-th point, represents the inverse of the pose of the autonomous mobile device obtained by interpolating the first point in the current point cloud frame, p′ i Indicates the coordinate value of the point after removing the distortion.

[0133] c. Downsampling

[0134] In order to reduce the amount of calculation and save computing costs, voxel filtering can be performed on the point cloud frame output by the ranging sensor to achieve the purpose of downsampling.

[0135] Preferably, in an embodiment of the present application, the voxel filtering method in the related art can be improved. Specifically, instead of calculating the centroid of all points in the current grid, a method of randomly selecting one point from all points as the representative value of the current grid is used for calculation.

[0136] At the same time, you can also set the minimum number of points in the downsampled point cloud frame to control the voxel size.

[0137] d. Coordinate system conversion

[0138] It should be understood that the research object in the embodiments of this application is an autonomous mobile device. To facilitate data processing, the point cloud in the sensor coordinate system can be converted to the vehicle coordinate system of the autonomous mobile device. Specifically, the coordinate system conversion matrix of the point cloud can be obtained by jointly calibrating the ranging sensor and the inertial measurement unit (IMU).

[0139] S630: Determine whether the autonomous mobile device is located on a single floor in the parking lot based on the slope of the ground where the autonomous mobile device is located.

[0140] Specifically, if the pitch angle of the autonomous mobile device's posture is small, it can be determined that the autonomous mobile device is in an intra-floor state, that is, the ground it is located on can be considered flat. In multi-story parking lots, the autonomous mobile device may also be in a cross-floor state (for example, when traveling from Basement 1 to Basement 2 through a transition area). In this case, the ground slope is relatively steep, and mapping and positioning can be performed using posture determination methods suitable for inter-floor transition areas.

[0141] S640: When it is determined that the autonomous mobile device is located on a single floor in the parking lot, a 2.5 DSLAM process is performed based on the current frame point cloud.

[0142] Specifically, the SLAM process in the current floor may include:

[0143] a. Extract valid information from the current frame point cloud;

[0144] Because the portion of the scene space environment that is higher than the average vehicle height does not change with the movement of other vehicles, and the ranging sensors equipped with the autonomous mobile device can obtain point cloud information in this portion, using this information (referred to as valid information in this embodiment) for SLAM significantly reduces map updates and thus reduces map maintenance costs.

[0145] Specifically, in order to extract the valid information in the current point cloud frame, the point cloud frame can be subjected to through-filtering according to the height value of each point in the current point cloud frame. For details, please refer to the following formula:

[0146]

[0147] Where pz represents the height value of each point in the current frame point cloud, σ min represents the preset minimum static height in the parking lot, σ max Indicates the preset maximum static height in a parking lot.

[0148] b. Perform scan-to-submap matching to estimate the current pose;

[0149] c. Update the map with valid information.

[0150] Compared with the technical solutions in the related art, the embodiment of the present application makes the following improvements to the front-end matching and loop detection of the SLAM process: when constructing the cost function, in addition to using occupancy information, the constraint of semantic information is also added. For details, please refer to the following formula:

[0151]

[0152] Where, T ξ represents the current position of the autonomous mobile device, h k Represents the kth point in the point cloud of the current frame. M smooth :R 2 →R, based on the coordinates of the converted point, use bilinear interpolation in the grid map to calculate the probability value corresponding to the current point. n , C′ n They represent the semantic information extracted from the current point cloud frame and the semantic information matching it in the map, It represents the 2-norm operation on the vector.

[0153] In order to facilitate the storage of semantic information, the embodiment of the present application expands the probability grid map structure in the related art, and adds a semantic list (for example Figure 5When updating the map with new measurements, the grid index can be calculated as follows, and the semantic information can be directly updated to the semantic list of the corresponding grid.

[0154]

[0155]

[0156] Where px and py represent the coordinate values ​​of the points in the point cloud in the map coordinate system, index x 、index y Represented as the index value of the grid corresponding to the coordinate value of the point in the point cloud, res x 、res y It represents the grid size along the X and Y directions, and floor() represents the floor function.

[0157] The update of the probability value in the grid can refer to the following formula:

[0158]

[0159] Where odds(s) represents the odds value when the state is s, odds(s|z) represents the odds value when the state is s under the condition that the observation is z, and p(z|s) represents the probability value of the observation z under the condition that the state is s.

[0160] The map construction method provided in the embodiment of the present application takes into account that the local environment in the parking lot is constantly changing and frequent map updates will increase maintenance costs. Therefore, when performing SLAM mapping, the collected point cloud is subjected to direct filtering processing, and points in the point cloud whose height values ​​are within a preset height range are retained. Based on these points, it is determined whether the map needs to be updated, thereby reducing invalid updates of the map and saving computing resources. At the same time, the general probability grid map is expanded, and semantic information such as lane lines, guide signs, parking lines, speed bumps, etc. in the scene is stored therein to realize the use of semantic information to construct constraint relationships. The dual constraint relationship is used in the front-end matching and loop detection links of SLAM to effectively avoid the mismatch problem caused by the existence of extremely similar scenes in the parking lot, thereby improving the accuracy of pose estimation. This will greatly reduce the mismatch caused.

[0161] Exemplary devices

[0162] Figure 7 FIG. 7 is a schematic diagram of a map construction device 700 provided in an embodiment of the present application. Figure 7 As shown, the map construction device 700 provided in the embodiment of the present application may include a collection module 710 , a data selection module 720 , a judgment module 730 and a calculation module 740 .

[0163] Among them, the acquisition module 710 can be used to collect environmental information in the parking lot through the ranging sensor provided on the autonomous mobile device to obtain the current point cloud frame; the data selection module 720 can be used to select the current local point cloud from the current point cloud frame, wherein the current local point cloud is composed of points in the current point cloud frame that are located above a preset static height in the parking lot; the judgment module 730 can be used to judge whether the posture of the autonomous mobile device has changed based on the current map and the current local point cloud, wherein the current map is a map of the parking lot constructed based on the 1st point cloud frame to the N-1th point cloud frame, and N is an integer greater than 1; the calculation module 740 can be used to use the current point cloud frame as the Nth point cloud frame when it is determined that the posture has changed, and update the current map based on the Nth point cloud frame.

[0164] Optionally, the judgment module 730 in the map construction device 700 may also be configured to: when it is determined that the position and posture have not changed, determine that the current map does not need to be updated.

[0165] In a preferred implementation, the preset static height may be equal to 0.5 to 0.9 times the height of the parking space. As another example, the preset static height may be set to any value greater than or equal to 1.6 m and less than or equal to 2.0 m.

[0166] Specifically, in some implementations, when updating the current map based on the Nth point cloud frame, the computing module 740 may perform the following steps: determining that the Nth point cloud frame includes point cloud information corresponding to a specific object, wherein the specific object is a static object of a preset type located in the parking lot; extracting the semantic information of the specific object based on the Nth point cloud frame as the Nth semantic information corresponding to the Nth point cloud frame; matching the Nth point cloud frame with the N-1th point cloud frame based on the Nth semantic information and the N-1th semantic information corresponding to the N-1th point cloud frame; and updating the current map based on the matching result.

[0167] Among them, static objects of preset types may include traffic safety signs, guidance signs, and auxiliary facilities.

[0168] Accordingly, in some implementations, the map construction device 700 may further include: a comparison module, configured to determine that the Nth point cloud frame includes point cloud information of a newly added specific object by comparing the Nth semantic information and the N-1th semantic information, wherein the newly added specific object is a specific object that does not appear in the N-1th point cloud frame; and a semantic setting module, configured to add the semantic information of the newly added specific object to the position corresponding to the newly added specific object in the current map.

[0169] In a preferred implementation, the current map may be a probabilistic grid map, wherein when adding semantic information of a newly added specific object to a position corresponding to the newly added specific object in the current map, the semantic setting module may perform the following steps: determining a specific grid corresponding to the newly added specific object in the current map; and adding the semantic information of the newly added specific object to the semantic list of the specific grid.

[0170] Furthermore, in some implementations, the map construction apparatus may further include: a loop detection module for performing loop detection based on the Nth semantic information and all semantic information in the current map; and an update module for updating the current map based on the loop detection result.

[0171] The map construction device provided in the embodiment of the present application can significantly reduce invalid map update operations in parking lot scenarios with frequent environmental changes by extracting local information within a preset height range from the environmental information collected by the ranging sensor and determining whether the map needs to be updated based on the local information. This can ensure high-precision positioning of autonomous mobile devices while significantly reducing the consumption of computing resources and saving computing costs.

[0172] It should be understood that the principles, functions, characteristics of the data used, data processing methods, and technical effects of all optional implementation methods of each module in the map construction device 700 provided in this embodiment can be referred to the corresponding contents in the exemplary method and will not be described in detail here.

[0173] Exemplary devices

[0174] Figure 8 FIG2 is a schematic diagram of an autonomous mobile device 800 provided in an embodiment of the present application. The autonomous mobile device 800 may be a vehicle with an autonomous driving function, or an electronic device such as a robot with an autonomous movement function.

[0175] like Figure 8 As shown, the autonomous mobile device may include at least one ranging sensor 810 and a processor 820. The at least one ranging sensor 810 may be used to collect spatial environmental information near the autonomous mobile device in the parking lot to obtain a point cloud frame. The processor 820 may use the environmental information collected by the ranging sensor to execute the map construction method provided in any of the above embodiments of the present application to achieve high-precision positioning of the autonomous mobile device.

[0176] Here, the at least one ranging sensor 810 may include multiple ranging sensors, and the types and acquisition ranges of the multiple ranging sensors may be different. For example, the multiple ranging sensors may include a laser radar (e.g., a mechanical rotating 3D laser radar, a solid-state / semi-solid-state 3D laser radar), a binocular vision sensor, or a structured light sensor (e.g., an RGB-D camera), as long as they can acquire 3D point cloud information of the environmental space within their acquisition range.

[0177] Figure 9 FIG. 1 is a schematic diagram of an exemplary electronic device 900 provided in an embodiment of the present application. Figure 9 As shown, the electronic device may include a processor 910 and a memory 920. The memory 920 stores computer instructions, and the processor 920 is configured to execute the computer instructions to implement the map construction method provided in any of the above embodiments.

[0178] Exemplary computer-readable storage media

[0179] Other embodiments of the present application further provide a computer-readable storage medium, including computer instructions stored thereon, which, when executed by a processor, enable the processor to implement the map construction method provided in any of the above embodiments.

[0180] Alternatively, the computer storage medium may be any tangible medium, such as a floppy disk, a CD-ROM, a DVD, a hard drive, or a network medium.

[0181] Exemplary computer program products

[0182] Other embodiments of the present application further provide a computer program product, which includes instructions. When the instructions are executed by a processor of a computer device, the computer device can execute the map construction method provided by any of the above embodiments of the present application.

[0183] The block diagrams of the devices, equipment, and systems involved in this application are only illustrative examples and are not intended to require or imply that they must be connected, arranged, and configured in the manner shown in the block diagrams. Those skilled in the art will appreciate that these devices, equipment, and systems can be connected, arranged, and configured in any manner. Words such as "comprise," "include," "have," and the like are open vocabularies, and can be used interchangeably with "including but not limited to," unless the context clearly indicates otherwise. The vocabulary "such as" used herein refers to the phrase "such as but not limited to," and can be used interchangeably with it.

[0184] It should also be noted that in the apparatus, device, and method of the present application, each module or each step can be decomposed and / or recombined, and such decomposition and / or recombination should be regarded as equivalent solutions of the present application.

[0185] The above description of the disclosed aspects is provided to enable any person skilled in the art to make or use the present application. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects without departing from the scope of the present application. Therefore, the present application is not intended to be limited to the above-described aspects, but rather to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0186] The above description is intended to illustrate and describe the technical solutions of the present application. Furthermore, this description is not intended to limit the embodiments of the present application to the scope of the above-disclosed forms. Although a number of exemplary aspects and embodiments have been discussed above, those skilled in the art can easily derive other variations, modifications, alterations, additions, and sub-combinations based on the above.

[0187] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent replacements, etc. made within the spirit and principles of the present application shall be included in the scope of protection of the present application.

Claims

1. A map construction method, characterized in that: include: The environmental information in the parking lot is collected by the ranging sensor installed on the autonomous mobile device to obtain the current point cloud frame; Selecting a current local point cloud from the current point cloud frame, wherein the current local point cloud is composed of points in the current point cloud frame that are located above a preset static height in the parking lot; Determining whether the position of the autonomous mobile device has changed based on a current map and the current local point cloud, wherein the current map is a map of the parking lot constructed based on point cloud frames 1 to N-1, where N is an integer greater than 1; When it is determined that the posture has changed, the current point cloud frame is used as the Nth point cloud frame, and the current map is updated according to the Nth point cloud frame.

2. The map construction method according to claim 1, characterized in that: Also includes: When it is determined that the position and posture have not changed, it is determined that the current map does not need to be updated.

3. The map construction method according to claim 1, characterized in that: The preset static height is equal to 0.5 to 0.9 times the height of the space in the parking lot.

4. The map construction method according to any one of claims 1 to 3, characterized in that: The updating of the current map according to the Nth point cloud frame includes: Determining that the Nth point cloud frame includes point cloud information corresponding to a specific object, wherein the specific object is a static object of a preset type located in the parking lot; Extracting the semantic information of the specific object according to the Nth point cloud frame as Nth semantic information corresponding to the Nth point cloud frame; Matching the Nth point cloud frame with the N-1th point cloud frame according to the Nth semantic information and the N-1th semantic information corresponding to the N-1th point cloud frame; The current map is updated according to the matching result.

5. The map construction method according to claim 4, characterized in that: Also includes: By comparing the Nth semantic information with the N-1th semantic information, determining that the Nth point cloud frame includes point cloud information of a newly added specific object, wherein the newly added specific object is a specific object that does not appear in the N-1th point cloud frame; The semantic information of the newly added specific object is added to the position corresponding to the newly added specific object in the current map.

6. The map construction method according to claim 5, characterized in that: The current map is a probabilistic grid map, wherein adding semantic information of the newly added specific object to a position corresponding to the newly added specific object in the current map includes: Determining a specific grid in the current map corresponding to the newly added specific object; The semantic information of the newly added specific object is added to the semantic list of the specific grid.

7. The map construction method according to claim 5, characterized in that: Also includes: Perform loop detection based on the Nth semantic information and all semantic information in the current map; The current map is updated according to the loop detection result.

8. The map construction method according to claim 4, characterized in that: The static objects of the preset types include traffic safety signs, guidance signs and auxiliary facilities.

9. A map construction device, characterized in that: include: An acquisition module is used to collect environmental information in the parking lot through a ranging sensor provided on an autonomous mobile device to obtain a current point cloud frame; a data selection module, configured to select a current local point cloud from the current point cloud frame, wherein the current local point cloud is composed of points in the current point cloud frame that are located above a preset static height in the parking lot; a determination module, configured to determine whether the position of the autonomous mobile device has changed based on a current map and the current local point cloud, wherein the current map is a map of the parking lot constructed based on point cloud frames 1 to N-1, where N is an integer greater than 1; a determination module, configured to store the current point cloud frame as an Nth point cloud frame when it is determined that the posture has changed; A calculation module is used to update the current map according to the Nth point cloud frame.

10. An autonomous mobile device, characterized in that include: Distance measurement sensor, used to collect environmental information in the parking lot; A processor, configured to execute the map construction method according to any one of claims 1 to 8.

11. An electronic device, characterized in that: include: Memory, for storing computer instructions; A processor, configured to execute the computer instructions to implement the map construction method according to any one of claims 1 to 8.

12. A computer-readable storage medium, characterized in that Computer instructions are stored, and when the computer instructions are executed by a processor, the map construction method according to any one of claims 1 to 8 is implemented.

Images