Point cloud registration method and device and intelligent mobile equipment

Abstract

The embodiment of the invention provides a point cloud registration method and device and intelligent mobile equipment, and relates to the field of artificial intelligence. The method comprises the following steps: acquiring a to-be-registered point cloud; determining an initial pose of the intelligent mobile device in a preset global coordinate system when the to-be-registered point cloud is generated; determining a target global pose of the to-be-registered point cloud in a global coordinate system based on the initial pose and distance values and normal vectors recorded by each grid of a preset grid map; wherein the distance value and the normal vector recorded by each grid of the grid map are obtained by updating the registration result of each frame of environment point cloud before the to-be-registered point cloud; and determining a registration result of the to-be-registered point cloud based on the target global pose. By applying the scheme provided by the embodiment of the invention, the global pose of each frame of environment point cloud detected by the intelligent mobile equipment in the global coordinate system can be determined in the process of constructing the scene map of the strange scene into which the intelligent mobile equipment enters.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a point cloud registration method, device and intelligent mobile device. Background Art

[0002] With the continuous development of artificial intelligence technology, for smart mobile devices such as sweeping robots and cruise robots that can automatically navigate and locate based on vision, they can build scene maps of the scenes they are in for continuous positioning, navigation, obstacle avoidance, etc.

[0003] Among them, the process of constructing the above-mentioned scene map by smart mobile devices usually involves environmental data collection, point cloud matching, map update and other links. When the smart mobile device detects the environmental point cloud of the scene where it is located, how to determine the global pose of the above-mentioned environmental point cloud in the global coordinate system, so as to use the global pose of the environmental point cloud and the environmental point cloud to realize the update of the map scene, plays a vital role in the construction of the above-mentioned scene map. Summary of the invention

[0004] The purpose of the embodiments of the present application is to provide a point cloud registration method, device and smart mobile device, so as to determine the global pose of each frame of environmental point cloud detected by the smart mobile device in the global coordinate system in the process of building a scene map of an unfamiliar scene entered by the smart mobile device. The specific technical solution is as follows:

[0005] In a first aspect, an embodiment of the present application provides a point cloud registration method, the method comprising:

[0006] Acquire a point cloud to be registered; wherein the point cloud to be registered is: a non-first frame environmental point cloud located in a preset local coordinate system and generated by performing environmental detection on the scene when the smart mobile device moves in the current scene;

[0007] Determining the initial position and posture of the intelligent mobile device in a preset global coordinate system when generating the point cloud to be registered;

[0008] Based on the initial pose, and the distance values ​​and normal vectors of each grid record of the preset grid map, determine the target global pose of the point cloud to be registered in the global coordinate system; wherein the distance values ​​and normal vectors of each grid record of the grid map are updated based on the registration results of each frame of the environment point cloud before the point cloud to be registered;

[0009] Based on the target global pose, a registration result of the point cloud to be registered is determined.

[0010] Optionally, in a specific implementation, the method further includes:

[0011] Based on the registration result of the point cloud to be registered, the distance value and the normal vector of each grid record of the grid map are updated.

[0012] Optionally, in a specific implementation, the method of updating the distance value and the normal vector of each grid record of the grid map based on the registration result of each frame of the environment point cloud includes:

[0013] Based on the registration result of each frame of the environment point cloud, the frame of the environment point cloud is converted into a global point cloud in the global coordinate system; wherein the registration result of the first frame of the environment point cloud is: the set position and posture in the global coordinate system;

[0014] Based on the global point cloud, determine the current distance value and the current normal vector of each target grid of the grid map; wherein the each target grid includes: the matching grids to which each point of the global point cloud belongs in the grid map, and the adjacent grids whose positional relationship with any matching grid meets the adjacent condition;

[0015] For each target grid, if the target grid records a distance value and a normal vector, the distance value and normal vector recorded in the target grid are updated respectively based on the determined current distance value and current normal vector of the target grid; otherwise, the determined distance value and normal vector of the target grid are recorded in the target grid.

[0016] Optionally, in a specific implementation, determining the target global pose of the point cloud to be registered in the global coordinate system based on the initial pose and the distance values ​​and normal vectors of each grid record of a preset grid map includes:

[0017] Determine each candidate angle based on an initial angle included in the initial posture and a preset angle search window, and determine each candidate position based on an initial position included in the initial posture and a preset position search window;

[0018] Determine each candidate pose based on each candidate angle and each candidate position; wherein each candidate pose includes: a candidate angle and a candidate position, and different candidate poses include different candidate angles and / or candidate positions;

[0019] Based on each candidate pose, the point cloud to be registered is converted into an initial global point cloud in the global coordinate system;

[0020] For each candidate pose, based on the initial global point cloud obtained by conversion, and the distance values ​​and normal vectors of each grid record of a preset grid map, a first type score of the candidate pose is calculated, and based on the initial global point cloud obtained by conversion, and the distance values ​​of each grid record of the grid map, a second type score of the candidate pose is calculated;

[0021] Obtaining candidate angles included in the candidate postures of the first category whose scores meet a first preset requirement as target angles, and obtaining candidate positions included in the candidate postures of the second category whose scores meet a second preset requirement as target positions;

[0022] A pose including the target angle and the target position is determined as the target global pose of the point cloud to be registered in the global coordinate system.

[0023] Optionally, in a specific implementation, the first type of score of the candidate pose is calculated based on the initial global point cloud obtained by conversion, and the distance values ​​and normal vectors of each grid record of the preset grid map, and the second type of score of the candidate pose is calculated based on the initial global point cloud obtained by conversion, and the distance values ​​of each grid record of the grid map, including:

[0024] For each point of the initial global point cloud obtained by the conversion, determine the current normal vector of the point and the normal vector of the grid record to which the point belongs in the preset grid map as the historical normal vector of the point, and determine the distance value of the grid record to which the point belongs in the grid map as the historical distance value of the point;

[0025] Filtering the maximum distance value among the distance values ​​of each grid record of the grid map;

[0026] Based on the current normal vector, historical normal vector and historical distance value of each point in the initial global point cloud obtained by transformation, and the maximum distance value, the first category score of the candidate pose is calculated, and based on the historical distance value and the maximum distance value of each point in the initial global point cloud obtained by transformation, the second category score of the candidate pose is calculated.

[0027] Optionally, in a specific implementation, the first type of score of the candidate pose is calculated based on the current normal vector, the historical normal vector and the historical distance value of each point in the initial global point cloud obtained by conversion, and the maximum distance value, and the second type of score of the candidate pose is calculated based on the historical distance value and the maximum distance value of each point in the initial global point cloud obtained by conversion, including:

[0028] Calculate the distance difference between the maximum distance value and the historical distance value of each point of the initial global point cloud obtained by conversion, and calculate the sum of all the obtained distance differences as the second type score of the candidate pose;

[0029] Calculate the normal vector difference between the current normal vector of each point of the initial global point cloud obtained by transformation and the historical normal vector of the point, and select the maximum normal vector difference among all the normal vector differences obtained;

[0030] The difference between the maximum normal vector difference and each normal vector difference is calculated as the target difference, and the sum of all the target differences and all the distance differences is calculated as the first category score of the candidate pose.

[0031] Optionally, in a specific implementation, determining the registration result of the point cloud to be registered based on the target global pose includes:

[0032] Based on the target global pose, using a preset objective function, calculating a pose increment of the target global pose;

[0033] The pose and value of the target global pose and the pose increment are calculated as the registration result of the point cloud to be registered.

[0034] Optionally, in a specific implementation, the calculating the pose increment of the target global pose based on the target global pose and using a preset objective function includes:

[0035] Based on the target global pose, converting the point cloud to be registered into the target global point cloud in the global coordinate system;

[0036] Based on the current normal vector of each point of the target global point cloud, the normal vector recorded in the grid corresponding to the point, the distance value recorded in the grid corresponding to the point, and the distance value recorded in the neighborhood grid of the grid corresponding to the point, the preset objective function is solved to obtain the posture increment of the target global posture; wherein the grid corresponding to each point of the target global point cloud is: the grid to which the point belongs in the grid map.

[0037] Optionally, in a specific implementation, the preset objective function is:

[0038]

[0039] Where, e1 = φ I (c i ), c i represents the i-th point of the target global point cloud, φ I (c i ) represents the result of bilinear interpolation of the distance value of the grid record corresponding to the i-th point and the distance value of the neighborhood grid record of the grid corresponding to the i-th point; is the current normal vector of the i-th point, Normal(c i ) is the normal vector of the grid record corresponding to the i-th point, and N is the total number of points in the target global point cloud;

[0040] The pose increment is:

[0041]

[0042] in,

[0043]

[0044] Among them, (p x ,p y ) is the point of the point cloud to be registered, For point (p x ,p y ), θ is the angle included in the target global pose, P 00 , P 01 , P 10 and P 11 are the points (p x ,p y ) in the four neighborhood grids of the grid map, Dis(P 10 )、Dis(P 00 )、Dis(P 11 ) and Dis(P 01 ) are the grids P 10 , P 00 , P 11 and P 01 The recorded distance value, (x, y) is the distance between the points (p x ,p y ) is the coordinate of the target in the global coordinate system after the global pose conversion, (x0, y0) is the grid p 00 The coordinates in the grid map, (x1, y1) are the coordinates of the grid p 11 The coordinates in the grid map.

[0045] Optionally, in a specific implementation, the method further includes:

[0046] In response to the intelligent mobile device passing through all areas within the scene, the current grid map recording distance values ​​and normal vectors is determined as the scene map of the scene.

[0047] In a second aspect, an embodiment of the present application provides a smart mobile device, including a detection sensor and a processor;

[0048] The detection sensor is used to detect the environment of the scene when the smart mobile device moves in the current scene, and generate each frame of the environment point cloud in a preset local coordinate system;

[0049] The processor is used to execute any point cloud registration method provided in the first aspect based on non-first frame environment point clouds in each frame environment point cloud.

[0050] In a third aspect, an embodiment of the present application provides a point cloud registration device, the device comprising:

[0051] The point cloud acquisition module is used to acquire the point cloud to be registered; wherein the point cloud to be registered is: a non-first frame environmental point cloud located in a preset local coordinate system and generated by performing environmental detection on the scene when the smart mobile device moves in the current scene;

[0052] A posture determination module, used to determine the initial posture of the intelligent mobile device in a preset global coordinate system when generating the point cloud to be registered;

[0053] A point cloud registration module, used to determine the target global pose of the point cloud to be registered in the global coordinate system based on the initial pose and the distance values ​​and normal vectors of each grid record of the preset grid map; wherein the distance values ​​and normal vectors of each grid record of the grid map are updated based on the registration results of each frame of environmental point cloud before the point cloud to be registered;

[0054] The result determination module is used to determine the registration result of the point cloud to be registered based on the target global pose.

[0055] Optionally, in a specific implementation, the device further includes:

[0056] The grid update module is used to update the distance value and normal vector of each grid record of the grid map based on the registration result of the point cloud to be registered.

[0057] Optionally, in a specific implementation, the grid updating module is specifically used to:

[0058] Based on the registration result of each frame of the environment point cloud, the frame of the environment point cloud is converted into a global point cloud in the global coordinate system; wherein the registration result of the first frame of the environment point cloud is: the set position and posture in the global coordinate system;

[0059] Based on the global point cloud, determine the current distance value and the current normal vector of each target grid of the grid map; wherein the each target grid includes: the matching grids to which each point of the global point cloud belongs in the grid map, and the adjacent grids whose positional relationship with any matching grid meets the adjacent condition;

[0060] For each target grid, if the target grid records a distance value and a normal vector, the distance value and normal vector recorded in the target grid are updated respectively based on the determined current distance value and current normal vector of the target grid; otherwise, the determined distance value and normal vector of the target grid are recorded in the target grid.

[0061] Optionally, in a specific implementation, the point cloud registration module includes:

[0062] A candidate information determination submodule, configured to determine each candidate angle based on an initial angle included in the initial posture and a preset angle search window, and to determine each candidate position based on an initial position included in the initial posture and a preset position search window;

[0063] A candidate posture determination submodule is used to determine each candidate posture based on each candidate angle and each candidate position; wherein each candidate posture includes: a candidate angle and a candidate position, and different candidate postures include different candidate angles and / or candidate positions;

[0064] A point cloud conversion submodule, used for converting the point cloud to be registered into an initial global point cloud in the global coordinate system based on each candidate pose;

[0065] A candidate pose scoring submodule, for each candidate pose, based on the initial global point cloud obtained by conversion, and the distance values ​​and normal vectors of each grid record of a preset grid map, calculates a first type of score for the candidate pose, and based on the initial global point cloud obtained by conversion, and the distance values ​​of each grid record of the grid map, calculates a second type of score for the candidate pose;

[0066] A candidate posture selection submodule is configured to obtain candidate angles included in the candidate postures of the first category whose scores meet the first preset requirement as target angles, and to obtain candidate positions included in the candidate postures of the second category whose scores meet the second preset requirement as target positions;

[0067] The global pose determination submodule is used to determine the pose including the target angle and the target position as the target global pose of the point cloud to be registered in the global coordinate system.

[0068] Optionally, in a specific implementation, the candidate pose scoring submodule includes:

[0069] An information determination unit, for determining, for each point of the initial global point cloud obtained by the conversion, a current normal vector of the point and a normal vector of a grid record to which the point belongs in a preset grid map as a historical normal vector of the point, and determining a distance value of a grid record to which the point belongs in the grid map as a historical distance value of the point;

[0070] A distance value screening unit, used for screening the maximum distance value among the distance values ​​recorded in each grid of the grid map;

[0071] The pose scoring unit is used to calculate the first type score of the candidate pose based on the current normal vector, historical normal vector and historical distance value of each point of the initial global point cloud obtained by transformation, and the maximum distance value, and calculate the second type score of the candidate pose based on the historical distance value and the maximum distance value of each point of the initial global point cloud obtained by transformation.

[0072] Optionally, in a specific implementation, the posture scoring unit is specifically used to:

[0073] Calculate the distance difference between the maximum distance value and the historical distance value of each point of the initial global point cloud obtained by conversion, and calculate the sum of all the obtained distance differences as the second type score of the candidate pose;

[0074] Calculate the normal vector difference between the current normal vector of each point of the initial global point cloud obtained by transformation and the historical normal vector of the point, and select the maximum normal vector difference among all the normal vector differences obtained;

[0075] The difference between the maximum normal vector difference and each normal vector difference is calculated as the target difference, and the sum of all the target differences and all the distance differences is calculated as the first category score of the candidate pose.

[0076] Optionally, in a specific implementation, the result determination module includes:

[0077] An increment determination submodule, used to calculate a posture increment of the target global posture based on the target global posture using a preset objective function;

[0078] The result determination submodule is used to calculate the pose and value of the target global pose and the pose increment as the registration result of the point cloud to be registered.

[0079] Optionally, in a specific implementation, the increment determination submodule is specifically used to:

[0080] Based on the target global pose, converting the point cloud to be registered into the target global point cloud in the global coordinate system;

[0081] Based on the current normal vector of each point of the target global point cloud, the normal vector recorded in the grid corresponding to the point, the distance value recorded in the grid corresponding to the point, and the distance value recorded in the neighborhood grid of the grid corresponding to the point, the preset objective function is solved to obtain the posture increment of the target global posture; wherein the grid corresponding to each point of the target global point cloud is: the grid to which the point belongs in the grid map.

[0082] Optionally, in a specific implementation, the preset objective function is:

[0083]

[0084] Where, e1 = φ I (c i ), c i represents the i-th point of the target global point cloud, φ I (c i ) represents the result of bilinear interpolation of the distance value of the grid record corresponding to the i-th point and the distance value of the neighborhood grid record of the grid corresponding to the i-th point; is the current normal vector of the i-th point, Normal(c i ) is the normal vector of the grid record corresponding to the i-th point, and N is the total number of points in the target global point cloud;

[0085] The pose increment is:

[0086]

[0087] in,

[0088]

[0089] Among them, (p x ,p y ) is the point of the point cloud to be registered, For point (p x ,p y ), θ is the angle included in the target global pose, P 00 , P 01 , P 10 and P 11 are the points (p x ,p y ) in the four neighborhood grids of the grid map, Dis(P 10 )、Dis(P 00 )、Dis(P 11 ) and Dis(P 01 ) are the grids P10 , P 00 , P 11 and P 01 The recorded distance value, (x, y) is the distance between the points (p x ,p y ) is the coordinate of the target in the global coordinate system after the global pose conversion, (x0, y0) is the grid p 00 The coordinates in the grid map, (x1, y1) are the coordinates of the grid p 11 The coordinates in the grid map.

[0090] Optionally, in a specific implementation, the device further includes:

[0091] The map determination module is used to determine the current grid map recording the distance value and the normal vector as the scene map of the scene in response to the intelligent mobile device passing through the entire area within the scene.

[0092] In a fourth aspect, an embodiment of the present application provides an electronic device, including:

[0093] Memory, used to store computer programs;

[0094] The processor is used to implement any point cloud registration method provided in the first aspect when executing the program stored in the memory.

[0095] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the point cloud registration method provided in any one of the first aspects is implemented.

[0096] In a sixth aspect, an embodiment of the present application provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute any of the point cloud registration methods provided in the first aspect above.

[0097] Beneficial effects of the embodiments of the present application:

[0098] As can be seen from the above, by applying the solution provided by the embodiment of the present application, in order to construct a scene map of an unfamiliar scene entered by a smart mobile device, the smart mobile device is usually equipped with various detection sensors, such as laser radar, image acquisition equipment, etc. After the smart mobile device enters the scene, the smart mobile device can move in the scene. During the movement, the scene can be detected based on the detection sensor carried by the smart mobile device, and each frame of the environment point cloud located in a preset local coordinate system can be generated, and the scene map of the scene can be constructed using the above-mentioned each frame of the environment point cloud. In the above-mentioned scene map construction process, for the generated non-first frame environment point cloud, it is necessary to perform point cloud registration on the environment point cloud. In this way, in the above-mentioned map construction process, the non-first frame environment point cloud located in the preset local coordinate system generated by the environment detection of the scene during the movement of the smart mobile device in the current scene can be first obtained as the point cloud to be registered. Afterwards, the initial position of the smart mobile device in the preset global coordinate system when the registration point cloud is generated can be determined. At this time, for the preset grid map, the distance values ​​and normal vectors recorded in each grid of the grid map are updated based on the registration results of the environmental point clouds of each frame before the point cloud to be registered. Therefore, based on the above initial pose, and the distance values ​​and normal vectors recorded in each grid of the grid map, the target global pose of the point cloud to be registered in the above global coordinate system can be determined, and based on the above target global pose, the registration result of the point cloud to be registered can be determined. At this point, the point cloud registration of the point cloud to be registered is completed, and the global pose of the point cloud to be registered in the above global coordinate system is obtained. Then, the point cloud to be registered and the global pose of the point cloud to be registered in the above global coordinate system can be used to update the grid map. The above scene map is constructed by looping in sequence.

[0099] Based on this, by applying the solution provided in the embodiment of the present application, the grid map in which each grid records the distance value and the normal vector can be called a grid map based on the distance field and the normal vector field, so that the point cloud to be registered can be registered with the grid map based on the distance field and the normal vector field, and the global pose of the point cloud to be registered can be calculated, and the global pose can be understood as the pose of the smart mobile device. In this way, in the process of constructing a scene map of an unfamiliar scene entered by a smart mobile device, it is possible to determine the global pose of each frame of the environmental point cloud detected by the smart mobile device in the global coordinate system, and then, to update the map scene using each frame of the environmental point cloud and the global pose of each frame of the environmental point cloud.

[0100] Furthermore, the existing point cloud registration algorithm based on the discovery of the normal vector solves the pose by matching two frames of point clouds, while in the solution provided in the embodiment of the present application, the point cloud is matched with a grid map to solve the pose, wherein the normal vector constraint is considered in the point cloud registration process based on the grid map, and the consistency of the normal vector is taken into account, and then, the erroneous matches of inconsistent normal vectors are filtered out, thereby improving the robustness and accuracy of the point cloud registration.

[0101] Of course, implementing any product or method of the present application does not necessarily require achieving all of the advantages described above at the same time. BRIEF DESCRIPTION OF THE DRAWINGS

[0102] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application, and for ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0103] Figure 1 A schematic diagram of a process of a point cloud registration method provided in an embodiment of the present application;

[0104] Figure 2 A schematic diagram of a process flow of another point cloud registration method provided in an embodiment of the present application;

[0105] Figure 3 A flowchart of a method for updating the distance value and normal vector of each grid record of a grid map based on the registration result of each frame of the environment point cloud provided in an embodiment of the present application;

[0106] Figure 4 A schematic diagram of a grid map provided in an embodiment of the present application;

[0107] Figure 5 A schematic diagram of a flow chart of another point cloud registration method provided in an embodiment of the present application;

[0108] Figure 6 A schematic diagram of a scene map recording distance values ​​and normal vectors provided in an embodiment of the present application;

[0109] Figure 7 A flowchart of a specific implementation method of S103 provided in an embodiment of the present application;

[0110] Figure 8 A schematic diagram of the relationship between an initial angle and an angle search window provided in an embodiment of the present application;

[0111] Fig. 9 A schematic diagram of the relationship between an initial position and a position search window provided in an embodiment of the present application;

[0112] Fig.10 A schematic diagram of a geometric relationship satisfied by an angle search step, a grid map resolution, and a specified maximum distance provided in an embodiment of the present application when n=1 in formula (4);

[0113] Fig.11 A schematic diagram of a flow chart of a method for acquiring an environmental point cloud provided in an embodiment of the present application;

[0114] Fig.12 A schematic diagram of edge features of a ceiling area of ​​an image extracted according to an embodiment of the present application;

[0115] Fig.13 A schematic diagram of the capture angle of view of an image capture device carried by a smart mobile device provided in an embodiment of the present application;

[0116] Fig.14 A schematic diagram of the structure of a smart mobile device provided in an embodiment of the present application;

[0117] Fig.15 A schematic diagram of the structure of a point cloud registration device provided in an embodiment of the present application;

[0118] Fig.16 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application;

[0119] Fig.17 A schematic diagram of the relationship between points in a global point cloud and surrounding integer coordinate points after a point cloud to be registered is converted into a global point cloud in a global coordinate system according to an embodiment of the present application. DETAILED DESCRIPTION

[0120] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field based on the present application belong to the scope of protection of the present application.

[0121] Currently, in the process of constructing a scene map of an unfamiliar scene entered by a smart mobile device, how to determine the global pose of each frame of environmental point cloud detected by the smart mobile device in the global coordinate system, so as to use each frame of environmental point cloud and the global pose of each frame of environmental point cloud to update the map scene, plays a vital role in the construction of the above-mentioned scene map.

[0122] In order to solve the above technical problems, an embodiment of the present application provides a point cloud registration method.

[0123] Among them, this method can be applied to various application scenarios for constructing scene maps of scenes where smart mobile devices are located, for example, a cleaning robot in a home scene constructs a map of the home; an inspection robot in a factory scene constructs a map of the factory building, etc.

[0124] Furthermore, in the embodiments of the present application, the intelligent mobile device may be equipped with various detection sensors capable of generating environmental point clouds through environmental detection, such as various image acquisition devices including laser radar and monocular cameras, etc. Furthermore, considering the low cost of monocular cameras, when the detection sensor carried by the intelligent mobile device is a monocular camera, the production cost of the intelligent mobile device can be further reduced, thereby being more conducive to the application of the solution provided by the embodiments of the present application in low-cost intelligent mobile devices.

[0125] Among them, the method can be applied to the intelligent mobile device itself equipped with a detection sensor, and can also be applied to the detection sensor equipped on the intelligent mobile device and having a data processing function, and can also be applied to various electronic devices that can communicate with the intelligent mobile device equipped with a detection sensor and provide relocation services for the intelligent mobile device. For example, after the intelligent mobile device generates each frame of environmental point cloud using the detection sensor carried by itself, it uses the data processing module carried by itself to execute the method; for another example, after the detection sensor carried by the intelligent mobile device generates each frame of environmental point cloud, it can use the data processing module carried by the above detection sensor to execute the method; for example, the intelligent mobile device is connected to the management platform in communication, and the management platform can obtain each frame of environmental data generated by the detection sensor carried by the intelligent mobile device, and use the above frame of environmental data obtained to execute the method; in addition, when the execution subject of the method is a management platform composed of electronic devices, the management platform can be an independent electronic device, or it can be a device cluster composed of multiple electronic devices, hereinafter referred to as electronic device.

[0126] Based on this, the embodiments of the present application do not specifically limit the application scenarios and execution entities of the method, nor do they specifically limit the types of detection sensors carried by the smart mobile device.

[0127] A point cloud registration method provided in an embodiment of the present application may include the following steps:

[0128] Acquire a point cloud to be registered; wherein the point cloud to be registered is: a non-first frame environmental point cloud located in a preset local coordinate system and generated by performing environmental detection on the scene when the smart mobile device moves in the current scene;

[0129] Determining the initial position and posture of the intelligent mobile device in a preset global coordinate system when generating the point cloud to be registered;

[0130] Based on the initial pose, and the distance values ​​and normal vectors of each grid record of the preset grid map, determine the target global pose of the point cloud to be registered in the global coordinate system; wherein the distance values ​​and normal vectors of each grid record of the grid map are updated based on the registration results of each frame of the environment point cloud before the point cloud to be registered;

[0131] Based on the target global pose, a registration result of the point cloud to be registered is determined.

[0132] As can be seen from the above, by applying the solution provided by the embodiment of the present application, in order to construct a scene map of an unfamiliar scene entered by a smart mobile device, the smart mobile device is usually equipped with various detection sensors, such as laser radar, image acquisition equipment, etc. After the smart mobile device enters the scene, the smart mobile device can move in the scene. During the movement, the scene can be detected based on the detection sensor carried by the smart mobile device, and each frame of the environment point cloud located in a preset local coordinate system can be generated, and the scene map of the scene can be constructed using the above-mentioned each frame of the environment point cloud. In the above-mentioned scene map construction process, for the generated non-first frame environment point cloud, it is necessary to perform point cloud registration on the environment point cloud. In this way, in the above-mentioned map construction process, the non-first frame environment point cloud located in the preset local coordinate system generated by the environment detection of the scene during the movement of the smart mobile device in the current scene can be first obtained as the point cloud to be registered. Afterwards, the initial position of the smart mobile device in the preset global coordinate system when the registration point cloud is generated can be determined. At this time, for the preset grid map, the distance values ​​and normal vectors recorded in each grid of the grid map are updated based on the registration results of the environmental point clouds of each frame before the point cloud to be registered. Therefore, based on the above initial pose, and the distance values ​​and normal vectors recorded in each grid of the grid map, the target global pose of the point cloud to be registered in the above global coordinate system can be determined, and based on the above target global pose, the registration result of the point cloud to be registered can be determined. At this point, the point cloud registration of the point cloud to be registered is completed, and the global pose of the point cloud to be registered in the above global coordinate system is obtained. Then, the point cloud to be registered and the global pose of the point cloud to be registered in the above global coordinate system can be used to update the grid map. The above scene map is constructed by looping in sequence.

[0133] Based on this, by applying the solution provided in the embodiment of the present application, the grid map in which each grid records the distance value and the normal vector can be called a grid map based on the distance field and the normal vector field, so that the point cloud to be registered can be registered with the grid map based on the distance field and the normal vector field, and the global pose of the point cloud to be registered can be calculated, and the global pose can be understood as the pose of the smart mobile device. In this way, in the process of constructing a scene map of an unfamiliar scene entered by a smart mobile device, it is possible to determine the global pose of each frame of the environmental point cloud detected by the smart mobile device in the global coordinate system, and then, to update the map scene using each frame of the environmental point cloud and the global pose of each frame of the environmental point cloud.

[0134] Furthermore, the existing point cloud registration algorithm based on the discovery of the normal vector solves the pose by matching two frames of point clouds, while in the solution provided in the embodiment of the present application, the point cloud is matched with a grid map to solve the pose, wherein the normal vector constraint is considered in the point cloud registration process based on the grid map, and the consistency of the normal vector is taken into account, and then, the erroneous matches of inconsistent normal vectors are filtered out, thereby improving the robustness and accuracy of the point cloud registration.

[0135] Below, in conjunction with the accompanying drawings, a point cloud registration method provided in an embodiment of the present application is described in detail.

[0136] Figure 1 A schematic diagram of a point cloud registration method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the method may include the following steps S101-S104.

[0137] S101: Obtain the point cloud to be registered.

[0138] The point cloud to be registered is: a non-first frame environmental point cloud located in a preset local coordinate system, which is generated by performing environmental detection on the scene when the smart mobile device moves in the current scene.

[0139] When constructing a scene map, the smart mobile device may first be placed in the scene by moving the smart mobile device into the scene, or by controlling the smart mobile device to move into the scene.

[0140] Among them, the embodiment of the present application does not specifically limit the application scenario and execution subject of the method, so the above-mentioned scenario can be a family scenario, such as a family residence, or a factory scenario, such as a factory building, etc., and of course, it can also be other types of scenarios. Furthermore, the smart mobile device can be equipped with various detection sensors to achieve environmental exploration of the scene and generate environmental point clouds for each frame. For example, since the embodiment of the present application also does not specifically limit the type of detection sensor carried by the smart mobile device, the detection sensor carried by the smart mobile device can be a laser radar, an image acquisition device, etc.

[0141] After placing the smart mobile device in the scene, the smart mobile device can be controlled to move in the scene. During the above movement, the detection sensor carried by the smart mobile device can detect the environment of the scene, thereby generating each frame of environmental point cloud located in a preset local coordinate system according to the result of the environmental detection, and each frame of environmental point cloud is used for the subsequent construction of a scene map of the scene.

[0142] In this way, when constructing a scene map for the current scene of the smart mobile device, the environment point clouds of each frame generated by environmental detection of the scene and located in a preset local coordinate system can be obtained first while the smart mobile device is moving in the current scene.

[0143] In the solution provided in the embodiment of the present application, the global pose of the point cloud is calculated by aligning the point cloud with the grid map based on the distance field and the normal vector field. For the first frame of the environmental point cloud generated above, at this time, no distance value and normal vector are recorded in each grid of the grid map. Therefore, each frame point cloud except the first frame point cloud in the above-mentioned environmental point cloud can be used as the point cloud to be aligned, and used for alignment with the grid map based on the distance field and the normal vector field.

[0144] Based on this, in the solution provided in the embodiment of the present application, when performing point cloud registration, the point cloud to be registered is first obtained, and the point cloud to be registered is: a non-first frame environmental point cloud located in a preset local coordinate system, which is generated by environmental detection of the scene while the smart mobile device is moving in the current scene.

[0145] As for the first-frame point cloud mentioned above, a set pose in a preset global coordinate system can be directly pre-set for the first-frame point cloud as the registration result of the first-frame point cloud.

[0146] Among them, starting from the second frame of environmental point cloud, each time a frame of environmental point cloud is generated, the frame of environmental point cloud can be immediately obtained as the point cloud to be registered, and the subsequent point cloud registration process can be immediately performed on the frame of environmental point cloud. Alternatively, when the smart mobile device passes through all areas within the scene and completes the environmental exploration of the scene, all environmental point clouds generated in the above exploration process can be obtained at one time, and according to the generation order of each frame of environmental point cloud, starting from the second frame of environmental point cloud, the subsequent point cloud registration process is performed on each frame of environmental point cloud in sequence.

[0147] That is to say, in the embodiment of the present application, the registration result of the point cloud to be registered in the previous frame is determined according to the generation order of the environmental point cloud of each frame, and the distance values ​​and normal vectors of each grid record of the above-mentioned grid map are updated based on the registration result of the point cloud to be registered in the previous frame, and then the point cloud to be registered in the next frame is registered.

[0148] That is, first, determine that the registration result of the first frame point cloud is the set posture in the above-mentioned pre-set global coordinate system, and use the set posture to update the distance value and normal vector of each grid record of the above-mentioned grid map; then, determine the registration result of the second frame environment point cloud, and use the registration result of the second frame environment point cloud to update the distance value and normal vector of each grid record of the above-mentioned grid map again; then, determine the registration result of the third frame environment point cloud, and use the registration result of the third frame environment point cloud to update the distance value and normal vector of each grid record of the above-mentioned grid map again; loop in sequence, determine the registration result of each frame of the point cloud to be registered, and complete the update of the distance value and normal vector of each grid record of the above-mentioned grid map using the configuration result of the frame point cloud to be registered, thereby completing the gradual update of the above-mentioned grid map until the scene map of the above-mentioned scene is constructed.

[0149] In the embodiment of the present application, in the above-mentioned grid map, there may be grids that record both distance values ​​and normal vectors, and there may also be grids that record neither distance values ​​nor normal vectors. There are no grids that only record distance values ​​but not normal vectors, and there are no grids that do not record distance values ​​but only record normal vectors.

[0150] In addition, in the solution provided in the embodiment of the present application, the environment detection of the scene currently located by the smart mobile device is realized by the detection sensor carried by the smart mobile device, and the above-mentioned environmental point clouds of each frame obtained are: the environmental point clouds of each frame generated by the environmental detection of the scene by the above-mentioned detection sensor and located in the preset local coordinate system. Among them, for different detection sensors carried by the smart mobile device, when executing the solution provided in the embodiment of the present application, the pre-set local coordinates can be different.

[0151] For example, if the detection sensor carried by the smart mobile device is an image acquisition device such as a monocular camera, the camera coordinate system of the above image acquisition device can be used as the preset local coordinate system; if the detection sensor carried by the smart mobile device is a lidar, the radar coordinate system of the lidar can be used as the preset local coordinate system.

[0152] Among them, when the smart mobile device moves in the current scene, the detection sensor it carries completes the environmental detection of the scene once and generates a frame of environmental point cloud.

[0153] For example, when the detection sensor carried by the smart mobile device is an image acquisition device such as a monocular camera, the image acquisition device realizes environmental detection of the scene by acquiring images of the scene. Each time the image acquisition device completes environmental detection, that is, acquires a frame of image of the scene, a frame of environmental point cloud can be generated based on the acquired frame of image, and since the preset local coordinate system is the camera coordinate system of the above-mentioned image acquisition device, the above-mentioned generated frame of environmental point cloud is located in the camera coordinate system of the above-mentioned image acquisition device. In other words, each frame of environmental point cloud is generated based on a frame of image of the scene acquired by the above-mentioned image acquisition device, and is a point cloud located in the camera coordinate system of the above-mentioned image acquisition device.

[0154] In the embodiment of the present application, the smart mobile device can be equipped with a monocular camera as a detection sensor. In the related technologies for the smart mobile device to construct a scene map of the current scene, the smart mobile device usually uses high-end detection sensors such as TOF cameras, laser radars, depth cameras, etc. as detection sensors. However, since the embodiment of the present application ultimately realizes the registration of the environmental point clouds to be registered for each frame of the non-first frame environmental point cloud, the solution provided by the embodiment of the present application can be implemented using a monocular camera without reducing the accuracy of the point cloud registration. In this way, the use of a monocular camera can significantly reduce the performance requirements of the detection sensor carried by the smart mobile device, reduce the production cost of the smart mobile device, and make the solution provided by the embodiment of the present application more conducive to application in low-cost smart mobile devices.

[0155] For another example, when the detection sensor carried by the smart mobile device is a laser radar, the laser radar realizes the environmental detection of the above scene by transmitting radar signals and receiving echo signals of the radar signals, and the laser radar can directly generate and output the environmental contour point cloud of the above scene as the environmental point cloud through the above environmental detection. And because the preset local coordinate system is the radar coordinate system of the above laser radar, the environmental contour point cloud generated and output by the above laser radar as the environmental point cloud is located in the radar coordinate system of the laser radar.

[0156] S102: Determine the initial position and posture of the intelligent mobile device in a preset global coordinate system when generating the point cloud to be registered.

[0157] After obtaining the above-mentioned point cloud to be registered, the initial posture of the intelligent mobile device in the preset global coordinate system when generating the point cloud to be registered can be determined. The above-mentioned initial posture is the posture determined by the positioning sensor carried by the intelligent mobile device at the same time when the point cloud to be registered is generated.

[0158] For example, the above-mentioned intelligent mobile device is equipped with a laser odometer. For the point cloud to be registered, at the same time when the point cloud to be registered is generated, the laser odometer simultaneously determines a posture, and the posture can be used as the above-mentioned initial posture.

[0159] S103: Based on the initial pose, and the distance values ​​and normal vectors of each grid record of the preset grid map, determine the target global pose of the point cloud to be registered in the global coordinate system.

[0160] The distance value and normal vector of each grid record of the grid map are updated based on the registration results of each frame of the environment point cloud before the point cloud to be registered.

[0161] After obtaining the above-mentioned point cloud to be registered and determining the above-mentioned initial pose, the target global pose of the above-mentioned point cloud to be registered in the above-mentioned global coordinate system can be determined based on the initial pose and the distance values ​​and normal vectors recorded in each grid of the preset grid map.

[0162] In order to construct a scene map of the scene currently located by the smart mobile device, a grid map can be pre-set, and the map coordinate system of the grid map is the same coordinate system as the global coordinate system. For example, when the global coordinate system is the world coordinate system, the map coordinate system of the grid map is also the world coordinate system. At this time, the grid map can also be called a global grid map.

[0163] As mentioned above, according to the generation order of each frame environment point cloud, for each frame environment point cloud, after obtaining the registration result of the frame environment point cloud, the distance values ​​and normal vectors recorded in the above-mentioned grid map are updated based on the registration result of the frame environment point cloud. Therefore, for the above-mentioned non-first frame environment point cloud as the point cloud to be registered, when determining to obtain the above-mentioned initial pose, there are grids recording distance values ​​and normal vectors in the above-mentioned grid map. Therefore, based on the initial pose, and the distance values ​​and normal vectors recorded in each grid of the preset grid map, the target global pose of the point cloud to be registered in the global coordinate system can be determined.

[0164] For the sake of clarity, the specific implementation of the above step S103 and the method of updating the distance value and normal vector of each grid record of the grid map based on the registration result of each frame of the environment point cloud will be described below by examples.

[0165] S104: Determine the registration result of the point cloud to be registered based on the target global pose.

[0166] For the point cloud to be registered obtained as above, after obtaining the target global pose of the point cloud to be registered in the global coordinate system, the registration result of the point cloud to be registered can be determined based on the target global pose.

[0167] For example, the target global pose of the point cloud to be registered in the global coordinate system can be directly determined as the registration result of the point cloud to be registered.

[0168] Optionally, in a specific implementation, the above step S104 may include the following steps 1041-1042.

[0169] Step 1041: Based on the target global pose, using a preset objective function, calculate the pose increment of the target global pose.

[0170] Step 1042: Calculate the pose and value of the target global pose and the pose increment as the registration result of the point cloud to be registered.

[0171] In this specific implementation, after obtaining the target global pose of the point cloud to be registered in the global coordinate system, the point cloud to be registered and the above-mentioned grid map can be used to implement precise matching based on nonlinear optimization to perform error correction on the above-mentioned target global pose, thereby improving the accuracy of the registration result of the final point cloud to be matched.

[0172] Based on this, the pose increment of the target global position can be calculated based on the target global pose and using the preset objective function. Furthermore, the pose sum of the target global pose and the pose increment can be calculated, and the pose sum can be used as the registration result of the point cloud to be registered.

[0173] Optionally, in a specific implementation, the above step 1041 may include the following steps 1-2.

[0174] Step 1: Based on the target global pose, convert the point cloud to be registered into the target global point cloud in the global coordinate system.

[0175] Step 2: Based on the current normal vector of each point in the target global point cloud, the normal vector of the grid record corresponding to the point, the distance value of the grid record corresponding to the point, and the distance value of the neighborhood grid record of the grid corresponding to the point, solve the preset objective function to obtain the pose increment of the target global pose.

[0176] Among them, the grid corresponding to each point of the target global point cloud is: the grid to which the point belongs in the grid map.

[0177] In this specific implementation, after obtaining the target global pose of the point cloud to be registered in the global coordinate system, the point cloud to be registered can be first converted into the target global point cloud in the global coordinate system based on the target global pose. After that, the grid to which each point of the target global point cloud belongs in the above-mentioned grid map can be determined, and the neighborhood grid of the grid to which each point belongs can be determined, wherein the grid to which each point belongs is called the grid corresponding to the point. Then, the current normal vector of each point of the above-mentioned target global point cloud, the normal vector of the grid record corresponding to the point, the distance value of the grid record corresponding to the point, and the distance value of the neighborhood grid record of the grid corresponding to the point can be determined. In this way, the preset objective function can be solved based on the current normal vector of each point of the target global point cloud, the normal vector of the grid record corresponding to the point, the distance value of the grid record corresponding to the point, and the distance value of the neighborhood grid record of the grid corresponding to the point, and the pose increment of the target global pose can be obtained.

[0178] Optionally, in a specific implementation, the preset objective function is:

[0179]

[0180] Where, e1 = φ I (c i ), c i represents the i-th point of the target global point cloud, φ I (c i ) represents the result of bilinear interpolation of the distance value of the grid record corresponding to the i-th point and the distance value of the neighborhood grid record of the grid corresponding to the i-th point; is the current normal vector of the i-th point, Normal(c i ) is the normal vector of the grid record corresponding to the i-th point, and N is the total number of points in the target global point cloud;

[0181] Accordingly, in this specific implementation, the Gauss-Newton method can be used to iteratively solve the above preset objective function to obtain the posture increment, wherein the posture increment obtained by solving is:

[0182]

[0183] in,

[0184]

[0185]

[0186] Among them, (p x ,p y ) is the point of the point cloud to be registered, For point (p x ,p y ), θ is the angle included in the target global pose, P 00 , P 01 , P 10 and P 11 They are points (p x ,p y ) in the four neighborhood grids in the grid map, Dis(P 10 )、Dis(P 00 )、Dis(P 11 ) and Dis(P 01 ) are grids P 10 , P 00 , P 11 and P 01 The recorded distance value, (x, y) is the distance between the points (p x ,p y ) is the coordinate in the global coordinate system after the target global pose transformation, (x0, y0) is the coordinate of the grid P 00 The coordinates in the grid map, (x1, y1) are the coordinates of the grid P 11 The coordinates of the coordinates in the raster map.

[0187] As mentioned above, the point cloud to be registered is a point cloud in a preset local coordinate system, then (p x ,p y ) is the point cloud in the above local coordinate system; when the point cloud to be registered is converted into the global point cloud in the above global coordinate system by using the target initial pose of the point cloud to be registered in the preset global coordinate system, the point (p x ,p y ) corresponds to a point in the above global point cloud, and point (p x ,p y ) in the global point cloud above corresponds to a point with coordinates (x, y), and point (p x ,p y ) The corresponding point in the above global point cloud is the point in the above global coordinate system, and (x, y) is also the coordinate in the above global coordinate system.

[0188] Among them, Fig.17 As shown, point (p x ,p y) in the above global point cloud can be represented by p(x,y). Considering that (x,y) may be an integer coordinate or a non-integer coordinate, we can select four points with integer coordinates around the point p(x,y) and the closest distance to the point p(x,y) in the above global coordinate system, that is, select Fig.17 Point p in 00 (x0,y0), point p 01 (x0,y1), point p 10 (x1,y0) and point p 11 (x1,y1), where (x0,y0), (x0,y1), (x1,y0) and (x1,y1) are the points p respectively. 00 (x0,y0), point p 01 (x0,y1), point p 10 (x1,y0) and point p 11 (x1, y1) is the coordinate of the point p in the above global coordinate system. 00 (x0,y0), point p 01 (x0,y1), point p 10 (x1,y0) and point p 11 The grid P to which (x1, y1) belongs respectively 00 , P 01 , P 10 and P 11 , so that the above grid P 00 , P 01 , P 10 and P 11 As a point (p x ,p y ) in the four neighborhood grids in the grid map. Since the map coordinate system of the grid map is the same as the global coordinate system, the p in the global coordinate system 00 (x0,y0), point p 01 (x1,y1), point p 11 (x1,y0) and point p 11 The coordinates of (x1, y1) in the global coordinate system are the same as p 00 (x0,y0), point p 01 (x0,y1), point p 10 (x1,y1) and point p 11 The grid P to which (x1, y1) belongs respectively 11 , P 11 , P 10 and P 11 The coordinates in the map coordinate system of the above-mentioned raster map are the same.

[0189] In addition, in the use of subsequent Figure 7 In the specific implementation method shown, when determining the target global pose of the point cloud to be registered in the global coordinate system, the above-mentioned target global pose is the candidate pose including the target angle and the target position, and θ in the above formula is the target angle, that is, the candidate angle included in the candidate poses whose first category scores meet the first preset requirements.

[0190] And, the above matrix J can be called a Jacobi matrix.

[0191] In this specific implementation, the above-mentioned preset objective function not only considers the minimization of the distance value, but also considers the minimization of the normal vector difference, thereby reducing the dependence of the pose accuracy on the grid map resolution, and can obtain a higher-precision point cloud registration result on a lower-resolution grid map, thereby improving the point cloud registration accuracy.

[0192] Based on this, by applying the solution provided in the embodiment of the present application, the grid map in which each grid records the distance value and the normal vector can be called a grid map based on the distance field and the normal vector field, so that the point cloud to be registered can be registered with the grid map based on the distance field and the normal vector field, and the global pose of the point cloud to be registered can be calculated, and the global pose can be understood as the pose of the smart mobile device. In this way, in the process of constructing a scene map of an unfamiliar scene entered by a smart mobile device, it is possible to determine the global pose of each frame of the environmental point cloud detected by the smart mobile device in the global coordinate system, and then, to update the map scene using each frame of the environmental point cloud and the global pose of each frame of the environmental point cloud.

[0193] Furthermore, the existing point cloud registration algorithm based on the discovery of the normal vector solves the pose by matching two frames of point clouds, while in the solution provided in the embodiment of the present application, the point cloud is matched with a grid map to solve the pose, wherein the normal vector constraint is considered in the point cloud registration process based on the grid map, and the consistency of the normal vector is taken into account, and then, the erroneous matches of inconsistent normal vectors are filtered out, thereby improving the robustness and accuracy of the point cloud registration.

[0194] In addition, as mentioned above, according to the generation order of each frame environment point cloud, for each frame environment point cloud, after obtaining the registration result of the frame environment point cloud, the distance value and normal vector of each grid record of the above-mentioned grid map can be updated based on the registration result of the frame environment point cloud.

[0195] Based on this, optionally, in a specific implementation, such as Figure 2 As shown, a point cloud registration method provided in an embodiment of the present application may further include the following step S105.

[0196] S105: Based on the registration result of the point cloud to be registered, the distance value and the normal vector of each grid record of the grid map are updated.

[0197] In this specific implementation, for the above-mentioned point cloud to be registered, after obtaining the registration result of the point cloud to be registered, the distance value and normal vector of each grid record of the grid map can be updated based on the registration result of the point cloud to be registered. Thus, the updated grid map can be used for the point cloud registration process of the next point cloud to be registered of the above-mentioned point cloud to be registered.

[0198] Optionally, in a specific implementation, such as Figure 3 As shown, an embodiment of the present application provides a method for updating the distance value and normal vector of each grid record of a grid map based on the registration result of each frame of the environment point cloud, which may include the following steps S301-S303.

[0199] S301: Based on the registration result of each frame of the environment point cloud, convert the frame of the environment point cloud into a global point cloud in the global coordinate system.

[0200] Among them, the registration result of the first frame environment point cloud is: the set pose in the global coordinate system.

[0201] In this specific implementation, for each frame of environment point cloud, after obtaining the registration result of the frame of environment point cloud, the frame of environment point cloud can first be converted into a global point cloud in a global coordinate system based on the registration result of each frame of environment point cloud.

[0202] Among them, the registration result of each frame of the environment point cloud is: the global pose of the frame of the environment point cloud in the above global coordinate system.

[0203] Generally, a pose may include: position and attitude, and the position corresponds to a position point in the coordinate system to which the pose belongs, and the attitude may be represented by an angle, which is the angle between a specified direction of an object having the pose and a specified coordinate axis of the coordinate system to which the pose belongs. In this specific implementation, the pose of an environment point cloud may be understood as the pose of an intelligent mobile device when generating an environment point cloud.

[0204] Among them, considering that when constructing a scene map of the scene where the smart mobile device is currently located, the smart device is usually located on the ground within the scene, the horizontal plane of the above-mentioned global coordinate system and the ground can be regarded as the same plane. Therefore, the smart mobile device is located in the horizontal plane of the global coordinate system, and the height coordinate in the vertical direction is 0.

[0205] Based on this, in this specific implementation, the position included in the target global pose corresponding to each frame environment point cloud in the global coordinate system can usually be represented by a two-dimensional coordinate of a position point in the horizontal plane corresponding to the global coordinate system, while omitting the height coordinate in the global coordinate system. In addition, the posture included in the above target global pose can be represented by a specific angle, which can be: the angle between the orientation of the smart mobile device and the specified coordinate axis in the horizontal plane of the above global coordinate system when generating the frame environment point cloud.

[0206] For example, the above-mentioned global coordinate system is the world coordinate system, then the smart mobile device is located in the XOY plane (horizontal plane) of the world coordinate system, and its coordinate in the Z axis (vertical direction) is 0. Therefore, the position included in the above-mentioned target global posture is usually a two-dimensional coordinate expressed in the form of (x-axis coordinate, y-axis coordinate). The position point corresponding to the two-dimensional coordinate in the XOY plane of the world coordinate system can be understood as the position of the smart mobile device in the XOY plane, and the posture included in the above-mentioned target global posture is expressed by a specific angle, which is the angle between the orientation of the smart mobile device and the X-axis or Y-axis of the world coordinate system.

[0207] For the first frame environment point cloud, its registration result is: set pose in the global coordinate system. That is to say, for the first frame environment point cloud, a pose in the global coordinate system can be directly pre-set for the first frame environment point cloud as the registration result of the first frame environment point cloud. The directly pre-set pose can be called the set pose.

[0208] Exemplarily, for the first frame environment point cloud, the set posture may be pre-set to include position (0,0) and angle a, and the X-axis in the horizontal plane of the global coordinate system may be set as the specified coordinate axis. The position (0,0) included in the set posture corresponds to the origin of the global coordinate system, that is, the position (0,0) indicates that when the first frame environment point cloud is to be generated, the position of the smart mobile device in the global coordinate system is set as the origin of the global coordinate system, and the angle a included in the set posture indicates that when the first frame environment point cloud is to be generated, the angle between the orientation of the smart mobile device and the X-axis of the global coordinate system is a.

[0209] Optionally, the smart mobile device may be equipped with a positioning sensor for determining the position and posture of the smart mobile device in the world coordinate system, such as an inertial measurement unit, a wheel speed recorder, a laser odometer, a visual odometer, etc. Therefore, when the above-mentioned global coordinate system is the world coordinate system, the position and posture determined by the positioning sensor at the same time can be read from the positioning sensor carried by the smart mobile device at the same time when the first frame of the environmental point cloud is generated, and used as the set position and posture.

[0210] For example, at the same time when the first frame of the environmental point cloud is generated, the posture determined by the inertial measurement unit read from the inertial measurement unit carried by the smart mobile device at the same time is M, and the posture M can be directly used as the above-mentioned set posture.

[0211] In this way, the position and posture of the intelligent mobile device determined by the above-mentioned positioning sensor is the position and posture of the intelligent mobile device in the world coordinate system, and the map coordinate system of the above-mentioned preset raster map is the same as the world coordinate system. Therefore, in the subsequent use of the constructed scene map, when positioning and navigating the intelligent mobile device located in the scene, there is no need to convert between the map coordinate system of the scene map and the world coordinate system, which simplifies the calculation process of positioning the intelligent mobile device in the world coordinate system, reduces the calculation pressure, and improves the positioning efficiency.

[0212] S302: Based on the global point cloud, determine the current distance value and the current normal vector of each target grid of the grid map.

[0213] Wherein, each target grid includes: the matching grids to which each point of the global point cloud belongs in the grid map, and the adjacent grids whose positional relationship with any matching grid meets the adjacent condition;

[0214] For each frame of the environment point cloud, when the frame of the environment point cloud is converted into a global point cloud in the global coordinate system based on the registration result of the frame of the environment point cloud, the normal vector of each point of the global point cloud can be determined at the same time.

[0215] Since the map coordinate system of the above-mentioned grid map is the same coordinate system as the above-mentioned global coordinate system, for each point of the above-mentioned global point cloud, the position of the point in the above-mentioned grid map can be determined according to the coordinates of the point in the above-mentioned global coordinate system, and thus, according to the above-mentioned position, the grid to which the point belongs in the above-mentioned grid map is determined as the matching grid. Moreover, for each matching grid, the grid whose positional relationship with the matching grid meets the adjacent condition can be screened in the above-mentioned grid map as the adjacent grid. Furthermore, the above-mentioned matching grid and the above-mentioned adjacent grid can both be used as target grids, and then, the current distance value and the current normal vector of each of the above-mentioned target grids can be determined.

[0216] Thus, for each target grid, if the target grid is the matching grid, the current distance value of the matching grid is set to a specified value, where the specified value is a constant, for example, 0. Furthermore, the current normal vector of the matching grid is determined to be: the normal vector of the point in the global point cloud belonging to the matching grid.

[0217] Furthermore, for each matching grid determined above, the point in the global point cloud belonging to the matching grid can be used as the point corresponding to the matching grid, so that the grid in the grid map whose distance from the matching grid along the normal vector direction of the point corresponding to the matching grid does not exceed the set distance can be used as the adjacent grid. For each adjacent grid, the matching grid closest to the adjacent grid is screened, so that the distance between the adjacent grid and the closest matching grid screened can be used as the current distance value of the adjacent grid, and the normal vector of the closest matching grid screened can be used as the current normal vector of the adjacent grid.

[0218] For example, Figure 4 As shown in the figure, the black grids are the matching grids to which the points of a certain frame of environment point cloud are converted into the global point cloud in the global coordinate system; the direction pointed by the arrow in each black grid is the normal vector direction of the point corresponding to the matching grid; and the preset distance is 1. For each black grid, the gray grid above it is the adjacent grid of the black grid, and the current distance value of each adjacent grid is the preset distance 1, and the current normal vector of each adjacent grid is: the direction pointed by the arrow in the black grid below it.

[0219] S303: For each target grid, if the target grid records a distance value and a normal vector, based on the determined current distance value and current normal vector of the target grid, the distance value and normal vector recorded in the target grid are updated respectively; otherwise, the determined distance value and normal vector of the target grid are recorded in the target grid.

[0220] For each frame of the environmental point cloud, after determining the current distance value and the current normal vector of each target grid in the above-mentioned grid map, for each target grid, if the target grid has already recorded a distance value and a normal vector, the distance value and the normal vector recorded in the target grid can be updated based on the determined current distance value and the current normal vector of the target grid. That is, based on the determined current distance value of the target grid, the distance value recorded in the target grid is updated, and based on the determined current normal vector of the target grid, the normal vector recorded in the target grid is updated.

[0221] On the contrary, for each target grid, if the target grid does not have a distance value and a normal vector recorded therein, the determined current distance value and current normal vector of the target grid may be recorded in the target grid.

[0222] Among them, for the above-mentioned first-frame environmental point cloud, when determining the current distance value and the current normal vector of each target grid, since the distance value and the normal vector are not recorded in each grid of the above-mentioned grid map, for the above-mentioned first-frame environmental point cloud, the current distance value and the current normal vector of each target grid can be directly recorded in the target grid.

[0223] For each frame of environmental point cloud except the first frame, there are grids with distance values ​​and normal vectors recorded in the above-mentioned grid map. Therefore, for each frame of environmental point cloud except the first frame, when determining the current distance value and current normal vector of each target grid, for each target grid, you can first check whether the distance value and normal vector are already recorded in the target grid. If the distance value and normal vector are already recorded in the target grid, then based on the determined current distance value and current normal vector of the target grid, update the distance value and normal vector recorded in the target grid respectively; otherwise, record the determined current distance value and current normal vector of the target grid in the target grid.

[0224] Optionally, in a specific implementation, for each target grid, if the target grid records a distance value and a normal vector, then based on the determined current distance value and current normal vector of the target grid, the distance value and normal vector recorded in the target grid are respectively updated, which may include the following steps 3031-3032.

[0225] Step 3031: Calculate a new distance value of the target grid using the current distance value of the target grid, the current weight of the target grid, the distance value recorded by the target grid, and the weight recorded by the target grid, and record the new distance value in the target grid.

[0226] Step 3032: Calculate a new normal vector of the target grid using the current normal vector of the target grid, the current weight of the target grid, the normal vector recorded by the target grid, and the weight recorded by the target grid, and record the new normal vector in the target grid.

[0227] In this specific implementation method, since the current distance value and the current normal vector of each target grid are directly recorded in the target grid for the above-mentioned first frame environment point cloud, this specific implementation method is executed for each frame of environment point cloud except the first frame environment point cloud, that is, it is executed for each frame of point cloud to be registered.

[0228] For each frame of environmental point cloud except the first frame of environmental point cloud, after determining the current distance value of each target grid, for each target grid, if the target grid records a distance value and the current weight of the target grid has been determined, the current distance value of the target grid, the current weight of the target grid, the distance value recorded by the target grid, and the weight recorded by the target grid can be used to calculate the new distance value of the target grid, and the new distance value is recorded in the target grid. In this way, for each target grid, if the target grid records a distance value, after updating the distance value recorded by the target grid based on the determined current distance value of the target grid, the distance value recorded by the target grid is: the new distance value calculated above.

[0229] Optionally, for each target grid, if the target grid records a distance value, the following formula (1) can be used to calculate a new distance value of the target grid.

[0230]

[0231] in, is the new distance value of the jth target grid, W(p) j is the weight of the j-th target grid record, DF(p) j is the distance value recorded by the jth target grid, df(p) j is the current distance value of the jth target grid, w(p) j is the current weight of the j-th target grid.

[0232] Correspondingly, for each frame of the environmental point cloud except the first frame, after determining the current normal vector of each target grid, for each target grid, if the target grid has a normal vector recorded, and the current weight of the target grid has been determined, the current normal vector of the target grid, the current weight of the target grid, the normal vector recorded by the target grid, and the weight recorded by the target grid can be used to calculate the new normal vector of the target grid, and record the new normal vector in the target grid. In this way, for each target grid, if the target grid has a normal vector recorded, then after updating the normal vector recorded by the target grid based on the determined current normal vector of the target grid, the normal vector recorded by the target grid is: the new normal vector calculated above.

[0233] Optionally, for each target grid, if the target grid records a normal vector, the following formula (2) can be used to calculate a new normal vector of the target grid.

[0234]

[0235] in, is the new normal vector of the j-th target grid, W(p) j The weight of the jth target grid record, NF(p) j is the normal vector recorded for the jth target grid, nf(p) j is the current normal vector of the j-th target grid, w(p) j is the current weight of the j-th target grid.

[0236] In order to implement the above formula (1) and formula (2), it is necessary to first obtain the current weight of the target grid and the weight recorded by the target grid.

[0237] Correspondingly, based on the above formula (1) and formula (2), a point cloud registration method provided in an embodiment of the present application may further include the following steps 3-4.

[0238] Step 3: For the first frame of the environmental point cloud, in response to obtaining the registration result of the frame of the environmental point cloud, determine the current weight of each target grid, and record the current weight of each target grid in the target grid.

[0239] Step 4: For each frame of environmental point cloud except the first frame of environmental point cloud, in response to obtaining the registration result of the frame of environmental point cloud, determine the current weight of each target grid, and use the current weight of the target grid to update the weight recorded in the target grid.

[0240] For each frame of environmental point cloud, after obtaining the registration result of the frame of environmental point cloud, and based on the registration result of the frame of environmental point cloud, converting the frame of environmental point cloud into a global point cloud in the global coordinate system, weights can be assigned to each point of the global point cloud to obtain the initial weights of each point of the global point cloud, and the current weight of each target grid can be determined based on the initial weights of each point of the target global point cloud.

[0241] For example, for each target grid, if the target grid is a matching grid, the current weight of the matching grid is: the weight of the points belonging to the target grid in the above global point cloud, and if the target grid is an adjacent grid, the current weight of the adjacent grid is: the current weight of the matching grid closest to the adjacent grid, that is, if the target grid is an adjacent grid, the current weight of the adjacent grid is: the weight of the points belonging to the matching grid closest to the adjacent grid in the above global point cloud.

[0242] Of course, the current weight of each target grid can also be determined based on the initial weights of each point of the target global point cloud in other ways. For example, the current weight of each target grid can be determined based on the initial weights of each point of the target global point cloud, combined with the current normal vector and pose uncertainty of the target grid. This application does not make any specific limitations on this.

[0243] As for the first frame environment point cloud, when determining the current weight of each target grid, since no weight is recorded in each grid of the above grid map, the current weight of each target grid can be directly recorded in the target grid.

[0244] In this way, for the first frame of the environment point cloud, in response to obtaining the registration result of the frame of the environment point cloud, the weight of each target grid is determined, and the current weight of each target grid is recorded in the target grid.

[0245] Correspondingly, for each frame of environmental point cloud except the first frame of environmental point cloud, in response to obtaining the alignment result of the frame of environmental point cloud, the current weight of each target grid is determined, and for each target grid, if the target grid does not have a weight recorded, the current weight of the target grid is recorded in the target grid; if the target grid has a weight recorded, the current weight of the target grid calculated as above is used to update the weight recorded in the target grid.

[0246] In this way, for each frame of environmental point cloud, after completing the registration results of the frame of environmental point cloud and updating the distance values ​​and normal vectors recorded in each grid in the grid map, for the frame of environmental point cloud, each target grid determined records the distance value, normal vector and weight at the same time.

[0247] That is to say, in the iterative process of updating the above-mentioned grid map using each frame of the environment point cloud, there may be grids in the above-mentioned grid map that record distance values, normal vectors and weights at the same time, or there may be grids that do not record distance values, normal vectors and weights at the same time, and there are no grids that only record any one or any two of the distance values, normal vectors and weights.

[0248] Optionally, for each frame of the environment point cloud except the first frame of the environment point cloud, if the target grid record has a weight, the following formula (3) can be used to calculate the new weight of the target grid.

[0249]

[0250] in, is the new weight obtained by updating the weight of the j-th target grid record based on the current weight of the j-th target grid, w(p) j is the current weight of the j-th target grid, W(p) j The weight of the j-th target grid record.

[0251] Optionally, for each frame of environmental point cloud, after obtaining the registration result of the frame of environmental point cloud, and based on the registration result of the frame of environmental point cloud, converting the frame of environmental point cloud into a global point cloud in the global coordinate system, the point cloud error model can be used to assign weights to each point of the global point cloud to obtain the initial weights of each point of the global point cloud.

[0252] Generally, according to the point cloud error model, the farther the point cloud is from the center of the detection sensor used to generate the point cloud, the greater the error of the point cloud. Therefore, for the same point cloud, the farther the point in the point cloud is from the center of the detection sensor used to generate the point cloud, the greater the error of the point. Based on this, for each frame of the environment point cloud, after converting the frame of the environment point cloud into a global point cloud in the global coordinate system, the weight of each point of the global point cloud can be determined based on the preset point cloud error model, and among the points of the global point cloud, the farther the distance from the center of the detection sensor used to generate the point cloud, the smaller the weight of the point.

[0253] For example, the above-mentioned global point cloud A includes point A and point B, and the distance between point A and the center of the detection sensor is a, the distance between point B and the center of the detection sensor is b, and a>b, then the initial weight of point A obtained by weight assignment is smaller than the initial weight of point B.

[0254] As mentioned above, according to the generation order of each frame of environmental point cloud, for each frame of environmental point cloud, after obtaining the registration result of the frame of environmental point cloud, the distance values ​​and normal vectors of each grid record of the above-mentioned grid map are updated based on the registration result of the frame of environmental point cloud. Therefore, the point cloud registration method provided in the above-mentioned embodiment of the present application is a link in the process of constructing the scene map of the scene in which the smart mobile device is currently located. In order to realize the application of smart mobile devices in the above-mentioned scenes, the above-mentioned point cloud registration method can be used to complete the construction of the above-mentioned scene map by updating the distance values ​​and normal vectors of each grid record of the above-mentioned grid map in accordance with the generation order of each frame of environmental point cloud, and obtaining a completed scene map.

[0255] Based on this, optionally, in a specific implementation, such as Figure 5 As shown, a point cloud registration method provided in an embodiment of the present application may further include the following step S106.

[0256] S106: In response to the smart mobile device passing through all areas within the scene, the current grid map recording the distance values ​​and normal vectors is determined as the scene map of the scene.

[0257] When constructing a scene map for an unfamiliar scene entered by a smart mobile device, in order to avoid missing areas in the unfamiliar scene, the smart mobile device has the function of checking whether there are areas in the scene that have not been passed through. Therefore, when the smart mobile device moves in the scene, while the detection sensor carried by it detects the environment of the scene and generates each frame of the environment point cloud, it can also record the areas in the scene that the smart mobile device passes through. Therefore, based on the movement data recorded during the movement, it can be detected whether the scene has passed through all areas in the scene.

[0258] For example, the smart mobile device can record the scene information of the area that has been passed, for example, record the image features of the scene image of the area that has been passed, so that when the smart mobile device leaves the current position and moves to the next position, it can compare the scene information of the area where the current position is moved to with the recorded scene information to determine whether the area where the current position is moved to is the area that has been passed. If so, leave the position and move to the next position again. If not, at the current position, the detection sensor can be used to perform environmental detection of the area where the current position is located to obtain an environmental point cloud. The cycle is repeated until the smart mobile device detects that no matter which direction it moves, the area where the next position it reaches belongs to the area that has been passed, and it is determined that the smart mobile device has passed all areas in the scene.

[0259] When the smart mobile device has passed through all areas of the scene, it can be considered that the smart mobile device has completed the environmental exploration of the scene. At this time, the smart mobile device can stop moving, stop the environmental exploration of the scene, and also stop generating the environmental point cloud. In this way, in response to the smart mobile device passing through all areas in the scene, the grid map currently recording the distance value and the normal vector can be determined as the scene map of the above scene.

[0260] For example, Figure 6 A schematic diagram of a scene map recording distance values ​​and normal vectors provided in an embodiment of the present application. Figure 6 As shown, the scene map is a grid map. Figure 6 Each square in is a grid in the map, the number in each grid is the distance value recorded in the grid, and the arrow in each grid is the normal vector recorded in the grid.

[0261] Optionally, in a specific implementation, such as Figure 7 As shown, the above S103, based on the initial pose, and the distance values ​​and normal vectors of each grid record of the preset grid map, determines the target global pose of the point cloud to be registered in the global coordinate system, which may include the following steps S1031-1037.

[0262] S1031: Determine each candidate angle based on the initial angle included in the initial posture and a preset angle search window.

[0263] S1032: Determine each candidate position based on the initial position included in the initial posture and a preset position search window.

[0264] For the acquired point cloud to be registered, the initial posture of the intelligent mobile device in the above-mentioned global coordinate system when the point cloud to be registered is generated can be determined, and the above-mentioned initial posture is the posture determined by the positioning sensor carried by the intelligent mobile device at the same time when the point cloud to be registered is generated, wherein the initial posture can include an initial position and an initial angle.

[0265] Taking into account the posture errors caused by equipment errors, temperature drift, etc., the initial posture determined by the above positioning sensor may have errors, and the error may be an error in the above initial position, an error in the above initial angle, or an error in both the above initial position and the above initial angle. Therefore, the angle search window and the position search window can be pre-set according to the angle range where the error in the above initial angle may be located, and the position range where the error in the above initial position may be located. In this way, each candidate angle can be determined based on the above initial angle and the angle search window, and each candidate position can be determined based on the above initial position and the position search window.

[0266] The angle search window is an angle range, the maximum value of the angle range is the angle value obtained by rotating the initial angle along the set positive direction by the first angle, and the minimum value of the angle range is the angle value obtained by rotating the initial angle along the set negative direction by the second angle. The first angle and the second angle may be the same or different. The angle search window includes the initial angle, and each of the determined candidate angles includes the initial angle.

[0267] For example, assuming that the clockwise direction is the negative direction and the counterclockwise direction is the positive direction, the maximum value of the angle search window is the angle value reached by rotating the initial angle counterclockwise by a first angle, and the minimum value of the angle search window is the angle value reached by rotating the initial angle clockwise by a second angle.

[0268] For example, Figure 8 As shown, it is a schematic diagram of the relationship between the initial angle and the angle search window. Assuming that the clockwise direction is the negative direction, the counterclockwise direction is the positive direction, the initial angle is 0 degrees, the first angle and the second angle are both α, then the obtained angle search window is (-α, α).

[0269] Similarly, the position search window is a position range, which is an area including the initial position and having a specified shape and size. The position search window includes the initial position, and each determined candidate position includes the initial position.

[0270] For example, the position search window is a positive direction area of ​​a specified length centered at the initial position.

[0271] For example, Fig. 9 As shown, it is a schematic diagram of the relationship between the initial position and the position search window, wherein the position of the grid where the solid triangle is located is the initial position, and the square area centered at the initial position and with the sum of the side lengths of 5 grids as the position search window, and the position of each grid in the position search window may be a candidate position.

[0272] Optionally, in a specific implementation, the above step S1031 may include the following step 5.

[0273] Step 5: Based on the initial angle included in the initial posture, the preset angle search window and the angle search step, determine each candidate angle that is within the angle search window and includes the initial angle.

[0274] In this specific implementation, for the point cloud to be registered, after obtaining the above initial pose, each candidate angle can be determined based on the initial angle included in the initial pose, the preset angle search window and the angle search step. Wherein, each candidate angle is located within the angle search window, and each candidate angle includes the initial angle.

[0275] The angle search step may be gradually reduced or increased starting from the maximum or minimum value of the angle search window, so that the result of each reduction or increase is used as each candidate angle. When the result of each reduction or increase includes the initial angle, the initial angle is determined as the candidate angle, and when the result of each reduction or increase does not include the initial angle, the initial angle may be added to the candidate angle after the angle search step is gradually reduced or increased, that is, the initial angle is also determined as the candidate angle.

[0276] For example, if the initial angle in the above initial posture is 0 degree, the preset angle search window is (-3 degrees, 3 degrees), and the angle search step is 1 degree, then the candidate angles can be determined to be: -2 degrees, -1 degree, 0 degrees, 1 degree and 2 degrees.

[0277] For another example, if the initial angle in the above initial posture is 0 degree, the preset angle search window is (-3 degrees, 3 degrees), and the angle search step is 2 degrees, then it can be determined that the candidate angles are: 1 degree, -1 degree and 0 degree.

[0278] The angle search step size may be set in a variety of ways according to actual application needs, for example, according to historical experience values.

[0279] Optionally, in a specific implementation, when determining the above-mentioned candidate angles, before the above-mentioned step 5, a point cloud registration method provided by an embodiment of the present invention can also perform the following step 6.

[0280] Step 6: Calculate the angle search step size based on the preset grid map resolution and the specified maximum distance.

[0281] The specified maximum distance is: the maximum distance between each point of the point cloud to be registered and the center of the detection sensor used to generate the point cloud to be registered.

[0282] In this specific implementation, for the point cloud to be registered, after obtaining the above-mentioned initial posture, before determining each candidate angle, the distance between each point of the point cloud to be registered and the center of the detection sensor used to generate the point cloud to be registered can be first determined, and the maximum distance among the above-mentioned distances is used as the specified maximum distance. After that, based on the resolution of the above-mentioned grid map and the above-mentioned specified maximum distance, the angle search step is calculated. Among them, the resolution of the above-mentioned grid map is the side length of each grid in the above-mentioned grid map. After calculating the above-mentioned angle search step, each candidate angle is determined based on the initial angle included in the initial posture, the preset angle search window and the above-mentioned calculated angle search step.

[0283] Optionally, the angle search step size may be calculated using the following formula (4):

[0284]

[0285] Among them, θ is the angle search step, r is the resolution of the grid map, and d max is the specified maximum distance, n is the super-resolution coefficient, and n≥1.

[0286] In the above formula (4), n is a constant greater than or equal to 1. When n=1, the angle search step, the resolution of the above grid map and the specified maximum distance satisfy the following equation: Fig.10 The geometric relationship shown in Figure 2 is Fig.10 The triangle shown is an isosceles triangle; when n>1, a smaller angle search range can be supported.

[0287] Optionally, in a specific implementation, the above step S1032 may include the following step 7.

[0288] Step 7: Based on the initial position included in the initial posture, the preset position search window and the position search step, determine each candidate position that is within the position search window and includes the initial position.

[0289] In this specific implementation, for the point cloud to be registered, after obtaining the above initial pose, each candidate position can be determined based on the initial position, the preset position search window and the position search step included in the initial pose. Wherein, each candidate position is located in the position search window, and each candidate window includes the initial position.

[0290] The above initial position may be determined as a candidate position, and taking the above initial position as a starting point, each position whose distance from the previous candidate position in any specified direction is the above position search step may be determined as each candidate position.

[0291] For example, Fig. 9 The initial position and position search window shown in the figure, assuming that the position search step length is the side length of the grid map, and the specified direction includes horizontal and vertical directions, then Fig. 9 The positions of each grid in the position search window shown can be used as candidate positions.

[0292] For example, Fig. 9 Assume that the position search step length is twice the side length of the grid map, and the specified direction includes horizontal and vertical directions. Fig. 9 The positions of the grids marked with "x" in the position search window shown and the above initial positions are used as candidate positions.

[0293] The angle search step length can be set in a variety of ways according to the needs of actual applications, for example, according to historical experience values. Optionally, the position search step length can be: a preset resolution of the grid map. That is, the position search step length can be: the side length of each grid in the grid map.

[0294] S1033: Determine each candidate pose based on each candidate angle and each candidate position.

[0295] Each candidate posture includes: a candidate angle and a candidate position, and different candidate postures include different candidate angles and / or candidate positions.

[0296] For the point cloud to be registered, after determining the above-mentioned candidate angles and candidate positions, each candidate pose can be determined based on each candidate angle and each candidate position, and each candidate pose is a pose in the above-mentioned global coordinate system.

[0297] Any candidate angle can be combined with any candidate position to obtain a candidate pose. In this way, each candidate pose obtained includes: a candidate angle and a candidate position, and different candidate poses include different candidate angles and / or candidate positions.

[0298] That is to say, for two different candidate postures, they may include the same candidate angle and different candidate positions, or may include different candidate angles and the same candidate position, or may include different candidate angles and different candidate positions. That is, the candidate angles and candidate positions included in different candidate postures are not completely the same.

[0299] For example, for the point cloud A to be registered, the determined candidate angles include angle A1, angle A2 and angle A3, and the determined candidate positions include position a1, position a2 and position a3. Then, candidate pose 1, candidate pose 2 and candidate pose 3 can be determined, where candidate pose 1 includes angle 1 and position a1, candidate pose 2 includes angle 1 and position a2, and candidate pose includes angle 2 and position a1.

[0300] Among them, all different postures of the candidate angles and / or candidate positions that can be obtained by combining the candidate angles and the candidate positions can be used as candidate postures, or only a part of them can be selected as candidate postures.

[0301] For example, for the point cloud A to be registered, the determined candidate angles include angle A1, angle A2 and angle A3, and the determined candidate positions include position a1, position a2 and position a3, then 9 poses 1-9 with different candidate angles and / or candidate positions can be combined, wherein pose 1 includes angle A1 and position a1, pose 2 includes angle A1 and position a2, pose 3 includes angle A1 and position a3, pose 4 includes angle A2 and position a1, pose 5 includes angle A2 and position a2, pose 6 includes angle A2 and position a3, pose 7 includes angle A3 and position a1, pose 8 includes angle A3 and position a2, and pose 9 includes angle A3 and position a3. Then, all of the above poses 1-9 can be used as candidate poses to obtain 9 candidate poses, or some of the poses can be selected from the above poses 1-9 as candidate poses. For example, poses 1-5 are selected as candidate poses.

[0302] S1034: Based on each candidate pose, convert the point cloud to be registered into an initial global point cloud in the global coordinate system;

[0303] S1035: For each candidate pose, based on the initial global point cloud obtained by conversion, and the distance values ​​and normal vectors of each grid record of the preset grid map, a first type score of the candidate pose is calculated, and based on the initial global point cloud obtained by conversion, and the distance values ​​of each grid record of the grid map, a second type score of the candidate pose is calculated;

[0304] For the point cloud to be registered, after obtaining the above-mentioned candidate poses, each candidate pose can be scored, and the target global pose of the frame environment point cloud in the global coordinate system can be selected according to the score of each candidate pose.

[0305] For each candidate pose, based on the candidate pose, the point cloud to be registered is converted into an initial global point cloud in the global coordinate system. Then, based on the converted initial global point cloud and the distance values ​​and normal vectors of each grid record of the above grid map, the first type score of the candidate pose is calculated, and based on the converted initial global point cloud and the distance values ​​of each grid record of the above grid map, the second type score of the candidate pose is calculated.

[0306] Optionally, in a specific implementation, the above S1035 may include the following steps 8-10.

[0307] Step 8: For each point of the initial global point cloud obtained by the conversion, determine the current normal vector of the point and the normal vector of the grid record to which the point belongs in the preset grid map as the historical normal vector of the point, and determine the distance value of the grid record to which the point belongs in the grid map as the historical distance value of the point.

[0308] Step 9: Filter the maximum distance value among the distance values ​​of each grid record of the grid map;

[0309] In this specific implementation, for each candidate pose, after the point cloud to be registered is converted into an initial global point cloud in a global coordinate system based on the candidate pose, the current normal vector of the point and the normal vector of the grid record to which the point belongs in the above-mentioned grid map can be determined for each point of the initial global point cloud as the historical normal vector of the point, and the distance value of the grid record to which the point belongs in the grid map can be determined as the historical distance value of the point.

[0310] Among them, for each point of the initial global point cloud, if the grid to which the point belongs in the above-mentioned grid map does not record a distance value and a normal vector, the current normal vector of the point can be used as the historical normal vector of the point, and the above-mentioned specified value can be used as the historical distance value of the point.

[0311] For the point cloud to be registered, when determining the registration result of the point cloud to be registered, if there are grids with distance values ​​recorded in the grid map, the maximum distance value among the distance values ​​recorded in each grid of the grid map can be screened.

[0312] Step 10: Based on the current normal vector, historical normal vector, historical distance value, and maximum distance value of each point in the initial global point cloud obtained by transformation, the first category score of the candidate pose is calculated, and based on the historical distance value and maximum distance value of each point in the initial global point cloud obtained by transformation, the second category score of the candidate pose is calculated.

[0313] In this way, for each candidate pose, the point cloud to be registered is converted into an initial global point cloud in the global coordinate system based on the candidate pose, and after determining the above-mentioned maximum distance value and the current normal vector, historical normal vector and historical distance value of each point of the above-mentioned initial global point cloud, the first type score of the candidate pose can be calculated based on the above-mentioned maximum distance value and the current normal vector, historical normal vector and historical distance value of each point of the initial global point cloud, and the second type score of the candidate pose can be calculated based on the above-mentioned maximum distance value and the historical distance value of each point of the initial global point cloud.

[0314] Optionally, in a specific implementation, the above step 10 may include the following steps 11-13:

[0315] Step 11: Calculate the distance difference between the maximum distance value and the historical distance value of each point of the initial global point cloud obtained by conversion, and calculate the sum of all the distance differences obtained as the second type score of the candidate pose;

[0316] In this specific implementation, for each candidate posture, the distance difference between the above-mentioned maximum distance value and the historical distance value of each point of the converted initial global point cloud can be first calculated, and the sum of all the distance differences obtained can be calculated as the second type score of the candidate posture.

[0317] For each candidate posture, the second category score of the candidate posture can be calculated using the following formula (5).

[0318]

[0319] in, is the i-th point in the initial global point cloud in the global coordinate system obtained by transforming the registration point cloud based on the k-th candidate pose, is the historical distance value of the above-mentioned i-th point, Dis maxis the maximum distance value mentioned above, N is the total number of points in the initial global point cloud in the global coordinate system obtained by transforming the point cloud to be registered based on the kth candidate pose, is the second category score of the kth candidate pose.

[0320] Step 12: Calculate the normal vector difference between the current normal vector of each point of the initial global point cloud obtained by conversion and the historical normal vector of the point, and select the maximum normal vector difference among all the normal vector differences obtained;

[0321] Step 13: Calculate the difference between the maximum normal vector difference and each normal vector difference as the target difference, and calculate the sum of all the target differences and all the distance differences as the first category score of the candidate pose.

[0322] For each candidate pose, after converting the point cloud to be registered into an initial global point cloud in the global coordinate system based on the candidate pose, the normal vector difference between the current normal vector of the point and the historical normal vector of the point can be calculated for each point in the initial global point cloud. After that, after all the normal vector differences are obtained, the maximum normal vector difference among all the normal vector differences can be screened. Then, the difference between the maximum normal vector difference and each normal vector difference obtained above can be calculated as the target difference. Then, the sum of all the target differences and all the distance differences obtained in the above step 11 can be calculated to obtain the first category score of the candidate pose.

[0323] For each candidate posture, the first category score of the candidate posture can be calculated using the following formulas (6)-(8).

[0324]

[0325]

[0326]

[0327] in, is the i-th point in the initial global point cloud in the global coordinate system obtained by transforming the registration point cloud based on the k-th candidate pose, is the current normal vector of the i-th point above, is the historical normal vector of the i-th point above, is the maximum normal vector difference mentioned above, Dis max is the maximum distance value mentioned above, is the historical distance value of the above-mentioned i-th point, N is the total number of points in the initial global point cloud in the global coordinate system obtained by transforming the registration point cloud based on the k-th candidate pose, is the first-category score of the k-th candidate pose.

[0328] In combination with the above steps 11-13 and the above formulas (5)-(8), the first type of score for each candidate pose can be called the normal vector field-based score of the candidate pose, and the second type of score for each candidate pose can be called the distance field-based score of the candidate pose.

[0329] Based on this, optionally, when the angle search step is calculated using the above formula (4), and then the obtained angle search step is used to determine each candidate pose, when n>1 in the above formula (4), not only a smaller angle search range can be supported, but also, when the candidate angle changes by an angle search step, the grid where the point cloud is located remains unchanged. Therefore, the score based on the distance field of each candidate pose remains unchanged, while the score based on the normal vector field changes.

[0330] S1036: Acquire candidate angles included in the candidate postures of the first category whose scores meet the first preset requirement as target angles, and acquire candidate positions included in the candidate postures of the second category whose scores meet the second preset requirement as target positions;

[0331] For the point cloud to be registered, after calculating the first category scores and the second category scores of the above-mentioned candidate postures, the candidate postures whose first category scores meet the first preset requirements are screened, and the candidate angles among the candidate postures whose first category scores meet the first preset requirements are determined as the target angles, and then, the candidate postures whose second category scores meet the second preset requirements are screened, and the candidate positions among the candidate postures whose second category scores meet the second preset requirements are determined as the target positions.

[0332] The first preset requirement and the second preset requirement may be the same or different.

[0333] Optionally, the candidate posture with the highest score in the first category can be screened, and the candidate angle in the candidate posture with the highest score in the first category can be used as the target angle. Then, the candidate posture with the highest score in the second category can be screened, and the candidate position in the candidate posture with the highest score in the second category can be used as the target position.

[0334] S1037: Determine a pose including a target angle and a target position as a target global pose of the point cloud to be registered in the global coordinate system.

[0335] After obtaining the target angle and target position, the pose including the target angle and target position can be determined as the target global pose of the point cloud to be registered in the global coordinate system. The target global pose of the point cloud to be registered in the global coordinate system can be called the optimal pose of the point cloud to be registered.

[0336] When the smart mobile device is equipped with different types of detection sensors, the methods of obtaining the above-mentioned environmental point clouds of each frame may be different. Below, taking the image acquisition device as a detection sensor as an example, the method of obtaining the environmental point clouds of each frame in the embodiment of the present application is illustrated.

[0337] Optionally, in a specific implementation, such as Fig.11 As shown, the method for obtaining the above-mentioned environment point cloud of each frame may include the following steps S1101-S1103:

[0338] S1101: When the smart mobile device moves in the current scene, each frame of image captured by an image capture device carried by the smart mobile device is acquired.

[0339] S1102: for each frame of image, extract edge features of the ceiling area of ​​the frame of image;

[0340] S1103: For each frame of image, the extracted edge features are converted into a local point cloud in a preset local coordinate system as an environment point cloud.

[0341] In this specific implementation, the image acquisition device carried by the smart mobile device performs image acquisition on the scene where the smart mobile device is currently located to obtain each frame of image. For each frame of image, the edge features of the ceiling area of ​​the frame of image can be extracted, and the extracted edge features can be converted into a local point cloud in a preset local coordinate system, and the local point cloud is the generated environmental point cloud. Among them, the generation order of each frame of environmental point cloud is the acquisition order of the image used to obtain each frame of environmental point cloud, and the first frame of environmental point cloud is the local point cloud determined based on the first frame of image. The above-mentioned local coordinate system is the camera coordinate system of the image acquisition device carried by the above-mentioned smart mobile device.

[0342] For example, Fig.12 As shown, it is a frame of image captured by the above-mentioned image acquisition device, wherein the area surrounded by white dots is the ceiling area, and the white dots are edge features of the ceiling area extracted based on the Canny algorithm.

[0343] In this specific implementation, multiple methods may be used to extract edge features of the ceiling area of ​​each frame of the image.

[0344] For example, when the smart mobile device is equipped with a GPU (Graphics Processing Unit) or NPU (Neural-network Processing Unit), the ceiling area of ​​each frame image can be segmented based on a deep learning algorithm, and the edge features of the segmented ceiling area can be extracted; exemplarily, a CNN (Convolutional Neural Network) is used to train a ceiling area edge feature extraction model for the edge features of the ceiling area of ​​each frame image; for another example, when the smart mobile device is not equipped with the GPU or NPU, the ceiling area of ​​each frame image can be segmented based on a traditional image processing algorithm, and the edge features of the segmented ceiling area can be extracted; exemplarily, a HOG (Histogram of Oriented Gradient) algorithm is used to extract the edge features of the ceiling area by calculating and counting the gradient directions of the local areas in each frame image. In this regard, the embodiments of the present application do not make specific limitations.

[0345] For each of the above-mentioned frames, after the edge features of the ceiling area of ​​the frame image are extracted, the edge features are converted into point clouds to obtain the local point cloud corresponding to the frame image in the camera coordinate system of the image acquisition device as the environment point cloud.

[0346] Among them, the edge features of each frame image are two-dimensional information located in the image coordinate system of the frame image, and the point cloud is usually information in a three-dimensional coordinate system. Therefore, in this specific implementation method, when the edge features of each frame image are converted into a point cloud, the edge features located in the two-dimensional coordinate system are converted into a point cloud in the three-dimensional coordinate system.

[0347] For example, based on the current posture of the image acquisition device and the camera intrinsic parameters, by assuming that the height of the edge features of the ceiling area is consistent, the extracted edge features can be converted into a point cloud in the camera coordinate system of the image acquisition device under the top view angle as the environmental point cloud. The current camera posture can be obtained by a positioning sensor such as an inertial measurement unit carried by a smart mobile device, or by calibrating the external parameters of the image acquisition device. The so-called top view angle refers to the image acquisition device's acquisition angle of view vertically upward, such as Fig.13 The camera perspective shown on the left, the image captured under the top view perspective can be considered as the top view of the ceiling of the scene where the smart mobile device is currently located.

[0348] In this specific implementation, the edge features of the ceiling area of ​​each frame image are used to construct the scene map. Therefore, there is no need to track the feature points in each frame image, and there is no need to accurately estimate the position of the tracked feature points, which can reduce the computing power requirements of the computing chip and save computing power. In addition, when tracking the feature points in each frame image, the illumination changes in the scene will affect the tracking accuracy of the feature points in the image, thereby reducing the accuracy of the position estimation of the feature points and the accuracy of the constructed scene map. The above illumination changes have a small impact on the edge features of the ceiling area in the image. Therefore, compared with the related technology of map construction, which tracks, describes and matches the visual features of the image to construct the scene map, in this specific implementation, the map is constructed based on the edge features of the ceiling area in each frame image, which is less affected by illumination changes and has higher robustness.

[0349] Optionally, for each of the above-mentioned frames, the ceiling area in the frame image can be first identified, for example, a ceiling area recognition model can be trained using a sample image with a ceiling area labeled, so that each of the above-mentioned frames is input into the ceiling area recognition model, and a recognition result output by the ceiling area recognition model is obtained, and the recognition result includes: an image with a ceiling area labeled. Furthermore, for each of the above-mentioned frames, after the ceiling area in the frame image is identified, edge features of the ceiling area can be extracted to obtain edge features of the ceiling area of ​​the frame image. For example, the edge features of the ceiling area of ​​each frame image can be extracted using the Canny edge detection algorithm, and the edge features of the ceiling area of ​​each frame image can be extracted using the Sobel operator.

[0350] In this specific implementation, usually, for the scene currently located by the intelligent mobile machine, since the distance between the ceiling and the ground of the scene is usually much larger than the distance between various objects placed on the ceiling and the ceiling, for example, the distance between the ceiling and the ground of the scene is usually close to 3 meters, while the distance between various objects placed on the ceiling, such as lights, decorations, etc., and the ceiling is usually only a dozen centimeters or even a few centimeters. Therefore, it can be generally considered that the ceiling of the scene is parallel to the ground, and thus, it can be assumed that the height of the edge features extracted from each frame of the image is the same.

[0351] Based on this, the preset ceiling height and the target intrinsic reference model corresponding to each frame of image can be used to perform point cloud conversion on the edge features of each frame of image to obtain the local point cloud corresponding to each frame of image as the environment point cloud, and the above-mentioned local coordinate system is the target camera coordinate system corresponding to the above-mentioned target intrinsic reference model.

[0352] Among them, the above-mentioned preset ceiling height can be determined based on empirical values, or by actual measurement of the scene in which the intelligent robot is currently located, for example, 2.7 meters, 3 meters, etc.; of course, in many cases, it can also be estimated based on the posture of the positioning sensor carried by the intelligent mobile robot before and after the intelligent mobile robot moves a certain distance.

[0353] Optionally, the edge features of each frame of the image can be converted into a point cloud using formulas (9) and (10);

[0354]

[0355]

[0356] Among them, (x mn ,y mn ) is the local point cloud corresponding to the nth edge feature of the mth frame image; (u mn ,y mn ) is the nth edge feature of the mth frame image; z is the ceiling height; f x 、f y 、c x and c y is the model parameter in the target internal reference model.

[0357] For different scenarios and different smart mobile devices, the image acquisition devices carried by different smart mobile devices can be installed at different positions of the smart mobile devices and have different acquisition viewing angles.

[0358] For example, Fig.13 As shown in , the intelligent mobile device is a robot, and the image acquisition device is a camera. Fig.13 As shown in the robot on the left, the camera is installed vertically upward, and the camera's acquisition angle of view is vertically upward, that is, the camera's acquisition angle of view is a top view angle. The machine vertically upward collects the ceiling image, and the collected image can be considered as a top view of the ceiling; Fig.13 As shown in the robot on the right, the camera can be installed tilted upward, and the camera's collecting angle of view is tilted upward. At this time, because the collecting angle of view is tilted at an angle to the vertical direction, the ceiling area in the collected image will be deformed, resulting in angle loss.

[0359] Based on this, optionally, the capturing angle of view of the image capturing device carried by the smart mobile device is vertically upward, and the above S1103 may include the following step 14.

[0360] Step 14: Using the preset target ceiling height and the internal reference model of the image acquisition device, perform point cloud conversion on the edge features of each frame of the image to obtain the local point cloud corresponding to each frame of the image in the camera coordinate system of the image acquisition device as the environment point cloud.

[0361] The target internal reference model is the internal reference model of the image acquisition device, and the target camera coordinate system corresponding to the target internal reference model is the camera coordinate system of the image acquisition device, so the camera coordinate system of the image acquisition device can be used as the local coordinate system. Therefore, the target ceiling height and the internal reference model of the image acquisition device can be directly preset to perform point cloud conversion on the edge features of each frame of the image, and obtain the local point cloud corresponding to each frame of the image in the camera coordinate system of the image acquisition device as the environmental point cloud.

[0362] Optionally, if the image acquisition device carried by the smart mobile device has an inclined acquisition viewing angle, then before the above S1103, a grid map construction method provided in an embodiment of the present application may further include the following steps 15-16.

[0363] Step 15: Based on the tilt angle of the capture viewing angle of the image capture device, perform an inverse perspective transformation on each frame of the image to obtain an inverse perspective image under a vertical upward capture viewing angle.

[0364] Step 16: Determine the target feature corresponding to the edge feature of each frame image in the inverse perspective image as the target feature of each frame image, and determine the internal reference model of the simulated camera used to simulate the acquisition of the inverse perspective image as the target internal reference model.

[0365] Correspondingly, the capturing angle of view of the image capturing device carried by the smart mobile device is tilted, and the above S1103 may include the following step 17.

[0366] Step 17: Using the preset ceiling height and the internal reference model of the simulated camera, the target features of each frame image are converted into a point cloud to obtain a local point cloud corresponding to each frame image in the camera coordinate system of the simulated camera as an environmental point cloud.

[0367] Among them, since the image acquisition device's acquisition angle of view is tilted upward, the ceiling area in the acquired image will be deformed and there will be angle loss. Therefore, before using the preset ceiling height for point cloud conversion, each frame image needs to be converted into an image under the acquisition angle of view of the image acquisition device.

[0368] Therefore, for each frame of image captured by the image acquisition device carried by the smart mobile device, firstly, based on the tilt angle of the acquisition viewing angle of the image acquisition device, each frame of image is subjected to inverse perspective transformation to obtain an inverse perspective image under a vertical upward acquisition viewing angle, and the inverse perspective image can be understood as an image obtained by the image acquisition device performing image acquisition on the ceiling area under a vertical upward acquisition viewing angle. Afterwards, based on the conversion relationship between the image coordinate system of the frame image and the image coordinate system of the inverse perspective image, the target feature corresponding to the edge feature of each frame of image in the inverse perspective image is determined as the target feature of each frame of image. Moreover, for the inverse perspective image, it is equivalent to an image obtained by using a simulated camera with a vertical upward acquisition viewing angle to perform simulated image acquisition on the ceiling area of ​​the scene, therefore, the memory model of the above-mentioned model camera can also be determined, and the memory model of the simulated camera can be used as the target internal reference model, and the camera coordinate system of the simulated camera can be used as the local coordinate system.

[0369] In this way, when performing point cloud conversion, the preset ceiling height and the target internal reference model of the simulated camera can be used to perform point cloud conversion on the target features of each frame image, and the local point cloud corresponding to each frame image in the camera coordinate system of the simulated camera can be obtained as the environmental point cloud.

[0370] That is to say, in this specific implementation, when the edge features of each frame image are converted into a point cloud, the edge features located in the image coordinate system of the inverse perspective image are converted into a point cloud in the camera coordinate system of the simulated camera used to simulate the acquisition of the inverse perspective image.

[0371] Based on the same inventive concept, corresponding to the above-mentioned embodiment of the present application Figure 1 A point cloud registration method is shown, and an embodiment of the present application also provides a smart mobile device.

[0372] Fig.14 A schematic diagram of the structure of a smart mobile device provided in an embodiment of the present application, such as Fig.14 As shown, the smart mobile device may include a detection sensor 100 and a processor 200 .

[0373] The detection sensor 100 is used to detect the environment of the scene when the smart mobile device moves in the current scene, and generate each frame of the environment point cloud in a preset local coordinate system;

[0374] The processor 200 is used to execute a point cloud registration method provided in an embodiment of the present application based on non-first frame environment point clouds in each frame environment point cloud.

[0375] In this specific implementation, in order to construct a scene map of the scene currently located by the smart mobile device, the detection sensor 100 can detect the environment of the scene while the smart mobile device is moving in the current scene, generate each frame of environmental point cloud located in a preset local coordinate system, and then send the generated each frame of environmental point cloud to the processor 200. In this way, the processor 200 can execute a point cloud registration method provided in an embodiment of the present application based on the non-first frame point cloud in each frame of environmental point cloud generated by the above detection sensor 100.

[0376] The specific implementation method of a point cloud registration method provided in an embodiment of the present application has been described in detail above. The processor 200 executes each step of a point cloud registration method provided in an embodiment of the present application according to the above specific implementation method, which will not be repeated here.

[0377] Based on the same inventive concept, corresponding to the above-mentioned embodiment of the present application Figure 1 A point cloud registration method is shown, and an embodiment of the present application also provides a point cloud registration device.

[0378] Fig.15 A schematic diagram of the structure of a point cloud registration device provided in an embodiment of the present application is shown in FIG. Fig.15 As shown, the device may include the following modules:

[0379] The point cloud acquisition module 1510 is used to acquire the point cloud to be registered; wherein the point cloud to be registered is: a non-first frame environmental point cloud located in a preset local coordinate system and generated by performing environmental detection on the scene when the smart mobile device moves in the current scene;

[0380] A posture determination module 1520 is used to determine the initial posture of the intelligent mobile device in a preset global coordinate system when generating the point cloud to be registered;

[0381] The point cloud registration module 1530 is used to determine the target global pose of the point cloud to be registered in the global coordinate system based on the initial pose and the distance values ​​and normal vectors of each grid record of the preset grid map; wherein the distance values ​​and normal vectors of each grid record of the grid map are updated based on the registration results of each frame of the environment point cloud before the point cloud to be registered;

[0382] The result determination module 1540 is used to determine the registration result of the point cloud to be registered based on the target global pose.

[0383] As can be seen from the above, by applying the solution provided by the embodiment of the present application, in order to construct a scene map of an unfamiliar scene entered by a smart mobile device, the smart mobile device is usually equipped with various detection sensors, such as laser radar, image acquisition equipment, etc. After the smart mobile device enters the scene, the smart mobile device can move in the scene. During the movement, the scene can be detected based on the detection sensor carried by the smart mobile device, and each frame of the environment point cloud located in a preset local coordinate system can be generated, and the scene map of the scene can be constructed using the above-mentioned each frame of the environment point cloud. In the above-mentioned scene map construction process, for the generated non-first frame environment point cloud, it is necessary to perform point cloud registration on the environment point cloud. In this way, in the above-mentioned map construction process, the non-first frame environment point cloud located in the preset local coordinate system generated by the environment detection of the scene during the movement of the smart mobile device in the current scene can be first obtained as the point cloud to be registered. Afterwards, the initial position of the smart mobile device in the preset global coordinate system when the registration point cloud is generated can be determined. At this time, for the preset grid map, the distance values ​​and normal vectors recorded in each grid of the grid map are updated based on the registration results of the environmental point clouds of each frame before the point cloud to be registered. Therefore, based on the above initial pose, and the distance values ​​and normal vectors recorded in each grid of the grid map, the target global pose of the point cloud to be registered in the above global coordinate system can be determined, and based on the above target global pose, the registration result of the point cloud to be registered can be determined. At this point, the point cloud registration of the point cloud to be registered is completed, and the global pose of the point cloud to be registered in the above global coordinate system is obtained. Then, the point cloud to be registered and the global pose of the point cloud to be registered in the above global coordinate system can be used to update the grid map. The above scene map is constructed by looping in sequence.

[0384] Based on this, by applying the solution provided in the embodiment of the present application, the grid map in which each grid records the distance value and the normal vector can be called a grid map based on the distance field and the normal vector field, so that the point cloud to be registered can be registered with the grid map based on the distance field and the normal vector field, and the global pose of the point cloud to be registered can be calculated, and the global pose can be understood as the pose of the smart mobile device. In this way, in the process of constructing a scene map of an unfamiliar scene entered by a smart mobile device, it is possible to determine the global pose of each frame of the environmental point cloud detected by the smart mobile device in the global coordinate system, and then, to update the map scene using each frame of the environmental point cloud and the global pose of each frame of the environmental point cloud.

[0385] Furthermore, the existing point cloud registration algorithm based on the discovery of the normal vector solves the pose by matching two frames of point clouds, while in the solution provided in the embodiment of the present application, the point cloud is matched with a grid map to solve the pose, wherein the normal vector constraint is considered in the point cloud registration process based on the grid map, and the consistency of the normal vector is taken into account, and then, the erroneous matches of inconsistent normal vectors are filtered out, thereby improving the robustness and accuracy of the point cloud registration.

[0386] Optionally, in a specific implementation, the device further includes:

[0387] The grid update module is used to update the distance value and normal vector of each grid record of the grid map based on the registration result of the point cloud to be registered.

[0388] Optionally, in a specific implementation, the grid updating module is specifically used to:

[0389] Based on the registration result of each frame of the environment point cloud, the frame of the environment point cloud is converted into a global point cloud in the global coordinate system; wherein the registration result of the first frame of the environment point cloud is: the set position and posture in the global coordinate system;

[0390] Based on the global point cloud, determine the current distance value and the current normal vector of each target grid of the grid map; wherein the each target grid includes: the matching grids to which each point of the global point cloud belongs in the grid map, and the adjacent grids whose positional relationship with any matching grid meets the adjacent condition;

[0391] For each target grid, if the target grid records a distance value and a normal vector, the distance value and normal vector recorded in the target grid are updated respectively based on the determined current distance value and current normal vector of the target grid; otherwise, the determined distance value and normal vector of the target grid are recorded in the target grid.

[0392] Optionally, in a specific implementation, the point cloud registration module 1530 includes:

[0393] A candidate information determination submodule, configured to determine each candidate angle based on an initial angle included in the initial posture and a preset angle search window, and to determine each candidate position based on an initial position included in the initial posture and a preset position search window;

[0394] A candidate posture determination submodule is used to determine each candidate posture based on each candidate angle and each candidate position; wherein each candidate posture includes: a candidate angle and a candidate position, and different candidate postures include different candidate angles and / or candidate positions;

[0395] A point cloud conversion submodule, used for converting the point cloud to be registered into an initial global point cloud in the global coordinate system based on each candidate pose;

[0396] A candidate pose scoring submodule, for each candidate pose, based on the initial global point cloud obtained by conversion, and the distance values ​​and normal vectors of each grid record of a preset grid map, calculates a first type of score for the candidate pose, and based on the initial global point cloud obtained by conversion, and the distance values ​​of each grid record of the grid map, calculates a second type of score for the candidate pose;

[0397] A candidate posture selection submodule is configured to obtain candidate angles included in the candidate postures of the first category whose scores meet the first preset requirement as target angles, and to obtain candidate positions included in the candidate postures of the second category whose scores meet the second preset requirement as target positions;

[0398] The global pose determination submodule is used to determine the pose including the target angle and the target position as the target global pose of the point cloud to be registered in the global coordinate system.

[0399] Optionally, in a specific implementation, the candidate pose scoring submodule includes:

[0400] An information determination unit, for determining, for each point of the initial global point cloud obtained by the conversion, a current normal vector of the point and a normal vector of a grid record to which the point belongs in a preset grid map as a historical normal vector of the point, and determining a distance value of a grid record to which the point belongs in the grid map as a historical distance value of the point;

[0401] A distance value screening unit, used for screening the maximum distance value among the distance values ​​recorded in each grid of the grid map;

[0402] The pose scoring unit is used to calculate the first type score of the candidate pose based on the current normal vector, historical normal vector and historical distance value of each point of the initial global point cloud obtained by transformation, and the maximum distance value, and calculate the second type score of the candidate pose based on the historical distance value and the maximum distance value of each point of the initial global point cloud obtained by transformation.

[0403] Optionally, in a specific implementation, the posture scoring unit is specifically used to:

[0404] Calculate the distance difference between the maximum distance value and the historical distance value of each point of the initial global point cloud obtained by conversion, and calculate the sum of all the obtained distance differences as the second type score of the candidate pose;

[0405] Calculate the normal vector difference between the current normal vector of each point of the initial global point cloud obtained by transformation and the historical normal vector of the point, and select the maximum normal vector difference among all the normal vector differences obtained;

[0406] The difference between the maximum normal vector difference and each normal vector difference is calculated as the target difference, and the sum of all the target differences and all the distance differences is calculated as the first category score of the candidate pose.

[0407] Optionally, in a specific implementation, the result determination module 1540 includes:

[0408] An increment determination submodule, used to calculate a posture increment of the target global posture based on the target global posture using a preset objective function;

[0409] The result determination submodule is used to calculate the pose and value of the target global pose and the pose increment as the registration result of the point cloud to be registered.

[0410] Optionally, in a specific implementation, the increment determination submodule is specifically used to:

[0411] Based on the target global pose, converting the point cloud to be registered into the target global point cloud in the global coordinate system;

[0412] Based on the current normal vector of each point of the target global point cloud, the normal vector recorded in the grid corresponding to the point, the distance value recorded in the grid corresponding to the point, and the distance value recorded in the neighborhood grid of the grid corresponding to the point, the preset objective function is solved to obtain the posture increment of the target global posture; wherein the grid corresponding to each point of the target global point cloud is: the grid to which the point belongs in the grid map.

[0413] Optionally, in a specific implementation, the preset objective function is:

[0414]

[0415] Where, e1 = φ I (c i ), c i represents the i-th point of the target global point cloud, φ I (c i ) represents the result of bilinear interpolation of the distance value of the grid record corresponding to the i-th point and the distance value of the neighborhood grid record of the grid corresponding to the i-th point; is the current normal vector of the i-th point, Normal(c i ) is the normal vector of the grid record corresponding to the i-th point, and N is the total number of points in the target global point cloud;

[0416] The pose increment is:

[0417]

[0418] in,

[0419]

[0420] Among them, (p x ,p y ) is the point of the point cloud to be registered, For point (p x ,p y ), θ is the angle included in the target global pose, P 00 , P 01 , P 10 and P 11 are the points (p x ,p y ) in the four neighborhood grids of the grid map, Dis(P 10 )、Dis(P 00 )、Dis(P 11 ) and Dis(P 01 ) are the grids P 10 , P 00 , P 11 and P 01 The recorded distance value, (x, y) is the distance between the points (p x ,p y ) is the coordinate of the target in the global coordinate system after the global pose conversion, (x0, y0) is the grid p 00 The coordinates in the grid map, (x1, y1) are the coordinates of the grid p 11 The coordinates in the grid map.

[0421] Optionally, in a specific implementation, the device further includes:

[0422] The map determination module is used to determine the current grid map recording the distance value and the normal vector as the scene map of the scene in response to the intelligent mobile device passing through the entire area within the scene.

[0423] The present application also provides an electronic device, such as Fig.16 As shown, including:

[0424] A memory 161, for storing computer programs;

[0425] The processor 162 is used to implement the steps of any grid map construction method provided in the above-mentioned embodiments of the present application when executing the program stored in the memory 161.

[0426] In addition, the electronic device may further include a communication bus and / or a communication interface, and the processor 162, the communication interface, and the memory 161 communicate with each other via the communication bus.

[0427] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0428] The communication interface is used for communication between the above electronic device and other devices.

[0429] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0430] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0431] In another embodiment provided in the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned point cloud registration methods are implemented.

[0432] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the steps of any point cloud registration method in the above embodiments.

[0433] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state hard disk (SSD), etc.

[0434] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0435] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0436] The above description is only a preferred embodiment of the present application and is not intended to limit the protection scope of the present application. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application are included in the protection scope of the present application.

Claims

1. A point cloud registration method, characterized in that: The method comprises: Acquire a point cloud to be registered; wherein the point cloud to be registered is: a non-first frame environmental point cloud located in a preset local coordinate system and generated by performing environmental detection on the scene when the smart mobile device moves in the current scene; Determining the initial position and posture of the intelligent mobile device in a preset global coordinate system when generating the point cloud to be registered; Based on the initial pose, and the distance values ​​and normal vectors of each grid record of the preset grid map, determine the target global pose of the point cloud to be registered in the global coordinate system; wherein the distance values ​​and normal vectors of each grid record of the grid map are updated based on the registration results of each frame of the environment point cloud before the point cloud to be registered; Based on the target global pose, a registration result of the point cloud to be registered is determined.

2. The method according to claim 1, characterized in that The method further comprises: Based on the registration result of the point cloud to be registered, the distance value and the normal vector of each grid record of the grid map are updated.

3. The method according to claim 1, characterized in that The method of updating the distance value and the normal vector of each grid record of the grid map based on the registration result of each frame of the environment point cloud includes: Based on the registration result of each frame of the environment point cloud, the frame of the environment point cloud is converted into a global point cloud in the global coordinate system; wherein the registration result of the first frame of the environment point cloud is: the set position and posture in the global coordinate system; Based on the global point cloud, determine the current distance value and the current normal vector of each target grid of the grid map; wherein the each target grid includes: the matching grids to which each point of the global point cloud belongs in the grid map, and the adjacent grids whose positional relationship with any matching grid meets the adjacent condition; For each target grid, if the target grid records a distance value and a normal vector, the distance value and normal vector recorded in the target grid are updated respectively based on the determined current distance value and current normal vector of the target grid; otherwise, the determined distance value and normal vector of the target grid are recorded in the target grid.

4. The method according to claim 1, characterized in that: The step of determining the target global pose of the point cloud to be registered in the global coordinate system based on the initial pose and the distance values ​​and normal vectors of each grid record of the preset grid map includes: Determine each candidate angle based on an initial angle included in the initial posture and a preset angle search window, and determine each candidate position based on an initial position included in the initial posture and a preset position search window; Determine each candidate pose based on each candidate angle and each candidate position; wherein each candidate pose includes: a candidate angle and a candidate position, and different candidate poses include different candidate angles and / or candidate positions; Based on each candidate pose, the point cloud to be registered is converted into an initial global point cloud in the global coordinate system; For each candidate pose, based on the initial global point cloud obtained by conversion, and the distance values ​​and normal vectors of each grid record of a preset grid map, a first type score of the candidate pose is calculated, and based on the initial global point cloud obtained by conversion, and the distance values ​​of each grid record of the grid map, a second type score of the candidate pose is calculated; Obtaining candidate angles included in the candidate postures of the first category whose scores meet a first preset requirement as target angles, and obtaining candidate positions included in the candidate postures of the second category whose scores meet a second preset requirement as target positions; A pose including the target angle and the target position is determined as the target global pose of the point cloud to be registered in the global coordinate system.

5. The method according to claim 4, characterized in that The first type of score of the candidate pose is calculated based on the initial global point cloud obtained by conversion, and the distance values ​​and normal vectors of each grid record of the preset grid map, and the second type of score of the candidate pose is calculated based on the initial global point cloud obtained by conversion, and the distance values ​​of each grid record of the grid map, including: For each point of the initial global point cloud obtained by the conversion, determine the current normal vector of the point and the normal vector of the grid record to which the point belongs in the preset grid map as the historical normal vector of the point, and determine the distance value of the grid record to which the point belongs in the grid map as the historical distance value of the point; Filtering the maximum distance value among the distance values ​​of each grid record of the grid map; Based on the current normal vector, historical normal vector and historical distance value of each point in the initial global point cloud obtained by transformation, and the maximum distance value, the first category score of the candidate pose is calculated, and based on the historical distance value and the maximum distance value of each point in the initial global point cloud obtained by transformation, the second category score of the candidate pose is calculated.

6. The method according to claim 5, characterized in that The first type of score of the candidate pose is calculated based on the current normal vector, the historical normal vector and the historical distance value of each point of the initial global point cloud obtained by the conversion, and the maximum distance value, and the second type of score of the candidate pose is calculated based on the historical distance value and the maximum distance value of each point of the initial global point cloud obtained by the conversion, including: Calculate the distance difference between the maximum distance value and the historical distance value of each point of the initial global point cloud obtained by conversion, and calculate the sum of all the obtained distance differences as the second type score of the candidate pose; Calculate the normal vector difference between the current normal vector of each point of the initial global point cloud obtained by transformation and the historical normal vector of the point, and select the maximum normal vector difference among all the normal vector differences obtained; The difference between the maximum normal vector difference and each normal vector difference is calculated as the target difference, and the sum of all the target differences and all the distance differences is calculated as the first category score of the candidate pose.

7. The method according to claim 1, characterized in that The step of determining the registration result of the point cloud to be registered based on the target global pose includes: Based on the target global pose, using a preset objective function, calculating a pose increment of the target global pose; The pose and value of the target global pose and the pose increment are calculated as the registration result of the point cloud to be registered.

8. The method according to claim 7, characterized in that The step of calculating the pose increment of the target global pose based on the target global pose and using a preset objective function includes: Based on the target global pose, converting the point cloud to be registered into the target global point cloud in the global coordinate system; Based on the current normal vector of each point of the target global point cloud, the normal vector recorded in the grid corresponding to the point, the distance value recorded in the grid corresponding to the point, and the distance value recorded in the neighborhood grid of the grid corresponding to the point, the preset objective function is solved to obtain the posture increment of the target global posture; wherein the grid corresponding to each point of the target global point cloud is: the grid to which the point belongs in the grid map.

9. The method according to claim 8, characterized in that The preset objective function is: Where, e1 = φ I (c i ), c i represents the i-th point of the target global point cloud, φ I (c i ) represents the result of bilinear interpolation of the distance value of the grid record corresponding to the i-th point and the distance value of the neighborhood grid record of the grid corresponding to the i-th point; is the current normal vector of the i-th point, Normal(c i ) is the normal vector of the grid record corresponding to the i-th point, and N is the total number of points in the target global point cloud; The pose increment is: in, Among them, (p x , p y ) is the point of the point cloud to be registered, For point (p x , p y ), θ is the angle included in the target global pose, P 00 , P 01 , P 10 and P 11 are the points (p x , p y ) in the four neighborhood grids of the grid map, Dis(P 10 )、Dis(P 00 )、Dis(P 11 ) and Dis(P 01 ) are the grids P 10 , P 00 , P 11 and P 01 The recorded distance value, (x, y) is the distance between the points (p x , p y ) is the coordinate of the target in the global coordinate system after the global pose conversion, (x0, y0) is the grid p 00 The coordinates in the grid map, (x1, y1) are the coordinates of the grid p 11 The coordinates in the grid map.

10. The method according to any one of claims 1 to 9, characterized in that: The method further comprises: In response to the intelligent mobile device passing through all areas within the scene, the current grid map recording distance values ​​and normal vectors is determined as the scene map of the scene.

11. A smart mobile device, characterized in that: including a detection sensor and a processor; The detection sensor is used to detect the environment of the scene when the smart mobile device moves in the current scene, and generate each frame of the environment point cloud in a preset local coordinate system; The processor is used to execute any method described in claims 1-10 based on the non-first frame environment point clouds in each frame environment point cloud.

12. A point cloud registration device, characterized in that: The device comprises: The point cloud acquisition module is used to acquire the point cloud to be registered; wherein the point cloud to be registered is: a non-first frame environmental point cloud located in a preset local coordinate system and generated by performing environmental detection on the scene when the smart mobile device moves in the current scene; A posture determination module, used to determine the initial posture of the intelligent mobile device in a preset global coordinate system when generating the point cloud to be registered; A point cloud registration module, used to determine the target global pose of the point cloud to be registered in the global coordinate system based on the initial pose and the distance values ​​and normal vectors of each grid record of the preset grid map; wherein the distance values ​​and normal vectors of each grid record of the grid map are updated based on the registration results of each frame of environmental point cloud before the point cloud to be registered; The result determination module is used to determine the registration result of the point cloud to be registered based on the target global pose.

13. An electronic device, characterized in that: include: Memory, used to store computer programs; A processor, for implementing any of the methods described in claims 1-10 when executing a program stored in a memory.

14. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 10 is implemented.

Images