A method and apparatus for constructing a map

Abstract

The specification discloses a method and device for constructing a map. An unmanned device acquires target images, extracts target points from the target images, and divides the target points to obtain target point sets required for multiple rounds of optimization. For each round of optimization in the multiple rounds of optimization, a target point set optimized in the previous round of optimization and a target point set newly added for optimization in the round of optimization are determined from the target point sets, a target point set to be optimized in the round of optimization is constructed, weights of target points included in the target point set to be optimized in the round of optimization are determined based on a pose for observing the target point set optimized in the previous round of optimization, the target point set to be optimized in the round of optimization is optimized based on the weights to obtain a target point set optimized in the round of optimization. After the multiple rounds of optimization, if a preset condition is met, the multiple rounds of optimization are stopped, and a map is constructed based on the target point set optimized.

Description

TECHNICAL FIELD

[0001] The present specification relates to the technical field of unmanned driving, and in particular, to a method and device for constructing a map. BACKGROUND

[0002] In an automatic driving environment, a map of an automatic driving vehicle needs to be constructed so that the automatic driving vehicle can make reasonable driving decisions.

[0003] At present, the main way to construct a map of an automatic driving vehicle is as follows: based on multiple pictures taken by a camera, feature points in the multiple pictures are extracted, the multiple feature points are matched one by one according to actual correspondence, a depth value of a feature point is obtained from information of the same feature point contained in the multiple pictures, and the feature point is arranged in a three-dimensional space, and based on this, multiple feature points in the three-dimensional space are obtained, and a three-dimensional environment in which the automatic driving vehicle travels is constructed. SUMMARY

[0004] The present specification provides a method and device for constructing a map to partially solve the above problems existing in the prior art.

[0005] The present specification adopts the following technical solutions:

[0006] The present specification provides a method for constructing a map, comprising:

[0007] obtaining each target image;

[0008] extracting each target point from the target images;

[0009] dividing each target point to obtain each target point set required for multiple rounds of optimization;

[0010] for each round of optimization in the multiple rounds of optimization, constructing a to-be-optimized target point set corresponding to the round of optimization based on an optimized target point set corresponding to a previous round of optimization and a target point set determined from the target point sets and required to be newly added for optimization in the round of optimization, determining weights of each target point included in the to-be-optimized target point set corresponding to the round of optimization based on a determined pose for observing the optimized target point set corresponding to the previous round of optimization, and optimizing the to-be-optimized target point set corresponding to the round of optimization based on the weights to obtain an optimized target point set corresponding to the round of optimization;

[0011] constructing a map based on the optimized target point set obtained after the multiple rounds of optimization meet a preset condition.

[0012] Optionally, dividing each target point to obtain each target point set required for multiple rounds of optimization specifically comprises:

[0013] For each target point, according to the target image related to the target point, the error corresponding to the target point is determined;

[0014] According to the error corresponding to each target point determined, the target points are divided to obtain the target point set required for multi-round optimization.

[0015] Optionally, according to the error corresponding to each target point determined, the target points are divided to obtain the target point set required for multi-round optimization, specifically including:

[0016] The target points with the error greater than a set error threshold are removed from the target points to obtain the remaining target points;

[0017] According to the error corresponding to each remaining target point, the remaining target points are divided to obtain the target point set required for multi-round optimization.

[0018] Optionally, for each target point, according to the target image related to the target point, the error corresponding to the target point is determined, specifically including:

[0019] For each target point, according to the target image related to the target point, the error corresponding to the target point is determined, specifically including:

[0020] According to the depth value of the target point, the target point is projected to a preset three-dimensional space to obtain the space point corresponding to the target point in the three-dimensional space;

[0021] The space point corresponding to the target point in the three-dimensional space is projected into the target image related to the target point to obtain the projection point of the target point;

[0022] According to the deviation between the projection point of the target point and the image position of the target point, the error of the target point is determined.

[0023] Optionally, according to the error corresponding to each target point determined, the target points are divided to obtain the target point set required for multi-round optimization, specifically including:

[0024] According to the error corresponding to each target point determined, the target points are divided to obtain the target point set;

[0025] According to the error corresponding to each target point included in each target point set from small to large, the target point sets are sorted to obtain a sorting result;

[0026] According to the sorting result, the target point set required to be newly added for optimization in each round of optimization is determined, wherein for each round of optimization, the greater the round number corresponding to the round of optimization, the greater the error corresponding to each target point included in the target point set required to be newly added for optimization in the round of optimization.

[0027] Optionally, according to the determined pose based on which the optimized target point set corresponding to the previous round of optimization is observed, the weight of each target point included in the target point set to be optimized corresponding to the current round of optimization is determined, specifically including:

[0028] According to the determined pose based on which the optimized target point set corresponding to the previous round of optimization is observed, and the weight of the optimized target point set corresponding to the previous round of optimization, the weight of each target point included in the target point set to be optimized corresponding to the current round of optimization is determined.

[0029] Optionally, according to the determined pose based on which the optimized target point set corresponding to the previous round of optimization is observed, and the weight of the optimized target point set corresponding to the previous round of optimization, the weight of each target point included in the target point set to be optimized corresponding to the current round of optimization is determined, specifically including:

[0030] For each target point included in the target point set newly added for optimization in the current round of optimization, a target point matched with the target point in the optimized target point set corresponding to the previous round of optimization is determined as the associated target point corresponding to the target point;

[0031] According to the determined pose based on which the optimized target point set corresponding to the previous round of optimization is observed, the spatial point corresponding to the associated target point in the preset three-dimensional space is projected into the target image related to the target point, so as to determine the error of the target point based on the associated target point;

[0032] According to the weight of the optimized target point set corresponding to the previous round of optimization, the weight corresponding to the target point is determined;

[0033] According to the weight of the optimized target point set corresponding to the previous round of optimization, and the weight of each target point included in the target point set newly added for optimization in the current round of optimization, the weight of each target point included in the target point set to be optimized corresponding to the current round of optimization is determined.

[0034] The present specification provides a device for constructing a map, comprising:

[0035] An acquisition module is configured to acquire target images;

[0036] An extraction module is configured to extract target points from the target images;

[0037] A division module is configured to divide the target points to obtain target point sets;

[0038] The optimization module constructs, for each round of optimization in the multiple rounds of optimization, a to-be-optimized target point set corresponding to the round of optimization according to the optimized target point set corresponding to the previous round of optimization and a target point set newly added for optimization in the round of optimization determined from the target point sets, determines weights of each target point included in the to-be-optimized target point set corresponding to the round of optimization according to a determined pose for observing the optimized target point set corresponding to the previous round of optimization, and performs optimization on the to-be-optimized target point set corresponding to the round of optimization according to the weights to obtain an optimized target point set corresponding to the round of optimization.

[0039] The construction module is configured to construct a map according to the optimized target point set obtained after the multiple rounds of optimization are stopped when the preset condition is met.

[0040] The present specification provides a computer-readable storage medium, which stores a computer program. The computer program is executed by a processor to implement the method for constructing a map.

[0041] The present specification provides an electronic device, which includes a memory, a processor, and a computer program stored in the memory and executable on the processor. The processor implements the method for constructing a map when executing the program.

[0042] The above at least one technical solution adopted by the present specification can achieve the following beneficial effects:

[0043] In the method for constructing a map provided by the present specification, an unmanned device obtains each target image, extracts each target point from each target image, and divides each target point to obtain each target point set required for multiple rounds of optimization. For each round of optimization in the multiple rounds of optimization, a to-be-optimized target point set corresponding to the round of optimization is constructed according to an optimized target point set corresponding to the previous round of optimization and a target point set newly added for optimization in the round of optimization determined from the target point sets. Weights of each target point included in the to-be-optimized target point set corresponding to the round of optimization are determined according to a determined pose for observing the optimized target point set corresponding to the previous round of optimization. The to-be-optimized target point set corresponding to the round of optimization is optimized according to the weights to obtain an optimized target point set corresponding to the round of optimization. After multiple rounds of optimization, the multiple rounds are stopped when a preset condition is met, and a map is constructed according to the obtained optimized target point set.

[0044] As can be seen from the above method, for the target points with certain errors, in the process of constructing the map, instead of directly eliminating these target points, the weight determined can be used to constantly optimize these target points, so that these target points can be used to construct the map, thereby increasing the number of target points involved in constructing the map, and improving the accuracy of the constructed map. Meanwhile, in the multiple rounds of optimization, the weight of each target point set obtained is used to constantly adjust the observation pose of the target image observing each target point, so that the pose obtained is more accurate than the pose directly calculated in the prior art, thereby reducing the error generated by target point matching. BRIEF DESCRIPTION OF DRAWINGS

[0045] The drawings described herein are used to provide further understanding of the present specification, and form a part of the present specification. The illustrative embodiments of the present specification and their descriptions serve to explain the present specification, and do not constitute an improper limitation on the present specification. In the drawings:

[0046] Figure 1 A flowchart of a method for constructing a map provided in the present specification;

[0047] Figure 2 A flowchart of a specific embodiment of a method for constructing a map provided in the present specification;

[0048] Figure 3 A schematic diagram of a control device of an unmanned device provided in the present specification;

[0049] Figure 4 A schematic diagram of an electronic device corresponding to Figure 1 provided in the present specification. DETAILED DESCRIPTION

[0050] In order to make the purpose, technical solutions and advantages of the present specification clearer, the technical solutions of the present specification will be described clearly and completely in combination with the specific embodiments of the present specification and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present specification, not all the embodiments. Based on the embodiments in the present specification, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present specification.

[0051] In the current mainstream way of building a map, due to the error of the feature points is inevitable, only by pre-set filtering rules to filter out the target point with larger error, and in the setting of the filtering rule is too strict, will lead to the number of feature points obtained finally is less, and in the setting of the filtering rule is too loose, will lead to the final construction of the map is introduced too much error, therefore need to spend a lot of resources to select a suitable filtering rule to ensure the accuracy of the final construction of the map. At the same time, in the process of matching the target point, due to the different environment when shooting, it is easy to cause the same point in reality in different pictures can not be corresponding, leading to the constructed driving map precision is low.

[0052] Therefore, how to ensure the construction of the driving map, reduce the error generated by the matching of the feature points, increase the number of feature points, so as to improve the precision of the constructed driving map, is a problem to be solved.

[0053] In order to solve the above-mentioned problems, the present specification provides a method for building a map, and the technical solutions provided by the embodiments of the present specification are described in detail below with reference to the accompanying drawings.

[0054] Figure 1 The flowchart of the method for building a map provided in the present specification includes the following steps:

[0055] S101: Obtain each target image.

[0056] In the present specification, the execution body of the method for building a map can be a terminal device such as a computer, a server, etc. set on a platform, or can be unmanned equipment such as a drone, an unmanned vehicle, an automatic driving robot, etc. For the sake of description, the method for building a map provided in the present specification is described below taking the server as an example.

[0057] At present, in the field of unmanned driving, the way to build a map (high-precision map) of unmanned equipment is often to obtain all the target images containing the map to be built, extract all the feature points from all the target images, eliminate the feature points with larger error, and use the remaining feature points to build a map.

[0058] However, the number of feature points obtained according to this way is often small, which cannot meet the demand of building a map, so that the final constructed map is not accurate enough.

[0059] Based on this, the specification provides a method for constructing a map. The server first extracts all target points from all target images, divides all target points into multiple target point sets, divides the optimization process into multiple rounds of optimization, optimizes one target point set in each round of optimization, and determines the weights corresponding to each target point included in the target point set of the next round of optimization according to the bit data based on the optimized target point set obtained after each round of optimization. In the case where the preset condition is met, the optimization of the target points is completed.

[0060] The method for constructing a map provided by the specification can reasonably utilize the extracted target points, increase the number of target points that can be used when constructing a map, and thus improve the accuracy of the constructed map.

[0061] It should be noted that the target points mentioned in the specification can be two-dimensional target points or three-dimensional target points.

[0062] In the method for constructing a map provided by the specification, the server first needs to obtain each target image, wherein the target image is an image containing at least part of the area of the map that needs to be constructed by the unmanned device, i.e., the image containing the to-be-constructed area is taken as the target image.

[0063] The server can obtain the target image according to the screening condition of the target image. For example, if the image contains key information in the pre-constructed map, such as the image including the lane line in the pre-constructed map, the image can be taken as the target image. For another example, the server can take an image whose number of extracted target points exceeds a set number threshold as the target image. Other screening conditions will not be described in detail here.

[0064] The target image can be obtained by an unmanned device, a manned driving device, a camera, etc. The unmanned device using the map constructed by the method provided by the specification can be used to perform delivery tasks in the delivery field, such as using the unmanned device to perform express delivery, logistics, take-out, etc.

[0065] S102: Extract each target point from each target image.

[0066] After the server obtains the target image, each target point is extracted from all target images. There are many kinds of target points. For example, if the lane line is involved in the construction of the map, the target point can be a plurality of points at the boundary of the lane line, so that the server can construct the lane line in the map according to these target points. For another example, if the surrounding buildings are involved in the construction of the map, the target points can be a plurality of points at the boundary of the buildings, so that the server can construct the buildings in the map according to these target points.

[0067] Since the target images are often obtained from different observation angles to obtain information of the pre-constructed map, the server needs to match the same target points in different target images one by one when extracting the target points, that is, the same target points recorded in different target images have a matching relationship.

[0068] The matching relationship between the target points can be that if any two target points in different target images are the same point in the three-dimensional space, it is considered that the two target points have a matching relationship.

[0069] In this specification, the matching method of the target points can adopt the existing conventional method, for example, using the optical flow method to determine whether two target points have a matching relationship, that is, if the brightness of the same target point in two consecutive frames is constant and meets the motion trend of the target point in the three-dimensional space, it is determined that the two target points in the two frames have a matching relationship. This specification does not limit the specific matching method.

[0070] S103: Divide the target points to obtain the target point set required for multiple rounds of optimization.

[0071] After the server obtains all the target points, the target points can be divided into target point sets required for multiple rounds of optimization according to the preset condition.

[0072] The preset condition can be that for each target point, the error corresponding to the target point is determined according to the target image involving the target point. After determining the error corresponding to the target point, the target points are divided according to the determined error corresponding to each target point to obtain the target point set required for multiple rounds of optimization.

[0073] Since there are some target points with too large errors, if these target points with too large errors are used as the target points used for final mapping, it will affect the accuracy of the finally constructed map. Therefore, after dividing the target points, the target points with errors greater than the set error threshold are removed from the target points to obtain the remaining target points. In order to ensure the number of target points that can be used for optimization, the error threshold can be set according to the actual situation to filter out some target points with too large errors.

[0074] In the specification, the method for determining the error of the target point can be: for each target point, the server determines the depth value of the target point according to the target image related to the target point. After obtaining the depth value of the target point, the target point is projected into the preset three-dimensional space according to the depth value of the target point, to obtain the corresponding space point of the target point in the three-dimensional space. The method for determining the depth value of the target point can be: selecting at least two target images containing different poses of the target point, and determining the depth value of the target point from the poses of the at least two target images and the image position of the target point in the target image.

[0075] The server projects the corresponding space point of the target point in the three-dimensional space into the target image related to the target point, to obtain the projection point of the target point, and determines the error of the target point according to the deviation between the projection point of the target point and the image position of the target point.

[0076] The projection point of the target point mentioned above is that the server determines the deviation of the target point contained in the target image by the distance between the projection point of the target point and the position of the target point in the target image, to obtain the projection point of the target point by projecting the space point into the target image containing the target point, and thereby determines the error of the target point. The greater the deviation between the projection point of the target point and the image position of the target point, the greater the error of the target point.

[0077] Of course, there are many ways to determine the error of the target point, for example, for each target point, the plane containing the target point in the target image and the target point in the three-dimensional space can be determined as the epipolar plane of the target point by the target point, the corresponding pose of the target image containing the target point, and the position of the target point in the three-dimensional space. Then, the intersection line of the other target images containing the target point and the epipolar plane is taken as the epipolar line of the target point, the deviation of the target point is obtained by determining the distance between the target point and the epipolar line in the other target images, and the error of the target point is determined by the deviation of the target point. Of course, there are many ways to determine the error of the target point, which will not be illustrated one by one here.

[0078] After the server determines the target point sets required for multiple rounds of optimization according to the errors, the target point sets can be sorted according to the errors corresponding to the target points contained in the target point sets from small to large to obtain a sorting result. Then, the server can determine the target point set that needs to be newly added for optimization in each round of optimization according to the sorting result. For each round of optimization, the greater the round number corresponding to the round of optimization, the greater the error corresponding to the target points contained in the target point set that needs to be newly added for optimization in the round of optimization.

[0079] For example, all target points are divided into three target point sets, an A target point set, a B target point set, and a C target point set, wherein the error of the target points included in the A target point set is less than the error of the target points included in the B target point set, and the error of the target points included in the B target point set is less than the error of the target points included in the C target point set. The target point set added in the first round of optimization is the A target point set, the target point set newly added in the second round of optimization is the B target point set, and the target point set newly added in the third round of optimization is the C target point set.

[0080] The reason for setting the relationship between the above rounds and the error is that the pose and other data of each target image after the last round of optimization need to be used in the next round of optimization. Therefore, the target points with smaller optimization errors are optimized first, and the pose and other data obtained from these target points are more accurate, thereby improving the accuracy of the target points included in the subsequent optimization process. In this specification, the pose refers to the pose of the collection device (such as an unmanned device, a manned device, etc.) when the target image is captured.

[0081] S104: For each round of optimization in the multiple rounds of optimization, the server constructs a to-be-optimized target point set corresponding to the round of optimization by combining the optimized target point set corresponding to the last round of optimization and the target point set newly added in the round of optimization. The server determines the weight of each target point included in the to-be-optimized target point set corresponding to the round of optimization according to the determined pose for observing the optimized target point set corresponding to the last round of optimization, and optimizes the to-be-optimized target point set corresponding to the round of optimization according to the weight to obtain the optimized target point set corresponding to the round of optimization.

[0082] After the server obtains each target point set and the optimization order of each target point set, the server can perform multiple rounds of optimization on the target points.

[0083] For each round of optimization in the multiple rounds of optimization, the server constructs a to-be-optimized target point set corresponding to the round of optimization by combining the optimized target point set corresponding to the last round of optimization and the target point set newly added in the round of optimization. That is, the to-be-optimized target point set involved in this type of optimization is obtained by combining the target point set that has been optimized in the last round and the target point set newly added in the current round.

[0084] The server determines the weight of each target point included in the to-be-optimized target point set corresponding to the round of optimization according to the determined pose for observing the optimized target point set corresponding to the last round of optimization, and optimizes the to-be-optimized target point set corresponding to the round of optimization according to the weight to obtain the optimized target point set corresponding to the round of optimization. That is, the server can determine the pose of the target point set newly added in the current round of optimization according to the pose obtained after the last round of optimization.

[0085] For example, the target image A contains two target points a and b. Since the error of the target point a is smaller, a is one of the target points in the target point set to be optimized in the first round of optimization. After the first round of optimization, the pose of the target image A can be obtained. Since the error of the target point b is larger, b is one of the target points newly added to the target point set for optimization in the second round of optimization. In the second round of optimization, the pose of the target image A can be used as the pose of the target point b.

[0086] In this specification, the server can determine the weight of each target point included in the target point set to be optimized in the current round of optimization according to the determined pose based on which the target point set after the last round of optimization is observed and the weight of the target point set after the last round of optimization.

[0087] Specifically, the server can determine, for each target point included in the target point set newly added to the optimization in the current round of optimization, a target point matching the target point in the target point set after the last round of optimization as the associated target point corresponding to the target point.

[0088] The server can project the spatial point corresponding to the associated target point in the preset three-dimensional space to the target image involved in the target point according to the determined pose based on which the target point set after the last round of optimization is observed, to determine the error of the target point based on the associated target point. And determine the weight corresponding to the target point according to the weight of the target point set after the last round of optimization.

[0089] The server can determine the weight of each target point included in the target point set to be optimized in the current round of optimization according to the weight of the target point set after the last round of optimization and the weight of each target point included in the target point set newly added to the optimization in the current round of optimization.

[0090] That is, the server determines the error of each target point included in the target point set newly added to the optimization in the current round of optimization according to the weight of each target point included in the target point set after the last round of optimization, and determines the weight of the target point set included in the target point set newly added to the optimization in the current round of optimization. According to the obtained weight of each target point included in the target point set after the last round of optimization and the weight of each target point included in the target point set newly added in the current round of optimization, the weight of each target point included in the target point set to be optimized is determined, and each target point in the target point set to be optimized is optimized according to the weight of each target point included in the target point set.

[0091] The target point set to be optimized is the set of the target point set after the last round of optimization and the target point set newly added to the optimization in the current round of optimization.

[0092] It should be noted that, for the first round of optimization, since the target point set to be optimized involved in the first round of optimization does not have weights for each target point, a same weight can be assigned to each target point in the target point set to be optimized involved in the first round of optimization, and then, after determining the corresponding spatial points of each target point in the preset three-dimensional space according to the poses of each target point, projecting into the target image involved by each target point, the error of each target point is obtained. According to the error of each target point obtained, the corresponding weight of each target point is determined.

[0093] S105: Constructing a map according to the optimized target point set obtained after stopping the multiple rounds of optimization that meet the preset condition.

[0094] The server can end the optimization process according to the preset condition. For example, if it is determined that the optimization of all target point sets will be completed, the optimization process is ended. For another example, if the number of target points obtained after optimization meets the required number of target points for constructing a map, it can be determined that the optimization is ended. For this way, the target points finally determined by the server for constructing a map are not necessarily all the target points extracted from the target image, that is, if it is determined that the number of target points required for constructing a map has met the requirement after the Nth round of optimization, the remaining target points can not be optimized.

[0095] After the server ends the optimization, it can project each target point in the optimized target point set into a three-dimensional space according to the obtained optimized target point set, and construct a map.

[0096] The following will be combined Figure 2 A flowchart of a specific embodiment of constructing a map provided in the present specification is provided to describe the method provided in the present specification.

[0097] The server can extract each target point set from the obtained target image after obtaining the target image. According to the error confirmation method mentioned above, each target point set is divided into multiple groups of target point sets.

[0098] After the server obtains multiple groups of target point sets, it can preferentially optimize the target point set with smaller error. For the first round of optimization, the weight of all target points in the first group of target point sets is set to 1, and the weight of each target point set in the first group of target point sets is obtained as the weight of the target point set to be optimized according to the weight acquisition method mentioned above. And determine the pose of each target point in the first group of target point sets, and use the bundle adjustment method (BA, Bundle-Adjustment), which can complete the optimization of each target point in the target point set according to the pose of each target point in the target point set and the corresponding weight of each target point. Obtain the optimized target point set and the corresponding weight of each target point in the target point set.

[0099] For each round of optimization, the above-mentioned method for multi-round optimization target point set is used for optimization. After each round of optimization, it is judged whether the preset condition that all target point sets are completed is met. If it is judged that the preset condition is not met, the pose corresponding to all target points obtained in this round is added to the next round of optimization as the basis data. If the preset condition is met, the optimization is ended, and the map is constructed by using the obtained optimized target points.

[0100] As can be seen from the above method, for the target points with certain errors, instead of discarding these target points, the weights are reasonably set so that these target points can be used to construct the map, thereby increasing the number of target points that can be used when constructing the map and improving the accuracy of the constructed map. By using the weights of the target points, the observation poses of the target image for observing the target points are continuously adjusted. The obtained poses are more accurate than the poses directly calculated in the prior art, thereby reducing the error caused by target point matching.

[0101] Figure 3 A schematic diagram of a control device of an unmanned device is provided in the present specification, comprising:

[0102] The acquisition module 301 is configured to acquire target images.

[0103] The extraction module 302 is configured to extract target points from the target images.

[0104] The division module 303 is configured to divide the target points to obtain target point sets.

[0105] The optimization module 304 is configured to, for each round of optimization in multi-round optimization, construct a target point set to be optimized corresponding to the round of optimization by using the target point set optimized in the last round of optimization and a target point set determined from the target point sets and required to be newly added to the optimization in the round of optimization, determine weights of the target points included in the target point set to be optimized corresponding to the round of optimization based on the poses for observing the target point set optimized in the last round of optimization, and optimize the target point set to be optimized corresponding to the round of optimization according to the weights to obtain a target point set optimized in the round of optimization.

[0106] The construction module 305 is configured to construct a map according to the target point set optimized after the multi-round optimization is stopped.

[0107] Optionally, the division module 303 is specifically configured to, for each target point, determine an error corresponding to the target point according to the target image involving the target point, and divide the target points according to the determined error corresponding to each target point to obtain the target point sets required for multi-round optimization.

[0108] Optionally, the dividing module 303 is specifically configured to: remove target points in the target points corresponding to errors greater than a set error threshold, to obtain remaining target points; divide the remaining target points according to the errors corresponding to each of the remaining target points, to obtain the target point set required for each round of optimization.

[0109] Optionally, the dividing module 303 is specifically configured to: for each target point, determine a depth value of the target point according to a target image involved by the target point; project the target point into a preset three-dimensional space according to the depth value of the target point, to obtain a spatial point corresponding to the target point in the three-dimensional space; project the spatial point corresponding to the target point in the three-dimensional space into the target image involved by the target point, to obtain a projection point of the target point; and determine an error of the target point according to a deviation between the projection point of the target point and an image position of the target point.

[0110] Optionally, the dividing module 303 is specifically configured to: divide the target points according to the errors corresponding to each of the determined target points, to obtain the target point set; sort the target point sets according to the errors corresponding to the target points included in each of the target point sets from small to large, to obtain a sorting result; and determine, according to the sorting result, the target point set that needs to be newly added for optimization in each round of optimization, wherein, for each round of optimization, the greater the round number corresponding to the round of optimization, the greater the errors corresponding to the target points included in the target point set that needs to be newly added for optimization in the round of optimization.

[0111] Optionally, the optimization module 304 is specifically configured to: determine the weights of the target points included in the target point set to be optimized in the round of optimization, according to the determined pose based on which the target point set optimized in the last round of optimization is observed, and the weight of the target point set optimized in the last round of optimization.

[0112] Optionally, the optimization module 304 is specifically configured to: for each target point included in the target point set newly added for optimization in the round of optimization, determine a target point matched with the target point in the target point set optimized in the last round of optimization as an associated target point corresponding to the target point; project a spatial point corresponding to the associated target point in a preset three-dimensional space into a target image involved by the target point, according to the determined pose based on which the target point set optimized in the last round of optimization is observed, to determine an error of the target point based on the associated target point; determine a weight corresponding to the target point according to the weight of the target point set optimized in the last round of optimization; and determine the weights of the target points included in the target point set to be optimized in the round of optimization, according to the weight of the target point set optimized in the last round of optimization, and the weight of each target point included in the target point set newly added for optimization in the round of optimization.

[0113] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 This provides a method for building maps.

[0114] This instruction manual also provides Figure 4 One of the corresponding Figure 1 A schematic diagram of the structure of an electronic device. (e.g.) Figure 4 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 The method for constructing the map is described above. Of course, in addition to software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. That is to say, the execution subject of the following processing flow is not limited to each logic unit, but can also be hardware or logic devices.

[0115] In the 1990s, it was quite obvious to distinguish whether an improvement in a technology was in hardware (e.g., improvement in circuit structures of diodes, transistors, switches, etc.) or in software (improvement in method flow). However, as technology has evolved, many improvements in method flow today can be considered as direct improvements in hardware circuit structures. Designers almost always obtain the corresponding hardware circuit structures by programming the improved method flow into hardware circuits. Therefore, it cannot be said that an improvement in a method flow cannot be implemented by hardware entity modules. For example, a programmable logic device (PLD) (e.g., a field programmable gate array (FPGA)) is an integrated circuit whose logic function is determined by user programming of the device. A digital system is "integrated" on a PLD by the designer programming it, rather than by asking a chip manufacturer to design and fabricate a custom integrated circuit chip. Moreover, instead of manually fabricating integrated circuit chips, this programming is now mostly implemented by "logic compiler" software, which is similar to software compilers used in program development, and the original code to be compiled is written in a specific programming language, which is called a hardware description language (HDL), and there are many such languages, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, RHDL (Ruby Hardware Description Language), etc., and the most commonly used are VHDL (Very-High-Speed Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should be aware that, as long as the method flow is logically programmed in the above-mentioned hardware description languages and programmed into an integrated circuit, a hardware circuit implementing the logical method flow can be easily obtained.

[0116] The controller can be implemented in any suitable way, for example, the controller can take the form of a microprocessor or processor and a computer readable medium storing computer readable program code, such as software or firmware, executable by the (micro)processor, logic gates, switches, an application specific integrated circuit (ASIC), a programmable logic controller and an embedded microcontroller, examples of which include but are not limited to the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20 and Silicone Labs C8051F320, the memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also know that, in addition to being implemented in pure computer readable program code, the controller can equally well be implemented to perform the same functions using logic gates, switches, an application specific integrated circuit, a programmable logic controller and an embedded microcontroller, etc. by means of a logical programming of the method steps. The controller can thus be considered as a hardware component, and the means comprised therein for performing the various functions can be considered as structures within the hardware component. Alternatively, the means for performing the various functions can even be considered as both a software module implementing the method and a structure within the hardware component.

[0117] The systems, apparatuses, modules or units illustrated by the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, the computer can be a personal computer, a laptop computer, a cellular phone, a camera phone, a smart phone, a personal digital assistant, a media player, a navigation device, an email device, a game console, a tablet computer, a wearable device, or a combination of any of these devices.

[0118] For the sake of description, the above apparatuses are described in various units by functions respectively. Of course, the functions of each unit can be implemented in the same or multiple software and / or hardware in implementing the present specification.

[0119] Those skilled in the art will understand that the embodiments of the present specification can be provided as a method, a system, or a computer program product. Therefore, the present specification can take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0120] The specification is presented with reference to flow diagrams and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the specification. It will be understood that each block of the flow diagrams and / or block diagrams, and combinations of blocks in the flow diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general purpose computer, special purpose computer, embedded processing element or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flow diagrams and / or block diagrams block or blocks. Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include one or more other steps, operations, functions or combinations of steps, operations, or functions in Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include one or more other steps, operations, functions or combinations of steps, operations, or functions in The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include one or more other steps, operations, functions or combinations of steps, operations, or functions in

[0121] The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include one or more other steps, operations, functions or combinations of steps, operations, or functions in Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include one or more other steps, operations, functions or combinations of steps, operations, or functions in Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include one or more other steps, operations, functions or combinations of steps, operations, or functions in The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include one or more other steps, operations, functions or combinations of steps, operations, or functions in

[0122] The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include one or more other steps, operations, functions or combinations of steps, operations, or functions in Figure 1 The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include one or more other steps, operations, functions or combinations of steps, operations, or functions in ​ The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include one or more other steps, operations, functions or combinations of steps, operations, or functions in The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include one or more other steps, operations, functions or combinations of steps, operations, or functions in

[0123] The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include one or more other steps, operations, functions or combinations of steps, operations, or functions in The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include one or more other steps, operations, functions or combinations of steps, operations, or functions in

[0124] The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include one or more other steps, operations, functions or combinations of steps, operations, or functions in The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include one or more other steps, operations, functions or combinations of steps, operations, or functions in

[0125] The flow diagrams and / or block diagrams in the specification can present a method, apparatus or computer program product according to embodiments of the specification. Although the flow diagrams and / or block diagrams can present a method, apparatus or computer program product in a particular, it is understood that the method, apparatus and computer program product can include one or more additional steps, operations, or functions, and the method, apparatus and computer program product can include one or more other steps, operations, functions or combinations of steps, operations, or functions inComputer-readable media includes permanent and non-permanent, movable and non-movable media that can be implemented by any method or technology to store information. The information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, compact disc read-only memory (CD-ROM), digital versatile disc (DVD) or other optical storage, magnetic cassette, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transmission medium that can be used to store information accessible to a computing device. According to the definition herein, computer-readable media does not include transitory media such as modulated data signals and carriers.

[0126] It should also be noted that the terms "comprising", "containing", or any other variant thereof are intended to cover non-exclusive inclusion, such that a process, method, article or apparatus that comprises a list of elements does not only include those elements, but can also include other elements not expressly listed or inherent to such process, method, article or apparatus. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of additional identical elements in the process, method, article or apparatus that includes the element.

[0127] Those skilled in the art will appreciate that embodiments of the present specification can be provided as methods, systems or computer program products. Therefore, the present specification can take the form of an entirely hardware embodiment, an entirely software embodiment or an embodiment combining software and hardware aspects. Moreover, the present specification can take the form of a computer program product implemented on one or more computer-usable storage media (including, but not limited to, magnetic disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0128] The present specification can be described in the general context of computer-executable instructions, such as program modules, being executed by a computer. Generally, program modules include routines, programs, objects, components, data structures, etc. that perform particular tasks or implement particular abstract data types. The present specification can also be practiced in distributed computing environments where tasks are performed by remote processing devices that are connected through a communication network. In a distributed computing environment, program modules can be located in both local and remote computer storage media including storage devices.

[0129] The various embodiments described in this specification are described using a numbering of embodiments approach: these are each individually integrated contributions pertaining to different aspects of the description. For each embodiment, the description focuses on the differences from the other embodiments. In particular, the description of the system embodiments is relatively brief, as the system embodiments are largely analogous to the method embodiments. The relevant parts of the description of the method embodiments are therefore referred to.

[0130] The above only describes the embodiments of the present specification and is not intended to limit the present specification. The present specification can have various modifications and changes for those skilled in the art. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present specification shall be included in the scope of claims of the present specification.

Claims

1. A method for constructing a map, characterized in that, include: Acquire the images of each target; Extract each target point from each target image; The target points are divided into groups to obtain the set of target points required for multiple rounds of optimization; For each round of optimization, the optimized target point set corresponding to the previous round of optimization, and the target point set that needs to be newly added to the optimization in this round of optimization, are used to construct the target point set to be optimized for this round of optimization. Based on the pose on which the optimized target point set corresponding to the previous round of optimization is based, the weight of each target point in the target point set to be optimized for this round of optimization is determined. Based on the weight, the target point set to be optimized for this round of optimization is optimized to obtain the optimized target point set for this round of optimization. A map is constructed based on the optimized target point set obtained after stopping multiple rounds of optimization when preset conditions are met.

2. The method as described in claim 1, characterized in that, The target points are divided into sets to obtain the target point sets required for multiple rounds of optimization, specifically including: For each target point, the error corresponding to that target point is determined based on the target image involving that target point; Based on the error corresponding to each target point, the target points are divided to obtain the set of target points required for multiple rounds of optimization.

3. The method as described in claim 2, characterized in that, Based on the error corresponding to each determined target point, the target points are divided to obtain the target point set required for multiple rounds of optimization, specifically including: Remove the target points whose errors are greater than a set error threshold from the target points to obtain the remaining target points; Based on the error corresponding to each remaining objective point, the remaining objective points are divided to obtain the set of objective points required for multiple rounds of optimization.

4. The method as described in claim 2, characterized in that, For each target point, based on the target image involving that target point, the corresponding error for that target point is determined, specifically including: For each target point, determine its depth value based on the target image it relates to; Based on the depth value of the target point, the target point is projected onto a preset three-dimensional space to obtain the spatial point corresponding to the target point in the three-dimensional space; Project the spatial point corresponding to the target point in the three-dimensional space onto the target image involving the target point to obtain the projection point of the target point; The error of the target point is determined based on the deviation between the projected point of the target point and the target point's position in the image.

5. The method as described in claim 2, characterized in that, Based on the error corresponding to each determined target point, the target points are divided to obtain the target point set required for multiple rounds of optimization, specifically including: Based on the error corresponding to each target point, the target points are divided to obtain a set of target points; The target point sets are sorted in ascending order of the errors corresponding to each target point, to obtain the sorting result. Based on the sorting results, the target point set that needs to be added to the optimization in each round is determined. For each round of optimization, the larger the round number, the larger the error of each target point in the target point set that needs to be added to the optimization in that round.

6. The method as described in claim 1, characterized in that, Based on the pose of the optimized target point set corresponding to the previous round of optimization, determine the weights of each target point in the target point set to be optimized in the current round of optimization, specifically including: Based on the pose on which the optimized target point set corresponding to the previous round of optimization is based, and the weights of the optimized target point set corresponding to the previous round of optimization, the weights of each target point in the target point set to be optimized in this round of optimization are determined.

7. The method as described in claim 6, characterized in that, Based on the pose on which the optimized target point set corresponding to the previous round of optimization is based, and the weights of the optimized target point set corresponding to the previous round of optimization, the weights of each target point in the target point set to be optimized in this round of optimization are determined, specifically including: For each target point in the newly added target point set in this round of optimization, a target point that matches the target point in the optimized target point set corresponding to the previous round of optimization is determined as the associated target point corresponding to the target point. Based on the pose of the optimized target point set corresponding to the previous round of observation optimization, the spatial point corresponding to the associated target point in the preset three-dimensional space is projected onto the target image involved by the target point to determine the error of the target point based on the associated target point. The weight of the target point is determined based on the weight of the target point set after optimization in the previous round of optimization. Based on the weights of the optimized target point set corresponding to the previous round of optimization, and the weights of each target point in the newly added target point set in this round of optimization, determine the weights of each target point in the target point set to be optimized in this round of optimization.

8. An apparatus for constructing a map, characterized in that, include: The acquisition module is used to acquire images of each target. The extraction module is used to extract each target point from each target image; The partitioning module is used to divide the target points into sets of target points. The optimization module, for each round of optimization, constructs a target point set to be optimized for that round by taking the optimized target point set corresponding to the previous round of optimization and the target point set that needs to be newly added to the optimization in this round, determined from the target point sets. Based on the pose on which the optimized target point set corresponding to the previous round of optimization is based, the module determines the weight of each target point included in the target point set to be optimized for that round of optimization. Based on the weight, the module optimizes the target point set to be optimized for that round of optimization to obtain the optimized target point set for that round of optimization. The construction module is used to build a map based on the optimized target point set obtained after stopping multiple rounds of optimization when preset conditions are met.

9. A computer-readable storage medium, characterized in that, The storage medium stores a computer program, which, when executed by a processor, implements the method described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that, When the processor executes the program, it implements the method described in any one of claims 1 to 7.

Images