Method, device, equipment and storage medium for constructing a map

Abstract

The present application discloses a method, device, equipment and storage medium for constructing a map, and belongs to the field of map processing technology. The method comprises: obtaining a first image set, wherein the first image set includes multiple images of the area to be mapped; marking points of interest in the first image set; performing three-dimensional reconstruction based on the first image set after marking the points of interest, and obtaining a first point cloud structure according to the reconstruction result; obtaining a first map of the area based on the first point cloud structure; and obtaining a second map with the points of interest marked based on the first map. When constructing a map, the present application only needs to collect images and does not rely on a base map. Even if the latest base map is missing, the latest map can be accurately constructed. And because the points of interest of the image are marked before reconstruction, when marking the points of interest on the map later, the map construction can be completed by projecting the points of interest on the map. The cost of marking the points of interest is low and the efficiency is high.

Description

Technical Field

[0001] The embodiments of the present application relate to the field of map processing technology, and in particular to a method, device, equipment and storage medium for constructing a map. Background Art

[0002] With the development of technology, building structures and distribution are becoming more and more complex, and people often have trouble finding their destinations. In order to help people find their destinations accurately and quickly even if they are not familiar with the environment, it is necessary to build maps and use them to guide users.

[0003] In the related art, a commonly used method for constructing a map is to use an RTK (Real time kinematic) network model to preliminarily construct a map, or to obtain a building base map, preliminarily construct a map based on the building base map, and then manually mark POI (Point of Interest) locations on the preliminarily constructed map to complete the map construction.

[0004] Among the above methods, the method of constructing a map using RTK network model technology has a high operating cost for the RTK network model itself; and the method of constructing a map based on a building base map has a low accuracy rate due to the poor timeliness of the building base map, and manually marking POIs on the initially constructed map is costly and inefficient. Summary of the invention

[0005] The embodiments of the present application provide a method, device, equipment and storage medium for constructing a map, which can be used to solve the problems in the related art. The technical solution is as follows:

[0006] In one aspect, an embodiment of the present application provides a method for constructing a map, the method comprising:

[0007] Acquire a first image set, wherein the first image set includes a plurality of images of an area to be mapped;

[0008] marking points of interest in the first image set;

[0009] Performing three-dimensional reconstruction based on the first image set after the interest points are marked, and obtaining a first point cloud structure according to the reconstruction result;

[0010] Acquire a first map of the area based on the first point cloud structure;

[0011] A second map with points of interest marked is obtained based on the first map.

[0012] In a possible implementation, performing three-dimensional reconstruction based on the first image set after the interest points are marked, and obtaining a first point cloud structure according to the reconstruction result includes:

[0013] Extracting image features of each image in the first image set after the marked interest point;

[0014] Performing image feature pairing based on the image features of each image to obtain an image pair;

[0015] Perform three-dimensional reconstruction based on the image pair, and obtain a first point cloud structure according to the reconstruction result.

[0016] In a possible implementation, performing three-dimensional reconstruction based on the image pair and acquiring a first point cloud structure according to the reconstruction result includes:

[0017] Acquire a positional relationship between a single image in the first image set after the marked interest point and the image pair, wherein the positional relationship is embodied in the form of coordinate points in the same coordinate system;

[0018] Acquire a second point cloud structure based on the positional relationship;

[0019] Iteratively optimizing the second point cloud structure to obtain a third point cloud structure with a controllable error range;

[0020] filtering the point cloud reconstructed based on the wrong image feature pairing in the third point cloud structure;

[0021] In response to satisfying an end condition, the filtered third point cloud structure is used as the first point cloud structure, and the end condition is obtaining the positional relationship between all images in the first image set after the annotated interest point and the image pair.

[0022] In a possible implementation manner, obtaining the positional relationship between the single image in the first image set after the annotated interest point and the image pair includes:

[0023] Obtain a three-dimensional point cloud structure based on the initialized image pair;

[0024] Iteratively optimizing the three-dimensional point cloud structure;

[0025] The point cloud reconstructed based on the wrong image feature pairing in the 3D point cloud structure after filtering optimization;

[0026] The preprocessed image pairs are incrementally registered based on the images in the first image set after the marked interest points, and the positional relationship between the images in the first image set after the marked interest points and the filtered and optimized three-dimensional point cloud structure is obtained based on the filtered and optimized three-dimensional point cloud structure.

[0027] In a possible implementation manner, acquiring a first map of the area based on the first point cloud structure includes:

[0028] Acquire a first plane where the image acquisition device is located based on a posture sequence of the image acquisition device, wherein the posture sequence is obtained in a process of three-dimensional reconstruction based on the first image set after the interest point is marked;

[0029] The first point cloud structure and the posture sequence of the image acquisition device are projected onto the first plane, and the first map is acquired according to the projection result.

[0030] In a possible implementation, after acquiring the second map on which points of interest are marked based on the first map, the method further includes:

[0031] Rebuild the map when the region structure changes.

[0032] In a possible implementation, after acquiring the second map on which points of interest are marked based on the first map, the method further includes:

[0033] When the interest point changes and the regional structure does not change, a second image set is collected, and the interest point in the second image set is marked, wherein the second image set includes images of the interest point change area and images of the interest point unchanged area, and the interest point unchanged area is connected to the interest point change area;

[0034] Performing three-dimensional reconstruction based on the second image set after the interest points are marked and the first point cloud structure, and acquiring a fourth point cloud structure according to the reconstruction result;

[0035] The map is updated based on the fourth point cloud structure.

[0036] In a possible implementation manner, updating the map based on the fourth point cloud structure includes:

[0037] Deleting a replacement candidate frame in the fourth point cloud structure, wherein the replacement candidate frame is determined based on a threshold;

[0038] Projecting the fourth point cloud structure after deleting the replacement candidate frame onto the first plane, and acquiring a third map based on the projection result;

[0039] A fourth map is obtained based on the third map, where the fourth map is a map with points of interest marked.

[0040] In another aspect, a device for constructing a map is provided, the device comprising:

[0041] A first acquisition module, configured to acquire a first image set, wherein the first image set includes a plurality of images of an area to be mapped;

[0042] a marking module, used for marking points of interest in the first image set;

[0043] A first reconstruction module is used to perform three-dimensional reconstruction based on the first image set after the interest points are marked, and obtain a first point cloud structure according to the reconstruction result;

[0044] A second acquisition module, configured to acquire a first map of the area based on the first point cloud structure;

[0045] The third acquisition module is used to acquire a second map with points of interest marked based on the first map.

[0046] In a possible implementation, the first reconstruction module includes:

[0047] An extraction unit, used to extract image features of each image in the first image set after the marked interest point;

[0048] A feature pairing unit, configured to perform image feature pairing based on the image features of each image to obtain an image pair;

[0049] The three-dimensional reconstruction unit is used to perform three-dimensional reconstruction based on the image pair and obtain a first point cloud structure according to the reconstruction result.

[0050] In a possible implementation, the 3D reconstruction unit includes:

[0051] A position relationship acquisition subunit, used to acquire the position relationship between the single image in the first image set after the marked interest point and the image pair, wherein the position relationship is embodied in the form of coordinate points in the same coordinate system;

[0052] A point cloud structure acquisition subunit, configured to acquire a second point cloud structure based on the positional relationship;

[0053] an optimization subunit, configured to iteratively optimize the second point cloud structure to obtain a third point cloud structure with a controllable error range;

[0054] A filtering subunit, configured to filter the point cloud reconstructed based on erroneous image feature pairing in the third point cloud structure;

[0055] The determination subunit is used to use the filtered third point cloud structure as the first point cloud structure in response to meeting an end condition, wherein the end condition is to obtain the positional relationship between all images in the first image set after the annotated interest point and the image pair.

[0056] In a possible implementation, the position relationship acquisition subunit is used to acquire a three-dimensional point cloud structure based on an initialized image pair; iteratively optimize the three-dimensional point cloud structure; filter the point cloud reconstructed based on erroneous image feature pairing in the optimized three-dimensional point cloud structure; perform incremental registration on the preprocessed image pair based on the image in the first image set after the annotated interest point, and acquire the position relationship between the image in the first image set after the annotated interest point and the filtered and optimized three-dimensional point cloud structure based on the filtered and optimized three-dimensional point cloud structure.

[0057] In a possible implementation, the second acquisition module is used to acquire the first plane where the image acquisition device is located based on the posture sequence of the image acquisition device, and the posture sequence is obtained during the three-dimensional reconstruction of the first image set after the marked interest points; project the first point cloud structure and the posture sequence of the image acquisition device to the first plane, and acquire the first map according to the projection result.

[0058] In a possible implementation manner, the device further includes:

[0059] The construction module is used to rebuild the map when the regional structure changes.

[0060] In a possible implementation manner, the device further includes:

[0061] A collection module, configured to collect a second image set and mark the interest points in the second image set when the interest point changes but the regional structure does not change, wherein the second image set includes images of the interest point change region and images of the interest point unchanged region, wherein the interest point unchanged region is connected to the interest point change region;

[0062] A second reconstruction module is used to perform three-dimensional reconstruction based on the second image set after the interest points are marked and the first point cloud structure, and obtain a fourth point cloud structure according to the reconstruction result; an update module is used to update the map based on the fourth point cloud structure.

[0063] In a possible implementation, the update module is used to delete the replacement candidate frame in the fourth point cloud structure, and the replacement candidate frame is determined based on a threshold; the fourth point cloud structure after deleting the replacement candidate frame is projected onto the first plane, and a third map is obtained based on the projection result; and a fourth map is obtained based on the third map, and the fourth map is a map with completed interest point annotation.

[0064] On the other hand, a computer device is provided, comprising a processor and a memory, wherein the memory stores at least one computer program, and the at least one computer program is loaded and executed by the processor so that the computer device implements any of the above-mentioned methods for constructing a map.

[0065] On the other hand, a computer-readable storage medium is provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor so that a computer implements any of the above-mentioned methods for constructing a map.

[0066] On the other hand, a computer program product or a computer program is also provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs any of the above-mentioned methods for constructing a map.

[0067] The technical solution provided by the embodiments of the present application brings at least the following beneficial effects:

[0068] In the construction of the map, it is only necessary to collect images of the area to be mapped, and then perform 3D reconstruction based on the collected images to obtain the map. This method does not rely on the base map supplier, and even if the latest base map is missing, a map with high timeliness can be accurately constructed. At the same time, because the points of interest of the image have been marked before the 3D reconstruction, when the points of interest are marked on the map later, the map construction can be completed based on the previously marked points of interest. This method has low cost and high efficiency in map construction. BRIEF DESCRIPTION OF THE DRAWINGS

[0069] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0070] Figure 1 It is a schematic diagram of an implementation environment provided by an embodiment of the present application;

[0071] Figure 2 is a flow chart of a method for constructing a map provided in an embodiment of the present application;

[0072] Figure 3 is a schematic diagram of a three-dimensional reconstruction method provided in an embodiment of the present application;

[0073] Figure 4is a flow chart of a three-dimensional reconstruction method provided in an embodiment of the present application;

[0074] Figure 5 This is a flowchart of building a map provided by an embodiment of the present application;

[0075] Figure 6 This is a flow chart of updating a map provided in an embodiment of the present application;

[0076] Figure 7 is a schematic diagram of a device for constructing a map provided in an embodiment of the present application;

[0077] Figure 8 It is a structural diagram of a server provided in an embodiment of the present application;

[0078] Fig. 9 It is a structural schematic diagram of a device for constructing a map provided in an embodiment of the present application. DETAILED DESCRIPTION

[0079] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0080] This application embodiment provides a method for constructing a map, please refer to Figure 1 , which shows a schematic diagram of an implementation environment of the method provided in an embodiment of the present application. The implementation environment may include: a terminal 11 and a server 12.

[0081] The terminal 11 is installed with an application capable of acquiring images. After the application acquires an image, the acquired image is sent to the server 12. The server 12 can construct a map using the method provided in the embodiment of the present application, and the map is stored by the server 12. Optionally, the terminal 11 can acquire the map from the server 12, and the map is stored by the terminal 11.

[0082] Alternatively, the terminal 11 is installed with an application capable of acquiring images. After the application acquires the image, the terminal 11 constructs a map based on the method provided in the embodiment of the present application, and the map is stored by the terminal 11. Alternatively, the server 12 can acquire the map from the terminal 11, and the server 12 stores the map.

[0083] Alternatively, the terminal 11 obtains an image from the server 12, constructs a map based on the method provided in the embodiment of the present application, and the terminal 11 stores the map. Alternatively, the server 12 may obtain a map from the terminal 11, and the server 12 stores the map.

[0084] Optionally, the terminal 11 can be any electronic product that can interact with the user through one or more methods such as keyboard, touchpad, touch screen, remote control, voice interaction or handwriting device, such as PC (Personal Computer), mobile phone, smart phone, PDA (Personal Digital Assistant), wearable device, PPC (Pocket PC), tablet computer, smart car machine, smart TV, smart speaker, etc. The server 12 can be a server, a server cluster composed of multiple servers, or a cloud computing service center. The terminal 11 establishes a communication connection with the server 12 through a wired or wireless network.

[0085] Those skilled in the art should understand that the above-mentioned terminal 11 and server 12 are only examples, and other existing or future terminals or servers that are applicable to the present application should also be included in the protection scope of the present application and are included here by reference.

[0086] Based on the above Figure 1 In the implementation environment shown, the present application embodiment provides a method for constructing a map as shown in Figure 2 As shown, the method for constructing a map can be executed by a terminal or a server. Taking the method applied to a terminal as an example, the method includes steps 201 to 205.

[0087] In step 201, a first image set is acquired, wherein the first image set includes a plurality of images of an area to be mapped.

[0088] In a possible implementation, the first image set is acquired by using an image acquisition device to acquire images covering an area to be mapped, and using the acquired images as the first image set.

[0089] The embodiment of the present application does not limit the image acquisition device. Optionally, the image acquisition device is a device that has installed and runs an application that can realize the annotation of points of interest. Optionally, the image acquisition device is a common device that can perform image acquisition.

[0090] In step 202, points of interest in a first image set are marked.

[0091] Regardless of which device is used to obtain the first image set, the points of interest in the first image set can be annotated. For example, if the image acquisition device is a device that installs and runs an application that can realize the annotation of points of interest, the application is called during the acquisition process to complete the annotation of points of interest. If the image acquisition device is an ordinary device that can perform image acquisition, the acquired first image set is input into the first network model, and the points of interest in the first image set are annotated based on the first network model. The first network model is a network model that can recognize text information in the image.

[0092] In step 203, three-dimensional reconstruction is performed based on the first image set after the interest points are marked, and a first point cloud structure is obtained according to the reconstruction result.

[0093] Among them, the process of performing three-dimensional reconstruction based on the first image set after marking the interest points and obtaining the first point cloud structure according to the reconstruction result is as follows: Figure 3 As shown: extracting image features of each image in the first image set after marking the interest points; performing image feature pairing based on the image features of each image to obtain image pairs; performing three-dimensional reconstruction based on the image pairs, and obtaining a first point cloud structure according to the reconstruction results.

[0094] The embodiment of the present application does not limit the extracted image features. Optionally, feature points of the images in the first image set after the interest points are marked and descriptors of the corresponding feature points are extracted. Optionally, the descriptors are feature vectors corresponding to the feature points.

[0095] After extracting the image features of each image in the first image set, image feature pairing can be performed based on the image features. In a possible implementation, the matching degree of two image features is sorted based on the descriptor, and the two image features with the highest matching degree are selected based on the sorting result to complete the image feature pairing and obtain the image pair.

[0096] In one possible implementation, Figure 3 As shown, after completing the image feature pairing, the image pairs may be geometrically verified. Regarding the geometric verification method, optionally, the distance between the image pairs is calculated, and the image pairs whose distance is greater than a preset value are discarded, thereby optimizing the image pairs.

[0097] After acquiring the image pair, three-dimensional reconstruction can be performed based on the image pair, and the first point cloud structure can be obtained according to the reconstruction result. Figure 3 The schematic diagram of the three-dimensional reconstruction method is shown as an example. Figure 4 As shown, the process of obtaining the first point cloud structure by using SFM (Structure from Motion) incremental 3D reconstruction includes the following steps 2031-2035.

[0098] Step 2031: obtaining the positional relationship between the single image and the image pair in the first image set after the interest point is marked, where the positional relationship is embodied in the form of coordinate points in the same coordinate system.

[0099] like Figure 3 As shown, in a possible implementation, before obtaining the positional relationship between the single image and the image pair in the first image set after the annotated interest point, it also includes: initializing the image pair. Regarding the method of selecting the image pair to be initialized, optionally, the image pair that matches the most image features is selected as the image pair to be initialized. Optionally, based on the matching quality, the image pair with the highest image feature matching degree is selected as the image pair to be initialized. By initializing the image pair, the coordinate origin and scale of the world coordinate system required for three-dimensional reconstruction are determined.

[0100] After the image pair is initialized, the initialized image pair can be input into the incremental reconstruction submodule for preprocessing. The incremental reconstruction submodule is a submodule of the three-dimensional reconstruction model. Exemplarily, a three-dimensional point cloud structure is obtained based on the initialized image pair; the three-dimensional point cloud structure is iteratively optimized; the point cloud reconstructed based on the wrong image feature pairing in the optimized three-dimensional point cloud structure is filtered; the preprocessed image pair is incrementally registered based on the image in the first image set after the interest point is marked, and the positional relationship between the image in the first image set after the interest point is marked and the filtered optimized three-dimensional point cloud structure is obtained.

[0101] The embodiment of the present application does not limit the method of obtaining the three-dimensional point cloud structure based on the initialized image. In a possible implementation method, the three-dimensional point cloud structure can be obtained through triangulation positioning.

[0102] After obtaining the 3D point cloud structure, the 3D point cloud structure can be iteratively optimized. Optionally, a least squares optimization problem is constructed to optimize the 3D point cloud error through iterative calculation to achieve a controllable error range of the 3D point cloud structure, thereby obtaining a higher quality 3D point cloud structure. After iteratively optimizing the 3D point cloud structure, the 3D point cloud structure can be filtered to filter the 3D point cloud obtained by triangulation based on incorrect image feature pairing. In the filtering process, incorrect image feature pairing relationships are also filtered to improve the accuracy of the point cloud structure reconstructed based on image feature pairing during subsequent incremental reconstruction.

[0103] After preprocessing the image pair, the image in the first image set can be input into the incremental reconstruction submodule for incremental registration. The embodiment of the present application does not limit the method of incremental registration. In one possible implementation method, based on the position of the feature points in the image pair and the image feature matching relationship between the input image and the image pair, the PNP (Perspective-n-Point, multi-point perspective imaging) network model is used to implement incremental registration based on the motion relationship between two-dimensional and three-dimensional. Among them, the position of the feature points in the image pair is obtained in the process of preprocessing the image pair. The image feature matching relationship between the input image and the image pair is obtained by pairing the image features. The motion relationship between two-dimensional and three-dimensional is obtained based on the process of obtaining the three-dimensional point cloud structure by triangulation positioning in the preprocessed image pair. In the process of incremental registration, the position and posture information of the image acquisition device is also obtained by calculation.

[0104] Through incremental registration, the positional relationship between the input image and the image pair is obtained, that is, the coordinate points in the same coordinate system. The coordinate system used for unification is the world coordinate system determined by initializing the image pair.

[0105] Step 2032: Acquire a second point cloud structure based on the positional relationship.

[0106] The embodiment of the present application does not limit the method for obtaining the second point cloud structure. In one possible implementation method, the second point cloud structure is obtained based on the position relationship using triangulation positioning.

[0107] Step 2033: Iteratively optimize the second point cloud structure to obtain a third point cloud structure with a controllable error range.

[0108] Regarding the method of iteratively optimizing the second point cloud structure, optionally, a least squares optimization problem is constructed, and the second point cloud structure is optimized by iterative calculation. During the iterative optimization process, in addition to optimizing the second point cloud structure to obtain a third point cloud structure with a higher quality and a controllable error range, the pose sequence of the image acquisition device calculated during the incremental registration process is also optimized.

[0109] Step 2034: Filter the point cloud reconstructed based on the wrong image feature pairing in the third point cloud structure.

[0110] In addition to filtering the point cloud reconstructed based on erroneous image feature pairing in the third point cloud structure, erroneous image feature pairing information is also filtered to improve the image feature matching quality in the subsequent three-dimensional reconstruction process and the accuracy of the point cloud structure obtained based on image feature matching, while reducing the amount of tasks in subsequent filtering.

[0111] Step 2035: In response to satisfying the end condition, the filtered third point cloud structure is used as the first point cloud structure, and the end condition is to obtain the positional relationship between all images and image pairs in the first image set after the interest points are marked.

[0112] In step 204, a first map of the area is acquired based on the first point cloud structure.

[0113] The embodiments of the present application do not limit the method of obtaining the first map based on the first point cloud structure, including but not limited to: obtaining the first plane where the image acquisition device is located based on the posture sequence of the image acquisition device, the posture sequence is obtained in the process of three-dimensional reconstruction of the first image set after the points of interest are marked; projecting the first point cloud structure and the posture sequence of the image acquisition device to the first plane, and obtaining the first map according to the projection result.

[0114] In a possible implementation, the first plane where the image acquisition device is located is restored by plane fitting the pose sequence using the ransac (Random Sample Consensus) method. After obtaining the first plane, the first point cloud structure and the pose sequence of the image acquisition device can be projected onto the first plane to obtain a two-dimensional point cloud map, which is used as the first map.

[0115] During the projection process, it is necessary to determine the basic topological structure based on the key frames in the first point cloud structure. The embodiment of the present application does not limit the method for determining the key frames. Optionally, the image frames registered during the three-dimensional reconstruction process are key frames. Optionally, based on the frequency requirements of the three-dimensional reconstruction for the image frames, image frames that meet the frequency requirements are selected as key frames.

[0116] By restoring the first plane and then projecting the first point cloud structure onto the first plane, the error range can be effectively controlled and the accuracy of the map can be improved. At the same time, the pose sequence is projected onto the first plane to determine the key frame pose information, such as the direction of the key frame.

[0117] In step 205, a second map with points of interest marked is obtained based on the first map.

[0118] The embodiment of the present application does not limit the method of obtaining the second map with the points of interest marked based on the first map. For example, a fifth map is obtained; a map with higher definition is obtained based on the fifth map and the first map, and the points of interest are marked on the map with higher definition to obtain the second map. Optionally, higher definition means that the map layout is clearer than the first map.

[0119] Optionally, the fifth map is a CAD (Computer Aided Drafting) map of the shopping mall. Optionally, when the first map is limited in scope, the fifth map is a map that can provide image matching pairs in the first image set. Taking the first map as an example where the first map is limited to the main road of the shopping mall, the first map obtained cannot show the store area, and the match is not complete. The fifth map is a map that can show the store area.

[0120] After obtaining a map with higher definition, the points of interest are marked on the map to obtain a second map. The embodiment of the present application does not limit the method of marking the points of interest. Optionally, the position of the points of interest in the first point cloud structure is determined based on the mapping relationship between the two-dimensional image feature points and the three-dimensional point cloud, and the points of interest are projected onto a map with higher definition to obtain a second map.

[0121] Among them, the mapping relationship between the two-dimensional image feature points and the three-dimensional point cloud is obtained in image registration.

[0122] In addition to obtaining a higher-definition map based on the fifth map and marking points of interest on the map, when the images in the first image set collected are fully matched and the obtained first map has a higher definition, points of interest can be directly marked on the first map.

[0123] In a possible implementation, when a POI on a map changes, a user may initiate a request to update the map when the point of interest changes, thereby updating the map.

[0124] The embodiment of the present application does not limit the method of updating the map. Optionally, when the regional structure changes, the map is reconstructed, and the method of reconstructing the map is steps 201 to 205.

[0125] Optionally, in addition to reconstructing the map, when the point of interest changes and the regional structure does not change, a second image set is collected and the points of interest in the second image set are marked, the second image set includes images of the area where the point of interest has changed and images of the area where the point of interest has not changed, and the area where the point of interest has not changed is connected to the area where the point of interest has changed; three-dimensional reconstruction is performed based on the second image set after the point of interest is marked and the first point cloud structure, and a fourth point cloud structure is obtained according to the reconstruction result; and the map is updated based on the fourth point cloud structure.

[0126] The embodiment of the present application does not limit the area range of the images in the second image set. For example, if the areas of changed interest points are concentrated together, the nearby unchanged interest point areas are selected to obtain the second image set with the goal of achieving the minimum collection amount. For example, when the indoor map of a shopping mall is updated, store images of changed interest points and images of three stores next to the store with unchanged interest points are collected. The method of marking the points of interest in the second image set is shown in step 202.

[0127] After obtaining the second image set, three-dimensional reconstruction can be performed based on the second image set and the first point cloud structure, and a fourth point cloud structure is obtained according to the reconstruction result. In one possible implementation method, the second image set that completes the image feature pairing is input into the incremental reconstruction submodule, and a new point cloud is registered in the first point cloud structure to obtain the fourth point cloud structure. The implementation method is shown in steps 2031 to 2035.

[0128] After the fourth point cloud structure is obtained, the map can be updated based on the fourth point cloud structure. The method of updating the map based on the fourth point cloud structure includes but is not limited to: deleting the replacement candidate frame in the fourth point cloud structure, the replacement candidate frame is determined based on a threshold; projecting the fourth point cloud structure after deleting the replacement candidate frame to the first plane, and obtaining the third map based on the projection result; obtaining the fourth map based on the third map, and the fourth map is a map with the points of interest marked.

[0129] The replacement candidate frame is a key frame in the first point cloud structure that is a certain threshold away from the acquisition frame, and the acquisition frame is determined during the process of acquiring the second image set. Since the fourth point cloud structure is obtained by incremental reconstruction based on the first point cloud structure, the replacement candidate frame in the first point cloud structure is also located at the same position of the fourth point cloud structure. By deleting the replacement candidate frame, the old key frame that has changed is deleted, and the latest key frame is retained. The embodiment of the present application does not limit the setting method of the threshold. Optionally, the channel distance is used as the threshold based on the channel distance or channel width of the point of interest in the map to be updated. Optionally, based on the channel distance or channel width of the point of interest in the map to be updated, half of the channel distance is used as the threshold.

[0130] After the candidate frames are deleted and replaced, the obtained new three-dimensional point cloud structure can be projected onto the first plane to obtain a new two-dimensional point cloud map, and the obtained two-dimensional point cloud map is used as the third map.

[0131] After the third map is obtained, a fourth map may be obtained based on the third map, where the fourth map is a map with points of interest marked. The method of obtaining the fourth map based on the third map is as shown in step 204 .

[0132] In summary, the method for constructing a map provided in the embodiment of the present application is to complete the construction of the map based on the collected images, without relying on the base map supplier, and even if the latest base map is missing, the map can be accurately constructed. At the same time, because the lightweight point of interest annotation work has been completed during the image acquisition process, when annotating the points of interest on the map later, it is only necessary to project the points of interest on the map to complete the construction of the map, and the cost of constructing the map is low and the efficiency is high.

[0133] When updating the map, new images can be incrementally registered based on the first point cloud structure. Since the unchanged point of interest area near the changed point of interest area is selected during the acquisition process, fast matching can be achieved based on the image features of the unchanged point of interest area image, and the point cloud structure with changed points of interest that needs to be replaced can be accurately found, and the map update can be completed quickly and efficiently. Even if the regional structure changes, the incremental registration of new images is not applicable. Since the method for constructing a map provided by the embodiment of the present application is low-cost, the cost of reconstructing the map is small.

[0134] In one possible implementation, please see Figure 5 , Figure 5 This is a flowchart of building a map provided by an embodiment of the present application. Take the application of this method to a server as an example for explanation. Figure 5 As shown, when the method for constructing a map is used to construct an indoor map of a shopping mall, the method includes the following steps.

[0135] Step 501: Collect pictures and mark points of interest.

[0136] Exemplarily, a device having an application program capable of marking points of interest is used to collect multiple sets of images, and the collected images can cover the indoor environment of a shopping mall, and the points of interest are marked by the device during the collection process.

[0137] Step 502: Use the picture to perform three-dimensional reconstruction of the indoor environment of the shopping mall.

[0138] Step 503: Convert the 3D reconstructed point cloud and key frame information into a map.

[0139] Step 504: Annotate the collected points of interest into the map through key frame positions.

[0140] After the construction of the mall's indoor map is completed, when individual stores in the mall change, the indoor map can be updated. For details, please see Figure 6 .

[0141] Step 601: continuously collect pictures of unchanged interest points and pictures of changed interest points.

[0142] Exemplarily, the collected areas where no store changes have occurred and the areas where store changes have occurred are connected.

[0143] Step 602: Based on the acquired image, a new acquisition frame is registered in the generated point cloud, and a certain distance threshold is selected to be marked as a replacement candidate frame.

[0144] Exemplarily, the channel distance of the point of interest in the map to be updated is selected as a threshold, and the key frame with a certain distance threshold from the acquisition frame is selected as the replacement candidate frame.

[0145] Step 603: Update the point cloud and key frame information based on the replacement candidate frame, and project the updated point cloud and key frame information into the map.

[0146] Exemplarily, the candidate frames are deleted and replaced to complete the updating of the point cloud structure and key frame information, and a new two-dimensional point cloud map is obtained by projecting the updated point cloud structure onto the first plane where the original shopping mall map is located.

[0147] Step 604: According to the marked change form of the interest point and the key frame information, the interest point is processed and updated accordingly.

[0148] The indoor map of the shopping mall is updated by updating the points of interest, namely the store information.

[0149] See also Figure 7 An embodiment of the present application provides a device for constructing a map, which includes: a first acquisition module 701, a labeling module 702, a first reconstruction module 703, a second acquisition module 704, and a third acquisition module 705.

[0150] A first acquisition module 701 is used to acquire a first image set, where the first image set includes a plurality of images of an area to be mapped;

[0151] Annotation module 702, used to annotate points of interest in the first image set;

[0152] A first reconstruction module 703 is used to perform three-dimensional reconstruction based on the first image set after the interest points are marked, and obtain a first point cloud structure according to the reconstruction result;

[0153] A second acquisition module 704 is used to acquire a first map of the area based on the first point cloud structure;

[0154] The third acquisition module 705 is used to acquire a second map with points of interest marked based on the first map.

[0155] Optionally, the first reconstruction module 703 includes:

[0156] An extraction unit, used to extract image features of each image in the first image set after the interest point is marked;

[0157] A feature pairing unit, used for performing image feature pairing based on image features of each image to obtain an image pair;

[0158] The three-dimensional reconstruction unit is used to perform three-dimensional reconstruction based on the image pair and obtain a first point cloud structure according to the reconstruction result.

[0159] Optionally, the three-dimensional reconstruction unit comprises:

[0160] A position relationship acquisition subunit is used to acquire the position relationship between a single image and an image pair in the first image set after the interest point is marked, where the position relationship is embodied in the form of coordinate points in the same coordinate system;

[0161] A point cloud structure acquisition subunit, used for acquiring a second point cloud structure based on the position relationship;

[0162] An optimization subunit, used for iteratively optimizing the second point cloud structure to obtain a third point cloud structure with a controllable error range;

[0163] A filtering subunit, used for filtering the point cloud reconstructed based on the wrong image feature pairing in the third point cloud structure;

[0164] The determination subunit is used to use the filtered third point cloud structure as the first point cloud structure in response to meeting the end condition, and the end condition is to obtain the position relationship between all images and image pairs in the first image set after marking the interest point.

[0165] Optionally, a position relationship acquisition subunit is used to acquire a three-dimensional point cloud structure based on the initialized image pair; iteratively optimize the three-dimensional point cloud structure; filter the point cloud reconstructed based on erroneous image feature pairing in the optimized three-dimensional point cloud structure; perform incremental registration on the preprocessed image pair based on the image in the first image set after the interest point is marked, and acquire the positional relationship between the image in the first image set after the interest point is marked and the filtered and optimized three-dimensional point cloud structure based on the filtered and optimized three-dimensional point cloud structure.

[0166] Optionally, the second acquisition module 704 is used to acquire a first plane where the image acquisition device is located based on a posture sequence of the image acquisition device, wherein the posture sequence is obtained during a three-dimensional reconstruction of the first image set after the points of interest are marked; the first point cloud structure and the posture sequence of the image acquisition device are projected to the first plane, and a first map is acquired according to the projection result.

[0167] Optionally, the device further comprises:

[0168] The construction module is used to rebuild the map when the regional structure changes.

[0169] Optionally, the device further comprises:

[0170] A collection module, used for collecting a second image set and marking the interest points in the second image set when the interest points are changed and the regional structure is unchanged, wherein the second image set includes images of the area where the interest points are changed and images of the area where the interest points are unchanged, and the area where the interest points are unchanged is connected to the area where the interest points are changed;

[0171] A second reconstruction module is used to perform three-dimensional reconstruction based on the second image set after the interest points are marked and the first point cloud structure, and obtain a fourth point cloud structure according to the reconstruction result;

[0172] An updating module is used to update the map based on the fourth point cloud structure.

[0173] Optionally, an updating module is used to delete replacement candidate frames in the fourth point cloud structure, where the replacement candidate frames are determined based on a threshold; project the fourth point cloud structure from which the replacement candidate frames have been deleted onto the first plane, and obtain a third map based on the projection result; and obtain a fourth map based on the third map, where the fourth map is a map that completes the annotation of points of interest.

[0174] When constructing a map, the above device only needs to collect images of the area to be mapped, and can obtain a map by performing three-dimensional reconstruction based on the collected images. It does not rely on base map suppliers, and can accurately construct a map with high timeliness even if the latest base map is missing. At the same time, because the work of marking points of interest in the image has been completed before the three-dimensional reconstruction, when marking points of interest on the map later, the map construction can be completed based on the previously marked points of interest. The cost of constructing the map is low and the efficiency is high.

[0175] It should be noted that the device provided in the above embodiment only uses the division of the above functional modules as an example to implement its functions. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the device and method embodiments provided in the above embodiment belong to the same concept, and their specific implementation process is detailed in the method embodiment, which will not be repeated here.

[0176] Figure 8 It is a structural diagram of a server provided in an embodiment of the present application. The server may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 801 and one or more memories 802, wherein the one or more memories 802 store at least one computer program, and the at least one computer program is loaded and executed by the one or more processors 801, so that the server implements the method of building a map provided in the above-mentioned various method embodiments. Of course, the server may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The server may also include other components for implementing device functions, which will not be repeated here.

[0177] Fig. 91 is a schematic diagram of a map-building device provided in an embodiment of the present application. The device may be a terminal, such as a smart phone, a tablet computer, an MP3 (Moving Picture Experts Group Audio Layer III, Moving Picture Experts Compression Standard Audio Layer 3) player, an MP4 (Moving Picture Experts Group Audio Layer IV, Moving Picture Experts Compression Standard Audio Layer 4) player, a laptop computer or a desktop computer. The terminal may also be referred to as a user device, a portable terminal, a laptop terminal, a desktop terminal or other names.

[0178] Typically, the terminal includes: a processor 901 and a memory 902 .

[0179] The processor 901 may include one or more processing cores, such as a 4-core processor, an 8-core processor, and the like. The processor 901 may be implemented in at least one hardware form of DSP (Digital Signal Processing), FPGA (Field-Programmable Gate Array), and PLA (Programmable Logic Array). The processor 901 may also include a main processor and a coprocessor. The main processor is a processor for processing data in an awake state, also known as a CPU (Central Processing Unit); the coprocessor is a low-power processor for processing data in a standby state. In some embodiments, the processor 901 may be integrated with a GPU (Graphics Processing Unit), which is responsible for rendering and drawing the content to be displayed on the display screen. In some embodiments, the processor 901 may also include an AI (Artificial Intelligence) processor, which is used to process computing operations related to machine learning.

[0180] The memory 902 may include one or more computer-readable storage media, which may be non-transitory. The memory 902 may also include a high-speed random access memory, and a non-volatile memory, such as one or more disk storage devices, flash memory storage devices. In some embodiments, the non-transitory computer-readable storage medium in the memory 902 is used to store at least one instruction, which is used to be executed by the processor 901 so that the terminal implements the method for constructing a map provided in the method embodiment of the present application.

[0181] In some embodiments, the terminal may also optionally include: a peripheral device interface 903 and at least one peripheral device. The processor 901, the memory 902 and the peripheral device interface 903 may be connected via a bus or a signal line. Each peripheral device may be connected to the peripheral device interface 903 via a bus, a signal line or a circuit board. Specifically, the peripheral device includes: at least one of a radio frequency circuit 904, a display screen 905, a camera assembly 906, an audio circuit 907, a positioning assembly 908 and a power supply 909.

[0182] The peripheral device interface 903 may be used to connect at least one peripheral device related to I / O (Input / Output) to the processor 901 and the memory 902. In some embodiments, the processor 901, the memory 902, and the peripheral device interface 903 are integrated on the same chip or circuit board; in some other embodiments, any one or two of the processor 901, the memory 902, and the peripheral device interface 903 may be implemented on a separate chip or circuit board, which is not limited in this embodiment.

[0183] The radio frequency circuit 904 is used to receive and transmit RF (Radio Frequency) signals, also known as electromagnetic signals. The radio frequency circuit 904 communicates with communication networks and other communication devices through electromagnetic signals. The radio frequency circuit 904 converts electrical signals into electromagnetic signals for transmission, or converts received electromagnetic signals into electrical signals. Optionally, the radio frequency circuit 904 includes: an antenna system, an RF transceiver, one or more amplifiers, a tuner, an oscillator, a digital signal processor, a codec chipset, a user identity module card, and the like. The radio frequency circuit 904 can communicate with other terminals through at least one wireless communication protocol. The wireless communication protocol includes, but is not limited to: a metropolitan area network, various generations of mobile communication networks (2G, 3G, 4G and 5G), a wireless local area network and / or a WiFi (Wireless Fidelity) network. In some embodiments, the radio frequency circuit 904 may also include circuits related to NFC (Near Field Communication), which is not limited in this application.

[0184] The display screen 905 is used to display the UI (User Interface). The UI may include graphics, text, icons, videos, and any combination thereof. When the display screen 905 is a touch display screen, the display screen 905 also has the ability to collect touch signals on the surface or above the surface of the display screen 905. The touch signal can be input to the processor 901 as a control signal for processing. At this time, the display screen 905 can also be used to provide virtual buttons and / or virtual keyboards, also known as soft buttons and / or soft keyboards. In some embodiments, the display screen 905 can be one, set on the front panel of the terminal; in other embodiments, the display screen 905 can be at least two, respectively set on different surfaces of the terminal or in a folding design; in other embodiments, the display screen 905 can be a flexible display screen, set on a curved surface or a folding surface of the terminal. Even, the display screen 905 can also be set to a non-rectangular irregular shape, that is, a special-shaped screen. The display screen 905 can be made of materials such as LCD (Liquid Crystal Display), OLED (Organic Light-Emitting Diode, organic light-emitting diode).

[0185] The camera assembly 906 is used to capture images or videos. Optionally, the camera assembly 906 includes a front camera and a rear camera. Typically, the front camera is arranged on the front panel of the terminal, and the rear camera is arranged on the back of the terminal. In some embodiments, there are at least two rear cameras, which are any one of a main camera, a depth of field camera, a wide-angle camera, and a telephoto camera, so as to realize the fusion of the main camera and the depth of field camera to realize the background blur function, the fusion of the main camera and the wide-angle camera to realize panoramic shooting and VR (Virtual Reality) shooting function or other fusion shooting functions. In some embodiments, the camera assembly 906 may also include a flash. The flash can be a monochrome temperature flash or a dual-color temperature flash. A dual-color temperature flash refers to a combination of a warm light flash and a cold light flash, which can be used for light compensation at different color temperatures.

[0186] The audio circuit 907 may include a microphone and a speaker. The microphone is used to collect sound waves from the user and the environment, and convert the sound waves into electrical signals and input them into the processor 901 for processing, or input them into the radio frequency circuit 904 to achieve voice communication. For the purpose of stereo acquisition or noise reduction, there may be multiple microphones, which are respectively arranged at different parts of the terminal. The microphone may also be an array microphone or an omnidirectional acquisition microphone. The speaker is used to convert the electrical signal from the processor 901 or the radio frequency circuit 904 into sound waves. The speaker may be a traditional film speaker or a piezoelectric ceramic speaker. When the speaker is a piezoelectric ceramic speaker, it can not only convert the electrical signal into sound waves audible to humans, but also convert the electrical signal into sound waves inaudible to humans for purposes such as ranging. In some embodiments, the audio circuit 907 may also include a headphone jack.

[0187] The positioning component 908 is used to locate the current geographical location of the terminal to implement navigation or LBS (Location Based Service). The positioning component 908 can be a positioning component based on the US GPS (Global Positioning System), China's Beidou system, Russia's Grenas system or the European Union's Galileo system.

[0188] The power supply 909 is used to power various components in the terminal. The power supply 909 can be an alternating current, a direct current, a disposable battery, or a rechargeable battery. When the power supply 909 includes a rechargeable battery, the rechargeable battery can support wired charging or wireless charging. The rechargeable battery can also be used to support fast charging technology.

[0189] In some embodiments, the terminal further includes one or more sensors 910 , including but not limited to: an acceleration sensor 911 , a gyroscope sensor 912 , a pressure sensor 913 , a fingerprint sensor 914 , an optical sensor 915 , and a proximity sensor 916 .

[0190] The acceleration sensor 911 can detect the magnitude of acceleration on the three coordinate axes of the coordinate system established by the terminal. For example, the acceleration sensor 911 can be used to detect the components of gravity acceleration on the three coordinate axes. The processor 901 can control the display screen 905 to display the user interface in a horizontal view or a vertical view according to the gravity acceleration signal collected by the acceleration sensor 911. The acceleration sensor 911 can also be used for collecting game or user motion data.

[0191] The gyroscope sensor 912 can detect the body direction and rotation angle of the terminal, and the gyroscope sensor 912 can cooperate with the acceleration sensor 911 to collect the user's 3D actions on the terminal. The processor 901 can implement the following functions based on the data collected by the gyroscope sensor 912: motion sensing (such as changing the UI according to the user's tilt operation), image stabilization during shooting, game control, and inertial navigation.

[0192] The pressure sensor 913 can be set in the side frame of the terminal and / or the lower layer of the display screen 905. When the pressure sensor 913 is set in the side frame of the terminal, the user's holding signal of the terminal can be detected, and the processor 901 performs left and right hand recognition or shortcut operation according to the holding signal collected by the pressure sensor 913. When the pressure sensor 913 is set in the lower layer of the display screen 905, the processor 901 controls the operability controls on the UI interface according to the user's pressure operation on the display screen 905. The operability controls include at least one of a button control, a scroll bar control, an icon control, and a menu control.

[0193] The fingerprint sensor 914 is used to collect the user's fingerprint, and the processor 901 identifies the user's identity based on the fingerprint collected by the fingerprint sensor 914, or the fingerprint sensor 914 identifies the user's identity based on the collected fingerprint. When the user's identity is identified as a trusted identity, the processor 901 authorizes the user to perform relevant sensitive operations, which include unlocking the screen, viewing encrypted information, downloading software, paying, and changing settings. The fingerprint sensor 914 can be set on the front, back or side of the terminal. When a physical button or a manufacturer logo is set on the terminal, the fingerprint sensor 914 can be integrated with the physical button or the manufacturer logo.

[0194] The optical sensor 915 is used to collect the ambient light intensity. In one embodiment, the processor 901 can control the display brightness of the display screen 905 according to the ambient light intensity collected by the optical sensor 915. Specifically, when the ambient light intensity is high, the display brightness of the display screen 905 is increased; when the ambient light intensity is low, the display brightness of the display screen 905 is reduced. In another embodiment, the processor 901 can also dynamically adjust the shooting parameters of the camera component 906 according to the ambient light intensity collected by the optical sensor 915.

[0195] The proximity sensor 916, also known as a distance sensor, is usually arranged on the front panel of the terminal. The proximity sensor 916 is used to collect the distance between the user and the front of the terminal. In one embodiment, when the proximity sensor 916 detects that the distance between the user and the front of the terminal is gradually decreasing, the processor 901 controls the display screen 905 to switch from the screen-on state to the screen-off state; when the proximity sensor 916 detects that the distance between the user and the front of the terminal is gradually increasing, the processor 901 controls the display screen 905 to switch from the screen-off state to the screen-on state.

[0196] Those skilled in the art will understand that Fig. 9 The structure shown in the figure does not constitute a limitation of the device, and may include more or less components than shown in the figure, or combine certain components, or adopt a different component arrangement.

[0197] In an exemplary embodiment, a computer device is also provided, the computer device comprising a processor and a memory, wherein at least one computer program is stored in the memory. The at least one computer program is loaded and executed by one or more processors, so that the computer device implements any of the above-mentioned methods for constructing a map.

[0198] In an exemplary embodiment, a computer-readable storage medium is also provided, in which at least one computer program is stored. The at least one computer program is loaded and executed by a processor of a computer device to enable the computer to implement any of the above-mentioned methods for constructing a map.

[0199] In one possible implementation, the computer-readable storage medium may be a read-only memory (ROM), a random access memory (RAM), a compact disc (CD-ROM), a magnetic tape, a floppy disk, an optical data storage device, etc.

[0200] In an exemplary embodiment, a computer program product or a computer program is also provided, the computer program product or the computer program comprising computer instructions, the computer instructions being stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device performs any of the above-mentioned methods for constructing a map.

[0201] It should be understood that the "plurality" mentioned in this article refers to two or more. "And / or" describes the association relationship of the associated objects, indicating that there can be three relationships. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the associated objects are in an "or" relationship.

[0202] The above description is only an exemplary embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A method for constructing a map, characterized in that: The method comprises: Acquire a first image set, wherein the first image set includes a plurality of images of an area to be mapped; marking points of interest in the first image set; Performing three-dimensional reconstruction based on the first image set after the interest points are marked, and obtaining a first point cloud structure according to the reconstruction result; Acquire a first map of the area based on the first point cloud structure; Acquire a second map with points of interest marked based on the first map; The three-dimensional reconstruction is performed based on the first image set after the interest points are marked, and a first point cloud structure is obtained according to the reconstruction result, including: Extracting image features of each image in the first image set after the marked interest point; Performing image feature pairing based on the image features of each image to obtain an image pair; Performing three-dimensional reconstruction based on the image pair, and acquiring a first point cloud structure according to the reconstruction result; The performing three-dimensional reconstruction based on the image pair and acquiring a first point cloud structure according to the reconstruction result includes: Acquire a positional relationship between a single image in the first image set after the marked interest point and the image pair, wherein the positional relationship is embodied in the form of coordinate points in the same coordinate system; Acquire a second point cloud structure based on the positional relationship; Iteratively optimizing the second point cloud structure to obtain a third point cloud structure with a controllable error range; filtering the point cloud reconstructed based on the wrong image feature pairing in the third point cloud structure; In response to satisfying an end condition, using the filtered third point cloud structure as the first point cloud structure, wherein the end condition is obtaining a positional relationship between all images in the first image set after the annotated interest point and the image pair; The acquiring a first map of the area based on the first point cloud structure includes: Acquire a first plane where the image acquisition device is located based on a posture sequence of the image acquisition device, wherein the posture sequence is obtained in a process of three-dimensional reconstruction based on the first image set after the interest point is marked; The first point cloud structure and the posture sequence of the image acquisition device are projected onto the first plane, and the first map is acquired according to the projection result.

2. The method according to claim 1, characterized in that The obtaining of the positional relationship between the single image in the first image set after the annotated interest point and the image pair includes: Obtain a three-dimensional point cloud structure based on the initialized image pair; Iteratively optimizing the three-dimensional point cloud structure; The point cloud reconstructed based on the wrong image feature pairing in the 3D point cloud structure after filtering optimization; The preprocessed image pairs are incrementally registered based on the images in the first image set after the marked interest points, and the positional relationship between the images in the first image set after the marked interest points and the filtered and optimized three-dimensional point cloud structure is obtained based on the filtered and optimized three-dimensional point cloud structure.

3. The method according to any one of claims 1-2, characterized in that: After acquiring the second map with the points of interest marked based on the first map, the method further includes: Rebuild the map when the region structure changes.

4. The method according to claim 1, characterized in that: After acquiring the second map with the points of interest marked based on the first map, the method further includes: When the interest point changes and the regional structure does not change, a second image set is collected, and the interest point in the second image set is marked, wherein the second image set includes images of the interest point change area and images of the interest point unchanged area, and the interest point unchanged area is connected to the interest point change area; Performing three-dimensional reconstruction based on the second image set after the interest points are marked and the first point cloud structure, and acquiring a fourth point cloud structure according to the reconstruction result; The map is updated based on the fourth point cloud structure.

5. The method according to claim 4, characterized in that The updating of the map based on the fourth point cloud structure includes: deleting a replacement candidate frame in the fourth point cloud structure, wherein the replacement candidate frame is determined based on a threshold; Projecting the fourth point cloud structure after deleting the replacement candidate frame onto the first plane, and acquiring a third map based on the projection result; A fourth map is obtained based on the third map, where the fourth map is a map with points of interest marked.

6. A device for constructing a map, characterized in that: The device comprises: A first acquisition module, configured to acquire a first image set, wherein the first image set includes a plurality of images of an area to be mapped; a marking module, used for marking points of interest in the first image set; A first reconstruction module, used for performing three-dimensional reconstruction based on the first image set after the interest points are marked, and obtaining a first point cloud structure according to the reconstruction result; A second acquisition module, configured to acquire a first map of the area based on the first point cloud structure; The third acquisition module is used to acquire a second map with the points of interest marked based on the first map; the three-dimensional reconstruction is performed based on the first image set after the points of interest are marked, and a first point cloud structure is obtained according to the reconstruction result, including: Extracting image features of each image in the first image set after the marked interest point; Performing image feature pairing based on the image features of each image to obtain an image pair; Performing three-dimensional reconstruction based on the image pair, and acquiring a first point cloud structure according to the reconstruction result; The performing three-dimensional reconstruction based on the image pair and acquiring a first point cloud structure according to the reconstruction result includes: Acquire a positional relationship between a single image in the first image set after the marked interest point and the image pair, wherein the positional relationship is embodied in the form of coordinate points in the same coordinate system; Acquire a second point cloud structure based on the positional relationship; Iteratively optimizing the second point cloud structure to obtain a third point cloud structure with a controllable error range; filtering the point cloud reconstructed based on the wrong image feature pairing in the third point cloud structure; In response to satisfying an end condition, using the filtered third point cloud structure as the first point cloud structure, wherein the end condition is obtaining a positional relationship between all images in the first image set after the annotated interest point and the image pair; The acquiring a first map of the area based on the first point cloud structure includes: Acquire a first plane where the image acquisition device is located based on a posture sequence of the image acquisition device, wherein the posture sequence is obtained in a process of three-dimensional reconstruction based on the first image set after the interest point is marked; The first point cloud structure and the posture sequence of the image acquisition device are projected onto the first plane, and the first map is acquired according to the projection result.

7. A computer device, characterized in that: The computer device includes a processor and a memory, wherein at least one computer program is stored in the memory, and the at least one computer program is loaded and executed by the processor so that the computer device implements the method for constructing a map as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores at least one computer program, and the at least one computer program is loaded and executed by a processor so that the computer implements the method for constructing a map as described in any one of claims 1 to 5.

9. A computer program product, characterized in that The computer program product comprises a computer program or instructions, and the computer program or instructions are executed by a processor to enable a computer to implement the method for constructing a map as claimed in any one of claims 1 to 5.

Images