Map construction method, device, equipment and storage medium

Abstract

The present application provides a map construction method, device, equipment and storage medium, which relate to the field of map construction. Among them, the map construction method can generate a low-precision map that can provide a VPS based on feature points for the first area based on a large number of first images of the first area taken by the user when using the first visual positioning service in the first area, the low-precision positioning results corresponding to the first images, and the 3D model of the first area. Then, the low-precision map is bundle adjusted in combination with some key 3D point coordinates of the first area to construct a high-precision map that can provide a VPS based on feature points for the first area. During the map construction process, there is no need to spend extra manpower to collect map data, which can save a lot of collection manpower and collection costs. At the same time, the low-precision map is bundle adjusted in combination with some key 3D point coordinates of the first area, and the high-precision positioning capability of the map can also be maintained.

Description

Technical Field

[0001] The present application relates to the field of map construction technology, and in particular to a map construction method, device, equipment and storage medium. Background Art

[0002] With the advent of the 5G era, technologies such as augmented reality (AR) navigation and intelligent driving are developing rapidly, and these technologies rely on high-precision maps. Building high-precision maps is the key to the widespread application of technologies such as AR navigation and intelligent driving.

[0003] There are two main mapping solutions at present, namely, using satellite images to build maps and collecting map data on the ground to build maps. Among them, although the cost of building maps using satellite images is low, the accuracy of the constructed maps is also low. Although the method of building maps by collecting map data on the ground can build high-precision maps, the cost of collecting map data is high. For example, collecting map data by car collection, manual collection, etc. requires a lot of manpower and expensive equipment. In addition, the method of building maps by collecting map data on the ground cannot meet the update requirements of high-precision maps. Summary of the invention

[0004] The embodiments of the present application provide a map construction method, apparatus, device and storage medium, which can significantly reduce the cost of map collection while maintaining the high-precision positioning capability of the map, thereby achieving a low-cost and high-precision positioning effect.

[0005] In a first aspect, an embodiment of the present application provides a map construction method, the method comprising: obtaining a first image taken by a user when using a first visual positioning service in a first area, and a low-precision positioning result corresponding to the first image. The first visual positioning service includes a skyline-based visual positioning service and / or a contour-based visual positioning service; wherein, when the first image is an image corresponding to an outdoor scene of the first area, the low-precision positioning result is a positioning result of the first visual positioning service or a positioning result of a traditional positioning service; when the first image is an image corresponding to an indoor scene of the first area, the low-precision positioning result is an indoor positioning result. According to the first image, the low-precision positioning result corresponding to the first image, and the 3D model of the first area, the 3D point coordinates corresponding to the 2D feature points in the first image are determined. According to the first image, the low-precision positioning result corresponding to the first image, and the 3D point coordinates corresponding to the 2D feature points in the first image, a low-precision map capable of providing a visual positioning service based on feature points for the first area is constructed. According to the key 3D point coordinates of the first area, a global bundle adjustment is performed on the low-precision map capable of providing a visual positioning service based on feature points for the first area, so as to obtain a high-precision map capable of providing a visual positioning service based on feature points for the first area.

[0006] The key 3D points of the first area may be some 3D points in the first area whose precise coordinates are easily obtained. For example, the first area may be a city or a part of a city. In this case, the key 3D points of the first area may be some urban public control points in the first area. For example, the urban public control points may include: corner points and feature points of buildings in the first area.

[0007] The map construction method can generate a low-precision map that can provide a VPS based on feature points for the first area based on a large number of images (i.e., crowdsourced images) of the first area taken by the user when using the first visual positioning service in the first area, the low-precision positioning results corresponding to these images, and the 3D model of the first area, and then further combine the coordinates of some key 3D points in the first area to perform a bundle adjustment on the low-precision map, so as to construct a high-precision map that can provide a VPS based on feature points for the first area. In the map construction process, there is no need to spend extra manpower to collect map data, only the VPS based on the skyline can be provided for the first area, and the 3D model of the first area can be obtained. The VPS based on the skyline can be constructed by satellite images of the first area, and the 3D model of the first area can also be a patch model generated based on the satellite images of the first area. Therefore, it can be seen that the map construction method only needs to purchase satellite images from relevant suppliers, which can save a lot of collection manpower and collection costs. At the same time, since the map construction method can combine the coordinates of some key 3D points in the first area to perform a bundle adjustment on the low-precision map, while greatly reducing the map collection cost, the high-precision positioning capability of the map can also be maintained, thereby achieving the goal of building a low-cost and high-precision map.

[0008] In addition, it can be seen that in the embodiment of the present application, the map construction process and the process of the user using the VPS are independent of each other. The embodiment of the present application can build the map while providing the VPS to the user, and the VPS service can be provided not only after the map construction is completed.

[0009] Optionally, in the map construction method, after constructing a low-precision map capable of providing a visual positioning service based on feature points for the first area based on the first image, the low-precision positioning result corresponding to the first image, and the 3D point coordinates corresponding to the 2D feature points in the first image, the visual positioning service based on feature points in the low-precision map does not have too many constraints, and the support scope of positioning is not limited to images that must capture the skyline or contour lines. As long as there are locations with feature points, positioning can be performed, such as: only taking images of the building itself can also perform VPS. As a result, the low-precision map also expands the scope of positioning services, improves the success rate of positioning, and expands the usage scenarios, so that more users can provide more first images when using the low-precision map to perform positioning.

[0010] Optionally, the method of performing global bundle adjustment on the low-precision map that can provide feature point-based visual positioning services for the first area according to the key 3D point coordinates of the first area includes: calibrating 2D feature points in the first image constituting the low-precision map according to the key 3D point coordinates of the first area, and performing global bundle adjustment on the low-precision map. When performing global bundle adjustment on the low-precision map, the calibrated key 3D point coordinates in the first image constituting the low-precision map participate in iteration but not in update.

[0011] When performing global bundle adjustment on a low-precision map, the coordinates of the key 3D points calibrated in the first image that constitutes the low-precision map participate in the iteration but not in the update, so that the 3D points in the low-precision map will always be pulled toward the key 3D points of the first area, so that the 3D points in the low-precision map will exist in the same coordinate system as the key 3D points of the first area. In this way, the coordinates of the key 3D points in the first area can constrain the low-precision map to an accurate scale and position, restore the accurate absolute scale and absolute pose of the low-precision map, and the global bundle adjustment algorithm will output a high-precision map that can provide a feature point-based VPS for the first area.

[0012] In some other embodiments, when performing global bundle adjustment on a low-precision map that can provide a feature point-based VPS for the first area, an iterative closest point (ICP) algorithm can also be used to align the 3D points in the low-precision map with the key 3D points of the first area to a precise coordinate system, without limitation here.

[0013] Optionally, the method of accurately calibrating the 2D feature points in the first image constituting the low-precision map using the key 3D point coordinates of the first area may include manual calibration and automatic calibration. For example, a user may directly use a portable device such as a mobile phone or a tablet computer to calibrate the key 3D point coordinates corresponding to the corner points containing the building in the first image.

[0014] Optionally, when there is an existing high-precision map for the second area bordering the first area or the second area within the first area, the low-precision map that can provide feature point-based visual positioning services for the first area is globally adjusted based on the key 3D point coordinates of the first area, including: calibrating the 2D feature points in the first image constituting the low-precision map based on the key 3D point coordinates of the first area, and performing global bundle adjustment on the low-precision map and the existing high-precision map of the second area; wherein, when performing global bundle adjustment on the low-precision map and the existing high-precision map of the second area, the calibrated key 3D point coordinates in the first image constituting the low-precision map and the 3D point coordinates in the existing high-precision map of the second area participate in the iteration but not in the update.

[0015] Among them, the existing high-precision map of the second area can be constructed by map data collected by car collection, manpower collection, etc., or it can be constructed by the above-mentioned map construction method. When there is an existing high-precision map in the second area, in the map construction method, according to the coordinates of the key 3D points of the first area, the 2D feature points in the first image constituting the low-precision map are calibrated, and the low-precision map and the existing high-precision map of the second area are globally bundle adjusted. The accurate map points and images in the existing high-precision map can be used to correct the absolute scale and absolute posture of the low-precision map. In other words, since the existing high-precision map has a certain intersection with the low-precision map, this intersection can be used to pull the low-precision map to a coordinate system aligned with the high-precision map.

[0016] Since the map construction method can be combined with the existing high-precision map to perform global bundle adjustment on the low-precision map that can provide the feature point-based VPS for the first area, the embodiment of the present application can continuously acquire new first images taken by the user and update the map while the user is using the high-precision map based on the feature point VPS constructed by the map construction method.

[0017] In some implementations, when the first image is an image corresponding to an outdoor scene in the first area, the key 3D point coordinates of the first area are the absolute longitude and latitude coordinates of some building corners in the first area. When the area corresponding to the first image is covered by the first visual positioning service, the low-precision positioning result is the positioning result of the first visual positioning service; when the area corresponding to the first image is not covered by the first visual positioning service, the low-precision positioning result is the positioning result of the traditional positioning service.

[0018] Optionally, the traditional positioning service result is a positioning result of any one of the positioning methods including global positioning system GPS positioning, real-time kinematic differential method RTK positioning, magnetometer positioning, and network location provider NLP positioning.

[0019] Optionally, when the first image includes a skyline, the first visual positioning service result is a visual positioning service positioning result based on the skyline; when the first image does not include a skyline, the first visual positioning service result is a visual positioning service positioning result based on a contour line.

[0020] In some other embodiments, when the first image is an image corresponding to an indoor scene of the first area, the key 3D point coordinates of the first area are the absolute longitude and latitude coordinates of the 3D points marked in the CAD drawing of the first area, or the absolute longitude and latitude coordinates of some building corner points in the first area.

[0021] Optionally, the indoor positioning result is a positioning result of any one of Bluetooth positioning, WiFi positioning, geomagnetic positioning, and ultra-wideband UWB positioning.

[0022] Optionally, determining the 3D point coordinates corresponding to the 2D feature points in the first image based on the first image, the low-precision positioning result corresponding to the first image, and the 3D model of the first area includes: extracting the 2D feature points in the first image; inputting the 2D feature points in the first image, the camera intrinsic parameters corresponding to the first image, the low-precision positioning result corresponding to the first image, and the 3D model of the first area into a ray intersection algorithm to obtain the 3D point coordinates corresponding to the 2D feature points in the first image output by the ray intersection algorithm.

[0023] Optionally, when the first image is an image corresponding to an outdoor scene of the first area, the 3D model of the first area is an established CAD 3D model of the first area or a patch model generated according to a satellite image or an aerial image of the first area.

[0024] Optionally, when the first image is an image corresponding to an indoor scene of the first area, the 3D model of the first area is an established CAD 3D model of the first area.

[0025] In a second aspect, an embodiment of the present application provides a map construction device, which can be used to implement the method described in the first aspect above. The functions of the device can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions, for example, an acquisition module, a 3D point coordinate extraction module, a low-precision map construction module, a high-precision map construction module, etc.

[0026] Among them, the acquisition module is used to obtain the first image taken by the user when using the first visual positioning service in the first area, and the low-precision positioning result corresponding to the first image. The first visual positioning service includes a skyline-based visual positioning service and / or a contour-based visual positioning service; wherein, when the first image is an image corresponding to an outdoor scene of the first area, the low-precision positioning result is a positioning result of the first visual positioning service or a positioning result of a traditional positioning service; when the first image is an image corresponding to an indoor scene of the first area, the low-precision positioning result is an indoor positioning result. The 3D point coordinate extraction module is used to determine the 3D point coordinates corresponding to the 2D feature points in the first image according to the first image, the low-precision positioning result corresponding to the first image, and the 3D model of the first area. The low-precision map construction module is used to construct a low-precision map that can provide feature point-based visual positioning services for the first area according to the first image, the low-precision positioning result corresponding to the first image, and the 3D point coordinates corresponding to the 2D feature points in the first image. The high-precision map construction module is used to perform global bundle adjustment on a low-precision map that can provide feature point-based visual positioning services for the first area according to the coordinates of key 3D points of the first area, so as to obtain a high-precision map that can provide feature point-based visual positioning services for the first area.

[0027] Optionally, the high-precision map construction module is specifically used to calibrate the 2D feature points in the first image constituting the low-precision map according to the key 3D point coordinates of the first area, and perform global bundle adjustment on the low-precision map. When performing global bundle adjustment on the low-precision map, the key 3D point coordinates calibrated in the first image constituting the low-precision map participate in the iteration but not in the update.

[0028] Optionally, when there is an existing high-precision map for the second area bordering the first area or the second area within the first area, the high-precision map construction module is specifically used to calibrate the 2D feature points in the first image constituting the low-precision map according to the key 3D point coordinates of the first area, and perform global bundle adjustment on the low-precision map and the existing high-precision map of the second area; wherein, when performing global bundle adjustment on the low-precision map and the existing high-precision map of the second area, the key 3D point coordinates calibrated in the first image constituting the low-precision map and the 3D point coordinates in the existing high-precision map of the second area participate in the iteration but not in the update.

[0029] In some implementations, when the first image is an image corresponding to an outdoor scene in the first area, the key 3D point coordinates of the first area are the absolute longitude and latitude coordinates of some building corners in the first area. When the area corresponding to the first image is covered by the first visual positioning service, the low-precision positioning result is the positioning result of the first visual positioning service; when the area corresponding to the first image is not covered by the first visual positioning service, the low-precision positioning result is the positioning result of the traditional positioning service.

[0030] Optionally, the traditional positioning service result is a positioning result of any one of the positioning methods including global positioning system GPS positioning, real-time kinematic differential method RTK positioning, magnetometer positioning, and network location provider NLP positioning.

[0031] Optionally, when the first image includes a skyline, the first visual positioning service result is a visual positioning service positioning result based on the skyline; when the first image does not include a skyline, the first visual positioning service result is a visual positioning service positioning result based on a contour line.

[0032] In some other embodiments, when the first image is an image corresponding to an indoor scene of the first area, the key 3D point coordinates of the first area are the absolute longitude and latitude coordinates of the 3D points marked in the CAD drawing of the first area, or the absolute longitude and latitude coordinates of some building corner points in the first area.

[0033] Optionally, the indoor positioning result is a positioning result of any one of Bluetooth positioning, WiFi positioning, geomagnetic positioning, and ultra-wideband UWB positioning.

[0034] Optionally, a 3D point coordinate extraction module is specifically used to extract 2D feature points in the first image; the 2D feature points in the first image, the camera intrinsic parameters corresponding to the first image, the low-precision positioning results corresponding to the first image, and the 3D model of the first area are input into a ray intersection algorithm to obtain the 3D point coordinates corresponding to the 2D feature points in the first image output by the ray intersection algorithm.

[0035] Optionally, when the first image is an image corresponding to an outdoor scene of the first area, the 3D model of the first area is an established CAD 3D model of the first area or a patch model generated according to a satellite image or an aerial image of the first area.

[0036] Optionally, when the first image is an image corresponding to an indoor scene of the first area, the 3D model of the first area is an established CAD 3D model of the first area.

[0037] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, and a memory for storing processor-executable instructions; when the processor is configured to execute the instructions, the electronic device implements the map construction method described in the first aspect.

[0038] The electronic device may be a map building server cluster in the cloud.

[0039] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having computer program instructions stored thereon; when the computer program instructions are executed by an electronic device, the electronic device implements the map construction method as described in the first aspect.

[0040] In a fifth aspect, an embodiment of the present application further provides a computer program product, comprising a computer readable code, which, when executed in an electronic device, enables the electronic device to implement the map construction method described in the first aspect.

[0041] The beneficial effects of the second to fifth aspects mentioned above can be referred to in the first aspect and will not be repeated here.

[0042] In a sixth aspect, an embodiment of the present application further provides a map construction system, which can be used to implement the map construction method described in the first aspect. The map construction system can include: a terminal device and a cloud. The terminal device is connected to the cloud via a network, and the cloud can also be called a cloud server or a cloud platform.

[0043] The terminal device can be an electronic device with a shooting function, such as: the terminal device can be a mobile terminal such as a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an AR / VR device, a laptop computer, a super mobile personal computer, a netbook, a personal digital assistant, etc., or it can also be a digital camera, a SLR camera / micro-single camera, a sports camera, a gimbal camera, a drone, and other professional shooting equipment. The embodiments of the present application do not limit the specific type of the terminal device.

[0044] The cloud can include a map construction server cluster, a VPS server cluster, an elastic file service, and a vector retrieval service cluster. Among them, the map construction server cluster can include at least a matching server and an adjustment server, which are used to realize map construction, such as: it can be used to implement the map construction method described in the first aspect. The VPS server can include multiple VPS containers, each VPS container can provide LOD-VPS and feature point-based VPS for the first area. The elastic file service is also a server cluster that can provide ultra-large space storage services and data extraction services, such as storing low-precision maps and high-precision maps in the map construction process, and can also provide online readable usage data for the VPS execution process. The vector retrieval service cluster is a VPS plug-in vector retrieval service that can quickly search for the data that the VPS wants to use in the elastic file service.

[0045] It should be understood that the description of technical features, technical solutions, beneficial effects or similar language in this application does not imply that all features and advantages can be realized in any single embodiment. On the contrary, it is understood that the description of features or beneficial effects means that specific technical features, technical solutions or beneficial effects are included in at least one embodiment. Therefore, the description of technical features, technical solutions or beneficial effects in this specification does not necessarily refer to the same embodiment. Furthermore, the technical features, technical solutions and beneficial effects described in the present embodiment can also be combined in any appropriate manner. Those skilled in the art will understand that the embodiment can be realized without one or more specific technical features, technical solutions or beneficial effects of a specific embodiment. In other embodiments, additional technical features and beneficial effects can also be identified in a specific embodiment that does not embody all embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 A schematic diagram of the process of the map construction method provided in the embodiment of the present application is shown;

[0047] Figure 2 A schematic diagram of a patch model provided in an embodiment of the present application is shown;

[0048] Figure 3 A schematic diagram of a skyline image provided by an embodiment of the present application is shown;

[0049] Figure 4 A schematic diagram of an interface of a map application provided in an embodiment of the present application is shown;

[0050] Figure 5 A schematic diagram of a process for determining 3D point coordinates provided in an embodiment of the present application is shown;

[0051] Figure 6 A schematic diagram of a process for constructing a low-precision map of a VPS based on feature points provided in an embodiment of the present application is shown;

[0052] Figure 7 A schematic diagram showing the matching results of image 1 and image 2 provided in an embodiment of the present application;

[0053] Figure 8 Another schematic diagram showing the matching results of image 1 and image 2 provided in an embodiment of the present application;

[0054] Fig. 9 A schematic diagram showing a co-viewing relationship provided by an embodiment of the present application is shown;

[0055] Fig.10 A schematic diagram of data that does not form a loop provided in an embodiment of the present application is shown;

[0056] Fig.11A schematic diagram of data constituting a loop provided in an embodiment of the present application is shown;

[0057] Fig.12 A schematic diagram of a process for obtaining a high-precision map provided in an embodiment of the present application is shown;

[0058] Fig.13 Another schematic diagram of a process for obtaining a high-precision map provided in an embodiment of the present application is shown;

[0059] Fig.14 Another schematic diagram of the process of the map construction method provided by the embodiment of the present application is shown;

[0060] Fig.15 A schematic diagram showing the composition of a map construction system provided in an embodiment of the present application is shown;

[0061] Fig.16 A schematic diagram of the structure of a terminal device provided in an embodiment of the present application is shown;

[0062] Fig.17 A schematic diagram of the structure of a map construction device provided in an embodiment of the present application is shown. DETAILED DESCRIPTION

[0063] With the advent of the 5G era, technologies such as augmented reality (AR) navigation and intelligent driving are developing rapidly, and these technologies rely on high-precision maps. Building high-precision maps is the key to the widespread application of technologies such as AR navigation and intelligent driving.

[0064] There are two main mapping solutions at present, namely, using satellite images to build maps and collecting map data on the ground to build maps. Among them, although the cost of building maps using satellite images is low, the accuracy of the constructed maps is also low. Although the method of building maps by collecting map data on the ground can build high-precision maps, the cost of collecting map data is high. For example, collecting map data by car collection, manual collection, etc. requires a lot of manpower and expensive equipment. In addition, the method of building maps by collecting map data on the ground cannot meet the update requirements of high-precision maps.

[0065] Against this background technology, the present application embodiment provides a map construction method that can significantly reduce the map acquisition cost while maintaining the high-precision positioning capability of the map, thereby achieving a low-cost and high-precision positioning effect. Taking the construction of a map of the first area as an example, the present application embodiment can include two application scenarios: constructing a map of an outdoor scene in the first area, and constructing a map of an indoor scene in the first area. The map construction method provided by the present application embodiment is exemplarily described below for these two application scenarios.

[0066] It should be noted that, in the description of this application, "at least one" means one or more, and "plurality" means two or more. The words "first", "second", etc. are only used to distinguish the description and are not used to specifically limit a certain feature. "And / or" is used to describe the association relationship between associated objects, indicating that three relationships may exist. For example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. The character " / " generally indicates that the related objects before and after are in an "or" relationship.

[0067] First, the application of the map construction method provided in the embodiment of the present application in constructing a map of an outdoor scene in a first area is described.

[0068] Figure 1 FIG. 1 is a flow chart of a map construction method provided in an embodiment of the present application. Figure 1 As shown, when constructing a map of an outdoor scene in the first area, the map construction method provided in the embodiment of the present application may include:

[0069] S101: Collect satellite images of a first area, and provide a skyline-based visual positioning service (VPS) for the first area according to the satellite images of the first area.

[0070] Exemplarily, the process of providing a skyline-based VPS for the first area according to the satellite image of the first area may include: after acquiring the satellite image of the first area, using a structure from motion (SFM) algorithm, a multi-view stereo (MVS) algorithm, a mesh construction algorithm, and an automatic white film generation algorithm to process the satellite image of the first area to obtain a mesh model of the first area. For example, Figure 2 A schematic diagram of a patch model provided in an embodiment of the present application is shown. The patch model established for the first region can be referred to Figure 2 It should be noted that when the first regions are different, the corresponding patch models are also different.

[0071] Then, for each location in the first area (the location refers to all locations that people can reach), the above-mentioned patch model can be rendered onto an image by rendering to obtain a skyline image corresponding to each location in the first area, and the skyline image includes the boundary between the building and the sky. For example, Figure 3 FIG. 1 is a schematic diagram of a skyline image provided by an embodiment of the present application. For a certain location in the first area, the rendered skyline image corresponding to the location can be as follows: Figure 3As shown, it includes the boundary between the building and the sky. Figure 3 The black filled area in the first area represents the building that can be seen from the perspective of the location, and the unfilled area represents the sky. The boundary between the filled area and the unfilled area is the boundary between the building and the sky. It should also be noted that different locations have different corresponding skyline images, and some locations that are close may have similar corresponding skyline images.

[0072] According to the above method, a large number of skyline images of the first area can be obtained. After obtaining a large number of skyline images of the first area, a skyline-based VPS can be provided for the first area. The specific principle of the skyline-based VPS is: for a certain image taken in the first area, the skyline-based VPS can use the semantic segmentation result of the image to extract the boundary line between the building and the sky in the image (i.e., the skyline), and then compare the extracted skyline in the image with the large number of skyline images of the first area obtained based on the satellite image of the first area, so as to determine the location corresponding to the skyline image with the highest similarity between the skyline and the boundary line between the sky and the building in the image, which is the skyline-based VPS positioning result corresponding to the image. Generally speaking, the skyline-based VPS positioning result obtained here is a low precision positioning result.

[0073] S102: Acquire a first image taken by a user when using a skyline-based VPS in a first area, and a skyline-based VPS positioning result corresponding to the first image.

[0074] Exemplarily, a user may hold a terminal device with a shooting function (such as a mobile phone). When the user uses the terminal device to perform skyline-based VPS positioning in a first area, the terminal device may first capture an image of the area where the user is located (in the embodiment of the present application, the image of the outdoor scene in the first area captured by the user is referred to as the first image), and then send the first image to a server that can provide skyline-based VPS (such as a VPS server). The VPS server may locate the first image according to the aforementioned skyline-based VPS principle, and obtain a skyline-based VPS positioning result corresponding to the first image.

[0075] That is, the first image acquired in S102 is an image taken by the user using a terminal device in the first area, and the skyline-based VPS positioning result corresponding to the first image is a positioning result obtained by the VPS server according to the first image.

[0076] Optionally, the terminal device described in the embodiment of the present application can be a mobile terminal such as a mobile phone, a tablet computer, a wearable device, a vehicle-mounted device, an augmented reality (AR) / virtual reality (VR) device, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), or the like; or it can be a professional shooting device such as a digital camera, a SLR camera / micro-single camera, a sports camera, a gimbal camera, a drone, etc. The embodiment of the present application does not limit the specific type of the terminal device.

[0077] Taking a mobile phone as an example, assuming that the area where the user is currently located is the first area, when the user needs to use the mobile phone for navigation in the first area, the user can click to start the map application installed on the mobile phone. Figure 4 A schematic diagram of the interface of a map application provided in an embodiment of the present application is shown. Figure 4 When the map application of the mobile phone is started, the mobile phone can call the camera to shoot the scene of the user's current area to obtain a first image ( Figure 4 The scene shown in is only an example.) For the first image, the skyline-based VPS can obtain its corresponding skyline-based VPS positioning result according to the first image.

[0078] It is understandable that, for a VPS based on the skyline, the first image taken by the user should include the boundary between the building and the sky in the area where the user is located. In addition, it should be noted that the user described in S102 may be one or more users, which is not limited here.

[0079] Optionally, the process of the VPS server obtaining the skyline-based VPS positioning result corresponding to the first image based on the first image may be: after obtaining the first image taken by the user when using the skyline-based VPS in the first area, the first image taken by the user and the camera intrinsics (camera intrinsics) and inertial measurement unit (IMU) information corresponding to the first image are input into the skyline-based VPS positioning algorithm, and the skyline-based VPS positioning algorithm may output the skyline-based VPS positioning result corresponding to the first image based on the aforementioned input information. The skyline-based VPS positioning result corresponding to the first image indicates the location of the mobile phone (i.e., the current location of the user).

[0080] It should be understood that the camera intrinsic parameters, IMU information, etc. corresponding to the first image refer to the intrinsic parameters, IMU information, etc. of the camera of the terminal device (such as a mobile phone) when the first image is taken. Therefore, the parameter information belonging to the first image (or metadata information of the first image) can be obtained together when the first image is acquired. In other words, when acquiring the first image taken by the user when using the skyline-based VPS in the first area, the camera intrinsic parameters, IMU information and other parameter information of the first image will be acquired together.

[0081] Exemplarily, the camera intrinsic parameters corresponding to the first image include four parameters: fx, fy, cx, and cy, where fx represents the x-axis camera focal length, fy represents the y-axis camera focal length, cx represents the x-axis coordinate of the principal point, and cy represents the y-axis coordinate of the principal point. The IMU information corresponding to the first image includes information on three angles, namely, the camera roll angle (roll), the camera pitch angle (pitch), and the camera yaw angle (yaw). The camera intrinsic parameters and IMU information corresponding to the first image are both floating point numbers. The skyline-based VPS positioning result corresponding to the first image is a 6-degree-of-freedom (DOF) positioning result, which is represented by a rotation matrix R and a position vector t. The rotation matrix R is a 3x3-dimensional matrix, which is a rotation matrix representing the user's orientation. The position vector t is 3 float-type numbers, representing the XY axis coordinates and altitude in the universal transverse mercator grid system (UTM) coordinate system.

[0082] According to the method described in S102, a large number of first images of the first area taken by the user and the VPS positioning results based on the skyline corresponding to the first images can be obtained. After obtaining a large number of first images and the VPS positioning results based on the skyline corresponding to the first images, a high-precision map of the outdoor scene of the first area can be constructed according to the steps described in S103-S105 below.

[0083] S103, determining 3D point coordinates corresponding to 2D keypoints in the first image according to the first image, the skyline-based VPS positioning result corresponding to the first image, and a 3D model of the first area.

[0084] The 3D model of the first area may be a level of detail (LOD) 3D model of the first area. Alternatively, the LOD 3D model may be a patch model of the first area obtained from a satellite image of the first area in the process of establishing the skyline-based VPS of the first area. Alternatively, the LOD 3D model may also be a CAD 3D model of the first area that has been established, which is not limited here.

[0085] For example, Figure 5 FIG. 2 shows a schematic diagram of a process for determining 3D point coordinates provided in an embodiment of the present application. Figure 5 As shown, according to the first image, the skyline-based VPS positioning result corresponding to the first image, and the 3D model of the first area, the process of determining the 3D point coordinates corresponding to the 2D feature points in the first image may include: firstly extracting features from the first image to obtain the 2D feature points in the first image, such as: extracting the 2D feature points in the first image through a convolutional neural network (CNN). Then, the 2D feature points in the first image, the camera intrinsic parameters corresponding to the first image, the skyline-based VPS positioning result corresponding to the first image, and the 3D model of the first area are input into a raycast algorithm, and the raycast algorithm may output the 3D point coordinates corresponding to the 2D feature points in the first image according to the aforementioned input information.

[0086] The principle of the raycast algorithm is to emit the 2D feature points in the first image in the form of rays according to the camera intrinsic parameters corresponding to the first image, and intersect the plane of the 3D model of the first area at a certain 3D point, thereby obtaining the 3D point coordinates corresponding to the 2D feature points in the first image. Generally speaking, the 3D point coordinates are 3 float numbers, and the specific meaning is the same as the aforementioned position vector. It can be understood that the aforementioned extracted 2D feature points are actually the 2D point coordinates of the 2D feature points in the first image.

[0087] S104, constructing a low-precision map capable of providing a feature point-based VPS for the first area based on the first image, the skyline-based VPS positioning result corresponding to the first image, and the 3D point coordinates corresponding to the 2D feature points in the first image.

[0088] For example, Figure 6 FIG. 1 is a flow chart of constructing a low-precision map of a VPS based on feature points provided in an embodiment of the present application. Figure 6As shown, in an embodiment of the present application, the process of constructing a low-precision map capable of providing a feature point-based VPS for the first area based on the first image, the skyline-based VPS positioning result corresponding to the first image, and the 3D point coordinates corresponding to the 2D feature points in the first image mainly includes four stages of data processing.

[0089] First, the first image, the camera intrinsic parameters corresponding to the first image, the skyline-based VPS positioning result corresponding to the first image, the 2D feature points in the first image (i.e., 2D point coordinates), the feature vectors of the 2D feature points in the first image (the feature vectors are feature descriptors), and the 3D point coordinates corresponding to the 2D feature points in the first image (i.e., the 3D point coordinates in the space of the first area) can be used as the first stage data (1st stage crowd sourcing data).

[0090] For the data of the first stage, a matching operation can be performed on the first image according to the feature vector of the 2D feature point in the first image. By performing the matching operation on the first image, the coordinates of the points at the same spatial position in the two first images can be found in the two first images.

[0091] For example, when matching the first image according to the feature vector of the 2D feature point in the first image, the 2D feature points extracted from the two first images and the feature descriptors (i.e., feature vectors) of the 2D feature points can be input into the matching algorithm, and the matching algorithm can output the matching results of the two first images according to the aforementioned input information, and the matching results are N pairs of numbers (N is an integer greater than 0), each pair of numbers representing the 2D feature points that form a matching pair in the two input first images. For example, [1, 20] means that the feature point 1 in the first first image and the feature point 20 in the second first image form a matching pair.

[0092] Take the two first images as image 1 and image 2 as an example, Figure 7 FIG. 1 is a schematic diagram showing the matching results of image 1 and image 2 provided in an embodiment of the present application. Figure 7 As shown, Figure 7 Each line between image 1 and image 2 in the image 1 and image 2 connects two pixels in image 1 and image 2 corresponding to the points in the same spatial position. Matching image 1 and image 2 is to find the coordinates of the points in the same spatial position in image 1 and image 2 in image 1 and image 2 respectively.

[0093] Optionally, when performing a matching operation on the first image in the data of the first stage, all the first images may be matched pairwise, or for any first image, another first image that is closest to this first image (i.e., most similar, such as having the shortest cosine distance) may be selected for matching, and there is no limitation here.

[0094] In addition, generally speaking, when matching two first images, there may be a large number of false matches, so it is also necessary to perform a rejection operation on the matching results to eliminate false matches in the matching results. For example, the rejection operation can be performed using a random sample consensus (RANSAC) algorithm as a rejection algorithm.

[0095] For example, for Figure 7 The matching results of image 1 and image 2 shown in the figure may contain a large number of false matches. Therefore, it is necessary to use the random sampling consensus algorithm to Figure 7 The matching results shown are removed to remove the wrong matches. Figure 8 Another schematic diagram showing the matching results of image 1 and image 2 provided in an embodiment of the present application is shown. Figure 8 As shown, Figure 7 After the matching results of the images 1 and 2 shown in the figure are eliminated, a large number of wrong matches can be eliminated.

[0096] After matching and eliminating the data of the first stage, the common view relationship between the first images in the data of the first stage is constructed. As described above, it can be known that for two first images with a common view relationship, a corresponding matching pair represents the same point in the space of the first area, so the matching result represents the common view of the two first images, such as the size of the area where the same scene is seen in the two first images.

[0097] For example, Fig. 9 FIG. 2 is a schematic diagram showing a co-viewing relationship provided by an embodiment of the present application. Fig. 9 As shown, Fig. 9 The small circles in the figure represent the 2D feature points in the first image; the five small triangles c1, c2, c3, c4, and c5 represent five cameras (such as the camera module of the mobile phone used to take the first image); the lines between the small circles and the small triangles represent the relationship between the 2D feature points seen by the five cameras. For example, if c1 and c2 see certain scenes (2D feature points) together, these scenes can form a common viewing relationship.

[0098] Optionally, after matching and eliminating the data in the first stage and establishing the common view relationship between the first images, the redundant key frames in the first images can be eliminated. For example, if the distance between two first images with a common view relationship is too close (i.e., the image difference is too small), data waste will be caused. Therefore, the two first images whose distance is less than a preset distance threshold (the size is not limited) can be eliminated. The distance between the two first images can be a cosine distance, a Euclidean distance, etc.

[0099] After constructing the co-viewing relationship between the first images in the first stage data, the co-viewing relationship between the first images is added to the first stage data to form the second stage data (2stage crowd sourcing data). Figure 6 As shown, the data of the second stage is based on the data of the first stage and adds the common viewing relationship between the first images.

[0100] For the data in the second stage, loop detection can be performed to obtain the loop co-viewing relationship between the first images. Loop detection refers to detecting whether there is end-to-end data in the data of the second stage. For example, as the data in the second stage increases, the co-viewing image will become larger and larger. By extracting the global vector of the first image in the data of the second stage and matching it with the deep learning descriptor, it is possible to detect whether there is a loop in the data of the second stage.

[0101] For example, Fig.10 A schematic diagram of data that does not form a loop provided in an embodiment of the present application is shown. Fig.11 A schematic diagram of data constituting a loop provided in an embodiment of the present application is shown. Fig.10 and Fig.11 The scattered small circles in the figure represent data, such as Fig.10 As shown in , if a loop is not formed, the data ends will not be connected. Fig.11 As shown, if a loop is formed, the data is connected end to end in terms of geographical location.

[0102] After loop closure detection is performed on the data of the second stage to obtain the loop closure common view relationship between the first images, the loop closure common view relationship between the first images can be added to the data of the second stage to form the data of the third stage (3stage crowd sourcing data).

[0103] For the data of the third stage, a local bundle adjustment process can be performed to obtain optimized 3D point coordinates. Among them, bundle adjustment can also be called bundle adjustment. For example, the data of the third stage can be input into the local bundle adjustment algorithm, and the local bundle adjustment algorithm can optimize the VPS positioning results based on the skyline corresponding to the first image in the data of the third stage, and the 3D point coordinates corresponding to the 2D feature points in the first image, to obtain the optimized VPS positioning results and the optimized 3D point coordinates. In other words, the local bundle adjustment algorithm corrects and optimizes the VPS positioning results based on the skyline corresponding to the first image in the data of the third stage, and the 3D point coordinates corresponding to the 2D feature points in the first image.

[0104] In view of this, in the embodiment of the present application, before being processed by the local beam algorithm, the skyline-based VPS positioning result corresponding to the first image can be called the initial pose of the first image, and the 3D point coordinates corresponding to the 2D feature points in the first image can be called the initial 3D point coordinates. After being processed by the local beam algorithm, the skyline-based VPS positioning result corresponding to the first image can be called the optimized pose of the first image, and the 3D point coordinates corresponding to the 2D feature points in the first image can be called the optimized 3D point coordinates.

[0105] Please continue to refer to Figure 6 As shown, after the local bundle adjustment is performed on the data of the third stage, the data of the fourth stage (4stage crowd sourcing data) can be generated according to the result of the local bundle adjustment. The data of the fourth stage includes: the first image, the camera intrinsic parameters corresponding to the first image, the optimized VPS positioning result based on the skyline, the 2D feature points in the first image, the feature vectors of the 2D feature points in the first image, and the optimized 3D point coordinates.

[0106] The data of the fourth stage can be converted into the format required by the feature point-based VPS to obtain a low-precision map that can provide the feature point-based VPS for the first area. The low-precision map is a low-precision local map of the first area.

[0107] Compared with the aforementioned skyline-based VPS, the characteristic of the skyline-based VPS is that it uses the skyline features of the building for positioning, so the user must capture the boundary between the building and the sky to achieve positioning. The feature point-based VPS does not have too many constraints, and the support scope of positioning is not limited to images that must capture the skyline. As long as there are locations with feature points, positioning can be performed, such as: only taking images of the building itself can also execute VPS. Therefore, the feature point-based VPS can not only obtain the low-precision positioning results corresponding to the first image, but also expand the scope of positioning services, improve the positioning success rate, and expand the usage scenarios, so that more users can perform positioning at these locations to provide more images.

[0108] S105. According to the coordinates of the key 3D points of the first area, a global bundle adjustment is performed on the low-precision map that can provide a VPS based on feature points for the first area, so as to obtain a high-precision map that can provide a VPS based on feature points for the first area.

[0109] The key 3D points of the first area may be some 3D points in the first area whose precise coordinates are easily obtained. For example, the first area may be a city or a part of a city. In this case, the key 3D points of the first area may be some urban public control points in the first area. For example, the urban public control points may include: corner points and feature points of buildings in the first area.

[0110] Fig.12 The following is a schematic diagram of the process of obtaining a high-precision map provided in an embodiment of the present application. Fig.12 As shown, according to the key 3D point coordinates of the first area, the process of performing global bundle adjustment on the low-precision map that can provide a VPS based on feature points for the first area may include: first, using the key 3D point coordinates of the first area to accurately calibrate the 2D feature points in the first image that constitutes the low-precision map. For example, assuming that a certain first image includes building A, the 3D point coordinates of the corner points of building A can be used to accurately calibrate the 2D feature points corresponding to the corner points of building A in the first image. Then, the low-precision map that can provide a VPS based on feature points for the first area is input into the global bundle adjustment algorithm for processing. At this time, the key 3D point coordinates of the first area are calibrated in the first image that constitutes the low-precision map. When the global bundle adjustment algorithm performs global bundle adjustment on the low-precision map that can provide a VPS based on feature points for the first area, the key 3D point coordinates of the first area calibrated in the first image participate in the iteration but not in the update.

[0111] That is, when performing global bundle adjustment, the key 3D point coordinates of the first area are involved in calculating errors so that the bundle adjustment can be performed, but the key 3D points of the first area are equivalent to static points in the bundle adjustment process and will not participate in the update. The 3D points in the low-precision map will always be pulled toward the key 3D points of the first area, so that the 3D points in the low-precision map will exist in the same coordinate system as the key 3D points of the first area. In this way, the key 3D point coordinates of the first area can constrain the low-precision map to an accurate scale and position, restore the accurate absolute scale and absolute pose of the low-precision map, and the global bundle adjustment algorithm will output a high-precision map that can provide a feature point-based VPS for the first area.

[0112] Optionally, in some embodiments, when performing global bundle adjustment on a low-precision map that can provide a feature point-based VPS for the first area, an iterative closest point (ICP) algorithm can also be used to align the 3D points in the low-precision map with the key 3D points of the first area to a precise coordinate system, without limitation here.

[0113] It should be understood that the low-precision map and high-precision map described in the embodiments of the present application exist in relative terms, which means that the accuracy of the high-precision map obtained in S105 will be greater than that of the low-precision map obtained in S104.

[0114] In addition, in an embodiment of the present application, before executing the map construction method, it is also necessary to obtain the key 3D point coordinates of the first area. For example, the precise 3D point positions (i.e., absolute longitude and latitude coordinates) of some building corners in the first area can be obtained by surveying and mapping equipment such as a total station, or by purchasing drawings, purchasing information, etc., to obtain a city level public control point group (city level public control points) in the first area. The city level public control point group includes multiple city level public control points in the first area. The 3D coordinates of these city level public control points are the key 3D point coordinates of the first area.

[0115] Optionally, in an embodiment of the present application, the method of accurately calibrating the 2D feature points in the first image constituting the low-precision map using the key 3D point coordinates of the first area may include manual calibration and automatic calibration. Among them, manual calibration refers to: manually marking the key 3D point coordinates corresponding to some 2D feature points in the image on the first image. For example, the user can directly use a portable device such as a mobile phone or a tablet computer to calibrate the corresponding key 3D point coordinates for the corner points containing the building in the first image. Automatic calibration refers to: using a 2Stage target detection network (such as HardNet) to detect the key points (such as building corner points) in the first image, and then using a two-way matching method for the key points in the detected first image and the key 3D points of the aforementioned first area, to calibrate the corresponding key 3D point coordinates for the key points in the detected first image. Exemplarily, when matching, matching can be performed based on the feature vectors of the key points in the first image and the key 3D points in the first area, which will not be repeated.

[0116] As described above, the map construction method provided in the embodiment of the present application can generate a low-precision map that can provide a VPS based on feature points for the first area based on a large number of images (i.e., crowdsourced images) of the first area taken by the user when using a skyline-based VPS in the first area, the skyline-based VPS positioning results corresponding to these images, and the 3D model of the first area, and then further combine the coordinates of some key 3D points in the first area to perform bundle adjustment on the low-precision map, so as to construct a high-precision map that can provide a VPS based on feature points for the first area. In the map construction process, there is no need to spend extra manpower to collect map data, only the ability to provide a skyline-based VPS for the first area and the ability to obtain a 3D model of the first area are required. The skyline-based VPS can be constructed through satellite images of the first area, and the 3D model of the first area can also be a patch model generated based on satellite images of the first area. Therefore, it can be seen that the map construction method only needs to purchase satellite images from relevant suppliers, which can save a lot of collection manpower and collection costs. At the same time, since the map construction method can combine the coordinates of some key 3D points in the first area to perform bundle adjustment on the low-precision map, it can greatly reduce the cost of map acquisition while maintaining the high-precision positioning capability of the map, thereby achieving the goal of building a low-cost and high-precision map.

[0117] In addition, it can be seen that in the embodiment of the present application, the map construction process and the process of the user using the VPS are independent of each other. The embodiment of the present application can build the map while providing the VPS to the user, and the VPS service can be provided not only after the map construction is completed.

[0118] Optionally, when constructing a map for the first area, there may be existing high-precision maps for other areas bordering the first area, or some sub-areas in the first area (hereinafter such areas are referred to as second areas). The high-precision map can be constructed by map data collected by car collection, manpower collection, etc., or it can be constructed according to the map construction method provided in the embodiment of the present application. When there is an existing high-precision map for the second area, in the map construction method, when performing a global bundle adjustment on the low-precision map that can provide a feature point-based VPS for the first area based on the key 3D point coordinates of the first area, the existing high-precision map of the second area can also be taken into account. For example, Fig.13 Another schematic diagram of the process of obtaining a high-precision map provided in an embodiment of the present application is shown. Fig.13 As shown, after the 2D feature points in the first image constituting the low-precision map are accurately calibrated using the key 3D point coordinates of the first area, the low-precision map that can provide the feature point-based VPS for the first area and the existing high-precision map of the second area can be input into the global bundle adjustment algorithm for processing. The specific bundle adjustment principle is similar to that described in S105 above and will not be repeated here. The special thing is that, like the key 3D coordinates of the first area, during the bundle adjustment process, the 3D point coordinates in the high-precision map participate in the adjustment but not in the update. In this way, the absolute scale and absolute pose of the low-precision map can be corrected using the accurate map points and images in the existing high-precision map. In other words, since the existing high-precision map has a certain intersection with the low-precision map, this intersection can be used to pull the low-precision map to a coordinate system aligned with the high-precision map.

[0119] Since the map construction method provided in the embodiment of the present application can be combined with the existing high-precision map to perform global bundle adjustment on the low-precision map that can provide a feature point-based VPS for the first area, the embodiment of the present application can continuously acquire new first images taken by the user and update the map while the user is using the high-precision map based on the feature point VPS constructed by the map construction method.

[0120] Optionally, in other embodiments of the present application, when constructing a map of the first area, if there are existing high-precision maps for other areas bordering the first area or some sub-areas in the first area, S105 may not use the key 3D point coordinates of the first area to calibrate the first image constituting the low-precision map. Instead, the existing high-precision map and the low-precision map obtained in S104 can be directly subjected to global bundle adjustment to obtain a high-precision map that can provide feature point-based VPS for the first area.

[0121] Optionally, the satellite image of the first area described in the embodiment of the present application may also be replaced by an aerial image of the first area, and a skyline-based VPS may also be provided for the first area based on the aerial image of the first area, and the present application is not limited to this. It can be understood that the principle of providing a skyline-based VPS for the first area based on the aerial image of the first area is the same as the principle of providing a skyline-based VPS for the first area based on the satellite image of the first area, and reference may be made to the description in the aforementioned embodiment, which will not be repeated here.

[0122] In addition, it should be noted that the VPS based on the skyline described in the above-mentioned embodiment of the present application is only an exemplary description. In some other embodiments, a VPS based on contour lines can also be provided for the first area based on satellite images or aerial images. For example, the satellite image or aerial image of the first area can be processed by the SFM algorithm, the MVS algorithm, the mesh construction algorithm, and the automatic white film generation algorithm to obtain a patch model of the first area. Then, for each location in the first area, the patch model can be rendered onto the image by rendering to obtain a contour line image corresponding to each location in the first area, and the contour line image includes the contour line of the building. In this way, a large number of contour line images of the first area can be obtained. After obtaining a large number of contour line images of the first area, a VPS based on contour lines can be provided for the first area. The specific principle of the VPS based on contour lines is similar to that of the VPS based on the skyline, and will not be repeated. It should be understood that when a VPS based on contour lines is provided for the first area in S101, the subsequent steps in the map construction method are all implemented by the VPS positioning result based on contour lines corresponding to the first image, and the rest is the same as the above-mentioned embodiment, and will not be repeated one by one.

[0123] In some other embodiments, a VPS based on the skyline and a VPS based on the contour line can also be provided for the first area at the same time. Optionally, since the VPS based on the skyline and the VPS based on the contour line are both constructed according to the LOD3D model of the first area, the VPS based on the skyline and the VPS based on the contour line can also be referred to as LOD-VPS. When the VPS based on the skyline and the VPS based on the contour line are provided for the first area at the same time in S101, in the subsequent steps of the map construction method, the VPS positioning result based on the skyline corresponding to the first image can be obtained, and the VPS positioning result based on the contour line corresponding to the first image (both can be referred to as LOD-VPS positioning result) can be obtained, and map construction is realized. For example, when the first image contains a skyline, the VPS positioning result based on the skyline corresponding to the first image is obtained. When the first image does not contain a skyline, the VPS positioning result based on the contour line corresponding to the first image is obtained. The specific principle of map construction can also refer to the above-mentioned embodiment, and will not be repeated.

[0124] Optionally, when a map of the first area is constructed using the map construction method provided in an embodiment of the present application, there may be some areas in the first area that cannot be covered by LOD-VPS. For example, for some areas in urban canyon conditions, LOD-VPS may not be covered. For this part of the area not covered by LOD-VPS, when using the map construction method, the LOD-VPS positioning result corresponding to the first image can be replaced by the positioning result of traditional positioning services such as global positioning system (GPS) positioning, real-time kinematic (RTK) positioning, magnetometer positioning, network location provider (NLP) positioning (such as Wifi network positioning), etc.

[0125] That is, in the embodiment of the present application, for the first image taken by the user using a mobile phone, when obtaining the low-precision positioning result corresponding to the first image, if the first area where the user is located covers the LOD-VPS, the low-precision positioning result corresponding to the first image can be determined by the LOD-VPS. If the first area where the user is located does not cover the LOD-VPS, the low-precision positioning result corresponding to the first image can be determined by traditional GPS positioning, RTK positioning, magnetometer positioning, NLP positioning, etc.

[0126] The above exemplary description is made of the application of the map construction method provided in the embodiment of the present application to construct a map of an outdoor scene of a first area. The following exemplary description is made of the application of the map construction method provided in the embodiment of the present application to construct a map of an indoor scene of a first area.

[0127] Fig.14 Another schematic diagram of the map construction method provided in the embodiment of the present application is shown. Fig.14 As shown, when constructing a map of an indoor scene of a first area, the map construction method provided in the embodiment of the present application may include:

[0128] S1401. Obtain a first image taken by a user in a first area, and an indoor positioning result corresponding to the first image.

[0129] Exemplarily, the indoor scenes in the first area may include large shopping malls, exhibition halls, large garages, etc. The first image may be an image taken by a user using a terminal device with a shooting function (such as a mobile phone) in the first area (it can be understood that the first image here is an image corresponding to the indoor scene). For example, when a user needs to use a mobile phone for navigation in the first area, he can click to start the map application installed on the mobile phone. When the map application of the mobile phone is started, the mobile phone can call the camera to shoot the scene of the user's current area to obtain the first image.

[0130] For each first image, the indoor positioning result of the terminal device when the first image is taken can also be obtained, that is, the indoor positioning result corresponding to the first image. For example, when the terminal device takes the first image, any one of the positioning methods such as Bluetooth positioning, WiFi positioning, geomagnetic positioning, and ultra-wideband (UWB) positioning can be used to obtain the location information of the terminal device as the indoor positioning result corresponding to the first image.

[0131] Optionally, the first image photographed by the user in the first area described in S1401 above may also be taken by the user when using a skyline-based or contour-based VPS based on the first area (please refer to the description in the above embodiments for details). However, since the skyline-based and contour-based VPS cannot cover the indoor scenes of the first area, the indoor positioning result corresponding to the first image will be obtained in the embodiment of the present application for subsequent mapping.

[0132] S1402: Determine 3D point coordinates corresponding to 2D feature points in the first image according to the first image, the indoor positioning result corresponding to the first image, and the 3D model of the first area.

[0133] The 3D model of the first area may be a built CAD 3D model of the first area. It is understandable that for indoor scenes such as large shopping malls, exhibition halls, and large garages, the corresponding CAD 3D models can be obtained directly from the architect or designer.

[0134] Exemplarily, the process of determining the 3D point coordinates corresponding to the 2D feature points in the first image according to the first image, the indoor positioning result corresponding to the first image, and the 3D model of the first area is the same as the above Figure 5 The process of determining the 3D point coordinates corresponding to the 2D feature points in the first image according to the first image, the skyline-based VPS positioning result corresponding to the first image, and the 3D model of the first area is similar. For example, the first image is first feature extracted to obtain the 2D feature points in the first image. Then, the 2D feature points in the first image, the camera intrinsic parameters corresponding to the first image, the indoor positioning result corresponding to the first image, and the 3D model of the first area are input into the raycast algorithm, and the raycast algorithm can output the 3D point coordinates corresponding to the 2D feature points in the first image based on the aforementioned input information. The principle of the raycast algorithm will not be repeated. It can be understood that the aforementioned extracted 2D feature points are essentially the 2D point coordinates of the 2D feature points in the first image.

[0135] S1403. Construct a low-precision map capable of providing a feature point-based VPS for the first area based on the first image, the indoor positioning result corresponding to the first image, and the 3D point coordinates corresponding to the 2D feature points in the first image.

[0136] The specific process of S1403 can refer to the process described in S104 above, and a low-precision map capable of providing a VPS based on feature points for the first area can be constructed through four stages of data processing. The only difference is the difference in input parameters, such as: replacing the first image with the first image, replacing the skyline-based VPS positioning result corresponding to the first image with the indoor positioning result corresponding to the first image, and replacing the 3D point coordinates corresponding to the 2D feature points in the first image with the 3D point coordinates corresponding to the 2D feature points in the first image. However, the algorithms used in the data processing process of each stage (such as: matching algorithm, RANSAC outlier removal algorithm, bundle adjustment algorithm, etc.) are the same as those in the aforementioned embodiment and will not be repeated.

[0137] In the embodiment of the present application, before being processed by the local beam algorithm, the indoor positioning result corresponding to the first image can be referred to as the initial pose of the first image, and the 3D point coordinates corresponding to the 2D feature points in the first image can be referred to as the initial 3D point coordinates. After being processed by the local beam algorithm, the indoor positioning result corresponding to the first image can be referred to as the optimized pose of the first image, and the 3D point coordinates corresponding to the 2D feature points in the first image can be referred to as the optimized 3D point coordinates.

[0138] S1404. According to the coordinates of the key 3D points of the first area, a global bundle adjustment is performed on the low-precision map that can provide a VPS based on feature points for the first area, so as to obtain a high-precision map that can provide a VPS based on feature points for the first area.

[0139] Optionally, when the absolute longitude and latitude coordinates of some 3D points are marked in the CAD drawing corresponding to the indoor scene of the first area (such as the above-mentioned CAD model), these 3D points can be directly used as the key 3D points of the first area. When the absolute longitude and latitude coordinates of any 3D points are not marked in the CAD drawing corresponding to the indoor scene of the first area (such as the above-mentioned CAD model), the corner points of other buildings seen at the entrance can be used as the key 3D points of the first area. The coordinates of these key 3D points are easy to obtain, such as: directly obtaining the absolute longitude and latitude coordinates from the CAD drawing, or obtaining the 3D coordinates of the corner points of other buildings seen at the entrance in the same manner as the aforementioned embodiment.

[0140] After obtaining the key 3D point coordinates of the first area, the key 3D point coordinates of the first area can be used to form the 2D feature points in the first image of the low-precision map obtained in S1403 for precise calibration. Then, the low-precision map calibrated with the key 3D point coordinates of the first area is input into the global bundle adjustment algorithm for processing, and a high-precision map that can provide a VPS based on feature points for the first area can be obtained. The calibration process of the key 3D points and the principle of the global bundle adjustment are the same as those in the aforementioned embodiment, and will not be repeated here.

[0141] Optionally, similar to the aforementioned embodiment, for the indoor scene of the first area, when there is an existing high-precision map of the indoor scenes of other areas bordering the first area or the indoor scenes of some sub-areas in the first area (hereinafter such areas are also referred to as second areas), the high-precision map can also be compared with the low-precision map with the coordinates of the key 3D points of the first area calibrated. Figure 1 Perform global bundle adjustment. During the bundle adjustment process, the 3D point coordinates in the high-precision map participate in the adjustment but not in the update. The high-precision map can be constructed by map data collected by car collection, manual collection, etc., or it can be constructed according to the map construction method provided in the embodiment of the present application.

[0142] Optionally, based on the map construction method described in the foregoing embodiment, the embodiment of the present application further provides a map construction system for implementing the map construction method, and the map construction system can be used to implement map construction of the first area (including indoor scenes and outdoor scenes).

[0143] Fig.15 FIG. 1 shows a schematic diagram of the composition of the map construction system provided in the embodiment of the present application. Fig.15 As shown, the map construction system may include: a terminal device and a cloud. The terminal device is an electronic device with a shooting function, such as a mobile phone. The specific form of the terminal device can refer to the above-mentioned embodiment and will not be repeated. The terminal device is connected to the cloud through a network, and the cloud can also be called a cloud server or cloud platform.

[0144] The cloud can include a map construction server cluster, a VPS server cluster, a scalable file service (SFS), and a vector retrieval service cluster. Among them, the map construction server cluster can at least include a matching server and an adjustment server, which are used to realize map construction, such as: it can be used to realize the steps S105, S1404, etc. described in the aforementioned embodiment. The VPS server can include multiple VPS containers (two are given as examples in the figure), each VPS container can provide LOD-VPS and VPS based on feature points for the first area, and VPS based on feature points can also be called global visual positioning service (global visual positioning service, GVPS). For example, a Web server, LOD-VPS algorithm, GVPS algorithm, etc. can be configured in the VPS container, and the Web server is used to schedule positioning requests from terminal devices, and according to the image (such as the first image) taken by the terminal device, the LOD-VPS algorithm, the GVPS algorithm, etc. are called for positioning. SFS is also a server cluster that can provide ultra-large space storage services and data extraction services, such as low-precision maps, high-precision maps, etc. in the map construction process described in the aforementioned embodiment can be stored, and it can also provide online readable usage data for the VPS execution process. The vector search service cluster is a VPS plug-in vector search service that can quickly search for the data that the VPS wants to use in SFS.

[0145] It should be noted that Fig.15 The map construction system shown is only an example, and the present application does not limit the composition architecture of the map construction system.

[0146] Optionally, Fig.15A terminal device is given as an example. However, it should be understood that the terminal device in the end-cloud collaborative system may include one or more terminal devices, and the multiple terminal devices may be the same or different, which is not limited here.

[0147] For example, taking the terminal device as a mobile phone, Fig.16 The schematic diagram of the structure of the terminal device provided in the embodiment of the present application is shown. Fig.16 As shown, the mobile phone may include a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a battery 342, an antenna 1, an antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, an earphone interface 370D, a sensor module 380, a button 390, a motor 391, an indicator 392, a camera 393, a display screen 394, and a subscriber identification module (SIM) card interface 395, etc.

[0148] The processor 310 may include one or more processing units, for example, the processor 310 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0149] The controller can be the nerve center and command center of the mobile phone. The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of fetching and executing instructions.

[0150] The processor 310 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 310 is a cache memory. The memory may store instructions or data that the processor 310 has just used or cyclically used. If the processor 310 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 310, and thus improves the efficiency of the system.

[0151] In some embodiments, the processor 310 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM interface, and / or a USB interface, etc.

[0152] The external memory interface 320 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile phone. The external memory card communicates with the processor 310 through the external memory interface 320 to implement a data storage function. For example, files such as music and videos can be stored in the external memory card.

[0153] The internal memory 321 can be used to store computer executable program codes, which include instructions. The processor 310 executes various functional applications and data processing of the mobile phone by running the instructions stored in the internal memory 321. The internal memory 321 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the mobile phone (such as image data, a phone book, etc.), etc. In addition, the internal memory 321 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc.

[0154] The charging management module 340 is used to receive charging input from the charger. While the charging management module 340 is charging the battery 342, it can also power the mobile phone through the power management module 341. The power management module 341 is used to connect the battery 342, the charging management module 340, and the processor 310. The power management module 341 can also receive input from the battery 342 to power the mobile phone.

[0155] The wireless communication function of the mobile phone can be realized by antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, modulation and demodulation processor and baseband processor. Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the mobile phone can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antenna. For example, antenna 1 can be reused as a diversity antenna of a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0156] The mobile phone can realize audio functions such as music playing and recording through the audio module 370, the speaker 370A, the receiver 370B, the microphone 370C, the earphone interface 370D, and the application processor.

[0157] The sensor module 380 may include a pressure sensor 380A, a gyroscope sensor 380B, an air pressure sensor 380C, a magnetic sensor 380D, an acceleration sensor 380E, a distance sensor 380F, a proximity light sensor 380G, a fingerprint sensor 380H, a temperature sensor 380J, a touch sensor 380K, an ambient light sensor 380L, a bone conduction sensor 380M, and the like.

[0158] The camera 393 may include multiple types. For example, the camera 393 may include a telephoto camera, a wide-angle camera, or an ultra-wide-angle camera with different focal lengths. Among them, the telephoto camera has a small field of view and is suitable for photographing objects in a small range at a distance; the wide-angle camera has a larger field of view; the ultra-wide-angle camera has a larger field of view than the wide-angle camera and can be used to photograph large-scale images such as panoramas. In some embodiments, the telephoto camera with a smaller field of view can be rotated so that objects in different ranges can be photographed.

[0159] The mobile phone can capture RAW images through the camera 393. For example, the specific structure of the camera 393 can refer to the following. Figure 1The camera module described in at least includes a lens and a sensor. When taking a photo or shooting a video, the shutter is opened, and light can be transmitted to the sensor through the lens of the camera 393. The sensor can convert the light signal passing through the lens into an electrical signal, and then perform A / D conversion on the electrical signal to output a corresponding digital signal. The digital signal is the RAW image. Subsequently, the RAW image can be converted into an image visible to the naked eye by performing subsequent RAW domain processing, ISP processing, and YUV domain processing on the RAW image.

[0160] In one possible design, the photosensitive element of the sensor may be a charge coupled device (CCD), and the sensor also includes an A / D converter. In another possible design, the photosensitive element of the sensor may be a complementary metal-oxide-semiconductor (CMOS).

[0161] The display screen 394 is used to display images, videos, etc. The display screen 394 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light emitting diode or an active-matrix organic light emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the mobile phone may include 1 or N display screens 394, where N is a positive integer greater than 1. For example, the display screen 394 can be used to display a camera interface, a photo playback interface, etc.

[0162] The mobile phone implements display functions through a GPU, a display screen 394, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 394 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 310 may include one or more GPUs that execute program instructions to generate or change display information.

[0163] Understandably, Fig.16 The structure shown does not constitute a specific limitation on the mobile phone. In some embodiments, the mobile phone may also include Fig.16More or fewer components shown, or combining some components, or splitting some components, or arranging components differently, etc. Alternatively, Fig.16 Some of the components shown may be implemented in hardware, software, or a combination of software and hardware.

[0164] In addition, when the terminal device is other tablet computers, wearable devices, vehicle-mounted devices, AR / VR devices, laptops, UMPCs, netbooks, PDAs and other mobile terminals, or digital cameras, SLR cameras / micro-single cameras, sports cameras, PTZ cameras, drones and other professional shooting equipment, the specific structures of these other terminal devices can also be referred to. Fig.16 As shown. For example, other terminal devices may be Fig.16 Components are added or reduced on the basis of the given structure, which will not be described in detail here.

[0165] Corresponding to the map construction method described in the above embodiment, the present application embodiment also provides a map construction device, which can be applied to the cloud. The functions of the device can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules or units corresponding to the above functions. For example, Fig.17 FIG. 1 is a schematic diagram showing the structure of a map construction device provided in an embodiment of the present application. Fig.17 As shown, the map construction device may include: an acquisition module 1701, a 3D point coordinate extraction module 1702, a low-precision map construction module 1703, and a high-precision map construction module 1704.

[0166] Among them, the acquisition module 1701 is used to obtain the first image taken by the user when using the first visual positioning service in the first area, and the low-precision positioning result corresponding to the first image. The first visual positioning service includes a skyline-based visual positioning service and / or a contour-based visual positioning service; wherein, when the first image is an image corresponding to an outdoor scene of the first area, the low-precision positioning result is a positioning result of the first visual positioning service or a positioning result of a traditional positioning service; when the first image is an image corresponding to an indoor scene of the first area, the low-precision positioning result is an indoor positioning result. The 3D point coordinate extraction module 1702 is used to determine the 3D point coordinates corresponding to the 2D feature points in the first image according to the first image, the low-precision positioning result corresponding to the first image, and the 3D model of the first area. The low-precision map construction module 1703 is used to construct a low-precision map that can provide a feature point-based visual positioning service for the first area according to the first image, the low-precision positioning result corresponding to the first image, and the 3D point coordinates corresponding to the 2D feature points in the first image. The high-precision map construction module 1704 is used to perform global bundle adjustment on the low-precision map that can provide feature point-based visual positioning services for the first area according to the key 3D point coordinates of the first area, so as to obtain a high-precision map that can provide feature point-based visual positioning services for the first area.

[0167] Optionally, the high-precision map construction module 1704 is specifically used to calibrate the 2D feature points in the first image constituting the low-precision map according to the key 3D point coordinates of the first area, and perform global bundle adjustment on the low-precision map. When performing global bundle adjustment on the low-precision map, the key 3D point coordinates calibrated in the first image constituting the low-precision map participate in the iteration but not in the update.

[0168] Optionally, when there is an existing high-precision map for the second area bordering the first area or the second area within the first area, the high-precision map construction module 1704 is specifically used to calibrate the 2D feature points in the first image constituting the low-precision map according to the key 3D point coordinates of the first area, and perform global bundle adjustment on the low-precision map and the existing high-precision map of the second area; wherein, when performing global bundle adjustment on the low-precision map and the existing high-precision map of the second area, the key 3D point coordinates calibrated in the first image constituting the low-precision map and the 3D point coordinates in the existing high-precision map of the second area participate in the iteration but not in the update.

[0169] In some implementations, when the first image is an image corresponding to an outdoor scene in the first area, the key 3D point coordinates of the first area are the absolute longitude and latitude coordinates of some building corners in the first area. When the area corresponding to the first image is covered by the first visual positioning service, the low-precision positioning result is the positioning result of the first visual positioning service; when the area corresponding to the first image is not covered by the first visual positioning service, the low-precision positioning result is the positioning result of the traditional positioning service.

[0170] Optionally, the traditional positioning service result is a positioning result of any one of the positioning methods including global positioning system GPS positioning, real-time kinematic differential method RTK positioning, magnetometer positioning, and network location provider NLP positioning.

[0171] Optionally, when the first image includes a skyline, the first visual positioning service result is a visual positioning service positioning result based on the skyline; when the first image does not include a skyline, the first visual positioning service result is a visual positioning service positioning result based on a contour line.

[0172] In some other embodiments, when the first image is an image corresponding to an indoor scene of the first area, the key 3D point coordinates of the first area are the absolute longitude and latitude coordinates of the 3D points marked in the CAD drawing of the first area, or the absolute longitude and latitude coordinates of some building corner points in the first area.

[0173] Optionally, the indoor positioning result is a positioning result of any one of Bluetooth positioning, WiFi positioning, geomagnetic positioning, and ultra-wideband UWB positioning.

[0174] Optionally, the 3D point coordinate extraction module 1702 is specifically used to extract 2D feature points in the first image; the 2D feature points in the first image, the camera intrinsic parameters corresponding to the first image, the low-precision positioning results corresponding to the first image, and the 3D model of the first area are input into the ray intersection algorithm to obtain the 3D point coordinates corresponding to the 2D feature points in the first image output by the ray intersection algorithm.

[0175] Optionally, when the first image is an image corresponding to an outdoor scene of the first area, the 3D model of the first area is an established CAD 3D model of the first area or a patch model generated according to a satellite image or an aerial image of the first area.

[0176] Optionally, when the first image is an image corresponding to an indoor scene of the first area, the 3D model of the first area is an established CAD 3D model of the first area.

[0177] For example, the physical device corresponding to the map construction device can refer to the aforementioned Fig.15The cloud structure in the map construction system shown, such as: the acquisition module 1701 can be a receiving module in the cloud for receiving data, the 3D point coordinate extraction module 1702, the low-precision map construction module 1703, and the high-precision map construction module 1704 can be algorithm modules in the map construction server cluster in the cloud for implementing corresponding functions.

[0178] It should be understood that the division of units or modules (hereinafter referred to as units) in the above device is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or physically separated. And the units in the device can all be implemented in the form of software calling through processing elements; or all in the form of hardware; or some units can be implemented in the form of software calling through processing elements, and some units can be implemented in the form of hardware.

[0179] For example, each unit can be a separately established processing element, or it can be integrated in a certain chip of the device. In addition, it can also be stored in a memory in the form of a program, and called and executed by a certain processing element of the device. In addition, all or part of these units can be integrated together, or they can be implemented independently. The processing element described here can also be called a processor, which can be an integrated circuit with signal processing capabilities. In the implementation process, each step of the above method or each unit above can be implemented by an integrated logic circuit of hardware in the processor element or in the form of software called by a processing element.

[0180] In one example, the unit in the above device can be one or more integrated circuits configured to implement the above method, such as: one or more application specific integrated circuits (ASICs), or one or more digital signal processors (DSPs), or one or more field programmable gate arrays (FPGAs), or a combination of at least two of these integrated circuit forms.

[0181] For another example, when the units in the device can be implemented in the form of a processing element scheduling program, the processing element can be a general-purpose processor, such as a CPU or other processor that can call a program. For another example, these units can be integrated together and implemented in the form of a system-on-a-chip (SOC).

[0182] In one implementation, the units of the above apparatus implementing the corresponding steps in the above method can be implemented in the form of a processing element scheduling program. For example, the apparatus may include a processing element and a storage element, and the processing element calls the program stored in the storage element to execute the method described in the above method embodiment. The storage element may be a storage element on the same chip as the processing element, that is, an on-chip storage element.

[0183] In another implementation, the program for executing the above method may be in a storage element on a different chip from the processing element, i.e., an off-chip storage element. In this case, the processing element calls or loads the program from the off-chip storage element to the on-chip storage element to call and execute the method described in the above method embodiment.

[0184] For example, an embodiment of the present application may also provide a device, such as an electronic device, which may include a processor and a memory for storing instructions executable by the processor. When the processor is configured to execute the above instructions, the electronic device implements the map construction method described in the above embodiment. The memory may be located inside the electronic device or outside the electronic device. And the processor includes one or more. The electronic device may be a map construction server cluster in the cloud.

[0185] In another implementation, the unit of the device implementing each step in the above method may be configured as one or more processing elements, where the processing element may be an integrated circuit, for example, one or more ASICs, or one or more DSPs, or one or more FPGAs, or a combination of these types of integrated circuits. These integrated circuits may be integrated together to form a chip.

[0186] For example, an embodiment of the present application further provides a chip, which can be applied to the above electronic device. The chip includes one or more interface circuits and one or more processors; the interface circuits and the processors are interconnected through lines; the processor receives and executes computer instructions from the memory of the electronic device through the interface circuit to implement the map construction method as described in the above embodiment.

[0187] The embodiment of the present application also provides a computer program product, including a computer readable code. When the computer readable code is executed in an electronic device, the electronic device implements the map construction method as described in the above embodiment.

[0188] Through the description of the above implementation methods, technical personnel in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above-mentioned functional modules is used as an example. In actual applications, the above-mentioned functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0189] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0190] The units described as separate components may or may not be physically separated, and the components shown as units may be one physical unit or multiple physical units, that is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0191] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0192] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, such as a program. The software product is stored in a program product, such as a computer-readable storage medium, including a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, mobile hard drives, ROM, RAM, magnetic disks, or optical disks.

[0193] For example, the embodiment of the present application may also provide a computer-readable storage medium on which computer program instructions are stored. When the computer program instructions are executed by an electronic device, the electronic device implements the map construction method described in the above embodiment.

[0194] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A map construction method, characterized in that: The method comprises: Acquire a first image taken by a user when using a first visual positioning service in a first area, and a low-precision positioning result corresponding to the first image; the first visual positioning service includes a skyline-based visual positioning service and / or a contour-based visual positioning service; wherein, when the first image is an image corresponding to an outdoor scene of the first area, the low-precision positioning result is a positioning result of the first visual positioning service or a positioning result of a traditional positioning service; when the first image is an image corresponding to an indoor scene of the first area, the low-precision positioning result is an indoor positioning result; Determine, according to the first image, the low-precision positioning result corresponding to the first image, and the 3D model of the first area, the 3D point coordinates corresponding to the 2D feature points in the first image; Constructing a low-precision map capable of providing a feature point-based visual positioning service for the first area according to the first image, the low-precision positioning result corresponding to the first image, and the 3D point coordinates corresponding to the 2D feature points in the first image; According to the coordinates of the key 3D points of the first area, a global bundle adjustment is performed on the low-precision map capable of providing a visual positioning service based on feature points for the first area, so as to obtain a high-precision map capable of providing a visual positioning service based on feature points for the first area; The determining, according to the first image, the low-precision positioning result corresponding to the first image, and the 3D model of the first area, the 3D point coordinates corresponding to the 2D feature points in the first image includes: Extracting 2D feature points in the first image; The 2D feature points in the first image, the camera intrinsic parameters corresponding to the first image, the low-precision positioning results corresponding to the first image, and the 3D model of the first area are input into a ray intersection algorithm to obtain the 3D point coordinates corresponding to the 2D feature points in the first image output by the ray intersection algorithm.

2. The method according to claim 1, characterized in that The step of performing global bundle adjustment on the low-precision map capable of providing a feature point-based visual positioning service for the first area according to the key 3D point coordinates of the first area includes: According to the key 3D point coordinates of the first area, the 2D feature points in the first image constituting the low-precision map are calibrated, and the low-precision map is globally bundle adjusted; wherein, when the low-precision map is globally bundle adjusted, the key 3D point coordinates calibrated in the first image constituting the low-precision map participate in the iteration but not in the update.

3. The method according to claim 1 or 2, characterized in that: When there is an existing high-precision map for a second area bordering the first area or a second area within the first area, performing global bundle adjustment on the low-precision map capable of providing a feature point-based visual positioning service for the first area according to the key 3D point coordinates of the first area includes: According to the key 3D point coordinates of the first area, the 2D feature points in the first image constituting the low-precision map are calibrated, and the low-precision map and the existing high-precision map of the second area are globally bundle adjusted; wherein, when the low-precision map and the existing high-precision map of the second area are globally bundle adjusted, the key 3D point coordinates calibrated in the first image constituting the low-precision map and the 3D point coordinates in the existing high-precision map of the second area participate in the iteration but not in the update.

4. The method according to claim 1 or 2, characterized in that: The first image is an image corresponding to an outdoor scene in the first area, and the key 3D point coordinates in the first area are absolute longitude and latitude coordinates of corner points of some buildings in the first area.

5. The method according to claim 4, characterized in that When the area corresponding to the first image is covered by the first visual positioning service, the low-precision positioning result is the positioning result of the first visual positioning service; When the area corresponding to the first image is not covered by the first visual positioning service, the low-precision positioning result is the traditional positioning service positioning result.

6. The method according to claim 5, characterized in that The traditional positioning service positioning result is a positioning result of any one of the positioning methods including global positioning system GPS positioning, real-time dynamic differential method RTK positioning, magnetometer positioning, and network location provider NLP positioning.

7. The method according to claim 5, characterized in that When the first image includes a skyline, the first visual positioning service positioning result is a visual positioning service positioning result based on the skyline; When the first image does not include a skyline, the first visual positioning service positioning result is a contour-based visual positioning service positioning result.

8. The method according to claim 1 or 2, characterized in that: The first image is an image corresponding to the indoor scene of the first area, and the key 3D point coordinates of the first area are the absolute longitude and latitude coordinates of the 3D points marked in the CAD drawing of the first area, or the absolute longitude and latitude coordinates of some building corner points of the first area.

9. The method according to claim 8, characterized in that The indoor positioning result is a positioning result of any one of the positioning modes including Bluetooth positioning, WiFi positioning, geomagnetic positioning, and ultra-wideband UWB positioning.

10. The method according to claim 1 or 2, characterized in that: The first image is an image corresponding to an outdoor scene of the first area, and the 3D model of the first area is an established CAD 3D model of the first area or a patch model generated according to a satellite image or an aerial image of the first area.

11. The method according to claim 1 or 2, characterized in that: The first image is an image corresponding to an indoor scene of the first area, and the 3D model of the first area is an established CAD 3D model of the first area.

12. An electronic device, characterized in that: include: a processor, a memory for storing instructions executable by the processor; When the processor is configured to execute the instructions, the electronic device implements the method according to any one of claims 1 to 11.

13. A computer-readable storage medium having computer program instructions stored thereon; characterized in that: When the computer program instructions are executed by an electronic device, the electronic device implements the method according to any one of claims 1 to 11.

Images