Map construction method and device, storage medium and program product

Abstract

The embodiment of the invention provides a map construction method and device, a storage medium and a program product, and the method comprises the steps: obtaining a target region of a to-be-constructed map, dividing the target region into a plurality of grids, and carrying out the associated storage of image frame data smaller than or equal to a first preset number for each grid, and constructing a target map according to the stored image frame data associated with each grid. According to the map construction method provided by the embodiment of the invention, the area for constructing the map is subjected to grid division, the images are screened based on the grids, and map construction is performed through fewer screened images, so that resources can be saved, and the map construction efficiency can be improved.

Description

Technical Field

[0001] This disclosure relates to the field of extended reality technology, and in particular to a map building method, device, storage medium, and program product. Background Technology

[0002] Spatial anchoring technology is a key feature of XR devices. By building an anchor map, it is possible to ensure that the position of virtual objects remains unchanged in the real world, even after wearing the headset multiple times or restarting the application.

[0003] In related technologies, Simultaneous Localization and Mapping (SLAM) technology is typically used to build maps in real time based on the collected raw image data.

[0004] However, the inventors have discovered at least the following technical problems in the related technologies: the mapping efficiency is low when the user scenario is too large. Summary of the Invention

[0005] This disclosure provides a map building method, device, storage medium, and program product to improve map building efficiency.

[0006] In a first aspect, embodiments of this disclosure provide a map construction method, including:

[0007] Obtain the target area of ​​the map to be built; the target area is divided into multiple grids;

[0008] For each grid cell, image frame data less than or equal to a first preset number is associated and stored;

[0009] The target map is constructed by associating the image frame data of each raster with the data stored separately.

[0010] In a second aspect, embodiments of this disclosure provide an information display device, comprising:

[0011] The acquisition module is used to acquire the target area of ​​the map to be built; the target area is divided into multiple grids;

[0012] The storage module is used to associate and save image frame data less than or equal to a first preset number for each raster.

[0013] The construction module is used to build the target map based on the image frame data associated with each raster and saved separately.

[0014] Thirdly, embodiments of this disclosure provide an electronic device, including: a processor and a memory;

[0015] The memory stores computer-executed instructions;

[0016] The processor executes computer execution instructions stored in the memory, causing the at least one processor to perform the map construction method as described in the first aspect and various possible designs of the first aspect.

[0017] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer-executable instructions, which, when executed by a processor, implement the map construction method described in the first aspect and various possible designs of the first aspect.

[0018] Fifthly, embodiments of this disclosure provide a computer program product, including a computer program that, when executed by a processor, implements the map construction method described in the first aspect and various possible designs of the first aspect.

[0019] The map building method, device, storage medium, and program product provided in this embodiment include: acquiring a target area for map construction, wherein the target area is divided into multiple grids; for each grid, a number of image frames (less than or equal to a first preset number) are associated and stored; and constructing a target map based on the associated and stored image frame data for each grid. The map building method provided in this embodiment saves resources and improves mapping efficiency by dividing the map construction area into grids and filtering images based on the grids, thus constructing the map using a smaller number of filtered images. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of this disclosure or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0021] Figure 1 This is a schematic diagram of an application scenario for map construction provided in the embodiments of this disclosure. Figure 1 ;

[0022] Figure 2 This is a schematic diagram of an application scenario for map construction provided in the embodiments of this disclosure. Figure 2 ;

[0023] Figure 3 Flowchart of the map construction method provided in this embodiment of the disclosure Figure 1 ;

[0024] Figure 4 A schematic diagram of a first map provided for an embodiment of this disclosure;

[0025] Figure 5 Flowchart of the map construction method provided in this embodiment of the disclosure Figure 2 ;

[0026] Figure 6 A schematic diagram of the anchor point map provided in the embodiments of this disclosure;

[0027] Figure 7 Flowchart of the map construction method provided in this embodiment of the disclosure Figure 3 ;

[0028] Figure 8 This is a structural block diagram of an information display device provided in an embodiment of the present disclosure;

[0029] Figure 9 This is a schematic diagram of the hardware structure of an information display device provided in an embodiment of this disclosure. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0031] With the rapid development of the metaverse industry, extended reality (XR) devices such as augmented reality (AR) and mixed reality (MR) have received a lot of attention.

[0032] One of the most important technologies in XR devices is spatial computing, which primarily enables users to interact with real-world spaces. The core technology of spatial computing is spatial anchoring. Based on spatial anchoring, the user's usage scenario can be mapped, abstracting all virtual objects into points anchored in the constructed map. When the user's specific location in space is known, the relative relationship between the virtual anchor and the user's location can be determined based on the anchoring map. For example, if a virtual clock is placed on a physical desktop, its position can be fixed using anchoring.

[0033] In related technologies, a sparse point cloud map and coordinate lines are usually created for the space to be mapped, and the map is built based on the SLAM system built into the XR device to place anchor points.

[0034] However, when the user scenario is too large, the above method will slow down the map building process, increase the amount of internal and external memory required, and affect the user experience.

[0035] To address the aforementioned technical problems, the inventors of this disclosure have discovered that when constructing a map, the collected images can be filtered, and the map can be constructed based on the filtered images. Compared to processing the entire amount of collected image data, this method saves resources and improves mapping efficiency. Based on this, embodiments of this disclosure provide a map construction method.

[0036] refer to Figure 1 , Figure 1 This is a schematic diagram of an application scenario for map construction provided in the embodiments of this disclosure. Figure 1 .like Figure 1 As shown, a user wearing an XR device 101 walks in a space (e.g., an office area on a floor of an office building), starting from point A, and simultaneously collects images and builds a map during the walk. When building the map, the target area to be mapped can first be rasterized, and the collected images can be filtered based on the raster. The map is then built based on the filtered images. The map building method provided in this embodiment saves resources and improves mapping efficiency by rasterizing the map construction area and filtering images based on the raster, thus building the map using a smaller number of filtered images.

[0037] The inventors of this disclosure have further discovered that the common practice in related technologies of constructing a complete map for the entire space is another major reason for low mapping efficiency. In response, the inventors have discovered that mapping can be skipped when there is no need to place anchor points, and a map of a specific grid area can be constructed only when anchor points are created, thereby improving mapping efficiency and saving resources.

[0038] refer to Figure 2 , Figure 2 This disclosure provides an example of a map construction application scenario. Figure 2 .like Figure 2As shown, a user wearing XR device 101 walks in a space (e.g., an office area on a floor of an office building), starting from point A. If there is no need to place anchor points, no mapping is performed. If, upon reaching point B, an anchor point placement request b is received, region b can be determined based on point B, map b can be built based on region b, and anchor point b can be bound to map b. The user continues walking. If, upon reaching point C, an anchor point placement request c is received, and if point C is outside region b, region c can be determined based on point C, map c can be built based on region c, and anchor point c can be bound to map c. During the construction of maps b and c, the collected images are filtered, and the map is constructed based on the filtered images. The map construction method provided in this embodiment improves mapping efficiency and saves resources by constructing the map only when creating anchor points and using a smaller number of filtered images, compared to constructing a complete map for the entire space.

[0039] The technical solutions of this application will be described in detail below with reference to specific disclosed embodiments. These specific disclosed embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some disclosed embodiments.

[0040] refer to Figure 3 , Figure 3 Flowchart of the map construction method provided in this embodiment of the disclosure Figure 1 The method described in this embodiment can be applied to terminal devices or servers. This map construction method includes:

[0041] 301. Obtain the target area of ​​the map to be built; the target area is divided into multiple grids.

[0042] In this embodiment of the disclosure, the target area can be the area entered by the user wearing the XR device, such as a room or a floor of a shopping mall.

[0043] Considering that users typically move horizontally within the target area with minimal vertical movement, although the SLAM system built into XR devices can obtain three-dimensional spatial coordinates including the z-axis representing the direction of gravity, the x-axis perpendicular to the direction of gravity, and the y-axis, the target area in this embodiment generally refers to the area within the horizontal plane defined by the x-axis and y-axis. Grid division is also performed on the horizontal plane along the x-axis and y-axis, for example... Figure 1 The ground shown.

[0044] 302. For each grid cell, associate and save image frame data less than or equal to a first preset number.

[0045] Specifically, considering that users may repeatedly walk within a small area during their walk, the number of images collected in that area will be relatively large. However, the accuracy of map construction does not increase with the number of images. On the contrary, a large number of images will occupy storage space and slow down the mapping process. Therefore, in order to better balance the accuracy of map construction and the waste of resources, the number of image frame data associated with each grid can be limited.

[0046] The image frame data and the image frame are in one-to-one correspondence. The image frame data includes the two-dimensional coordinates of the feature points in the corresponding image frame, the three-dimensional coordinates of the spatial points corresponding to the feature points, the descriptor of the corresponding image frame, and the pose of the image frame.

[0047] For example, it may be limited to storing 30 to 40 frames of image frame data for each raster association.

[0048] In one embodiment of this disclosure, the timing for filtering image frame data can be either after all the acquired images have been saved and then deleted, or it can be done by capturing and saving images simultaneously until a preset number is reached and no further associated saving is required. This embodiment does not limit this approach.

[0049] In one embodiment of this disclosure, there are multiple ways to filter image frame data. In one possible implementation, the image frame data can be filtered based on the acquisition location. For example, multiple image frame data with similar acquisition locations can be filtered. Assuming that 30 images are acquired within 30 square centimeters, only 10 images can be retained. In order to ensure that the image frame data is evenly distributed, the 30 square centimeters can be further divided into 10 sub-regions, and each sub-region retains 1 image.

[0050] In another possible implementation, filtering can be based on the orientation angle corresponding to the image frame data. Specifically, the step of associating and storing less than or equal to a first preset number of image frame data for each grid may include: assigning multiple orientation ranges to each grid; the orientation ranges representing the range to which the orientation angle corresponding to the image frame data belongs; associating and storing less than or equal to a second preset number of image frame data for each orientation range in each grid; the second preset number is less than the first preset number.

[0051] In this embodiment, the angle difference between the two endpoints of multiple orientation ranges can be equal. The value of this angle difference can be set to range from 25 degrees to 50 degrees. There is no overlap between different orientation ranges.

[0052] In one embodiment of this disclosure, each grid in the first map includes at least one sector with the center of the grid as the center and the central angle as a preset angle, and different sector partitions correspond to different orientation angles; after constructing the first map by associating the image frame data stored with each grid in the first region, it may further include: for each sector partition, displaying the sector partition differently according to the number of image frame data stored with the sector partition.

[0053] In this embodiment of the disclosure, the target area refers to the area in a plane perpendicular to gravity, and the corresponding grid sector partition is obtained by dividing the plane into 360-degree angles, that is, the orientation angle refers to the circumferential angle perpendicular to the direction of gravity.

[0054] In this embodiment of the disclosure, the quantity of different image frame data can be distinguished by using different colors to represent different quantities, such as... Figure 6 As shown, red represents saving 1 frame, green represents saving 2 frames, blue represents saving 3 frames, and black represents saving more than 3 frames, reaching the threshold and no further saving is possible. Different distinctions can also be made based on different fill patterns, different numerical values, etc.

[0055] For example, such as Figure 4 As shown, assuming a user wearing an XR device walks to point B and receives an anchor placement request, multiple grids (e.g., 5x5) can be constructed centered on point B. To further limit the number of images within each grid, multiple orientation ranges can be configured for the grid. Taking the second grid in the second row as an example, assuming the orientation range is divided into 45-degree sections, we can obtain 8 sector partitions (1-8) in the diagram, each corresponding to one of the 8 orientation ranges. Orientation range 1 corresponds to 0 to 45 degrees, orientation range 2 corresponds to 45 to 90 degrees, and so on, with orientation range 8 corresponding to 315 to 360 degrees. In practical applications, each sector partition can be limited to retaining 3 frames, meaning each orientation range retains 3 frames. For example, if the user has captured 3 frames at orientation angles of 10 degrees, 30 degrees, and 25 degrees, any subsequent images captured within the 0 to 45 degree range will not be saved.

[0056] It should be noted that regardless of the image frame's location within the grid, as long as the orientation angle meets the corresponding orientation range, the image frame data for that frame will be saved within that orientation range. When visualizing the number of filtered image frames, it can be done as follows: Figure 4 As shown, a circle is constructed with the center point O of the grid as the center, and then divided into 8 sector partitions. Each sector partition displays the number of image frame data stored in the corresponding orientation range.

[0057] 303. Construct the target map based on the image frame data associated with each grid.

[0058] Specifically, as the associated and saved image frame data continues to increase, existing map building techniques, such as front-end feature point extraction, matching, triangulation, and data analysis (BA), can be used to construct the target map based on the associated and saved image frame data.

[0059] As can be seen from the above description, the map construction method provided in this disclosure can save resources and improve map construction efficiency by dividing the area for map construction into grids and filtering images based on the grids, and constructing a map using fewer filtered images.

[0060] refer to Figure 5 , Figure 5 Flowchart of the map construction method provided in this embodiment of the disclosure Figure 2 Based on the above-disclosed embodiments, for example in Figure 3 Based on the disclosed embodiments, this embodiment further addresses the problem of low mapping efficiency by adding a technical feature to determine a small area based on the anchor point placement location and constructing a map within that area, thereby improving mapping efficiency. Specifically, in this embodiment, the map construction method includes:

[0061] 501. In response to receiving a first anchor point placement request, if the current location is outside the second region, the current location is obtained, and the first region is determined based on the current location; the first region is divided into multiple grids; the second region is determined in response to receiving a second anchor point placement request.

[0062] In this embodiment of the disclosure, a first anchor point placement request is used to instruct the placement of a first anchor point. The first anchor point can be used to mark a location in the world coordinate system, allowing a virtual object to be placed at that location or rotated around it. The type of the first anchor point may include, but is not limited to, planar anchor points, bounding box anchor points, and point anchor points.

[0063] The current location refers to the location where the user is currently walking, which is also the current location of the XR device worn by the user.

[0064] Specifically, taking an XR device as an example, when a user wears an XR device and walks in space, upon receiving a request to place the first anchor point, the user can obtain the current location through the SLAM system carried by the XR device and determine whether the user has left the second area. If the current location is outside the second area, the user can determine the first area based on the current location.

[0065] In one embodiment of this disclosure, determining the first region based on the current location may include: constructing the first region centered on the current location. By determining the first region centered on the current location, the user can continue to acquire image frame data regardless of which direction they move in, facilitating the collection of more images for accurately constructing the first map.

[0066] In one embodiment of this disclosure, determining the first region based on the current location may include: predicting the user's direction of travel, constructing an ellipse with the current location as the first focus and a point at a preset distance from the current location along the direction of travel as the second focus, and defining the elliptical region as the first region. The direction of travel can be determined based on trajectory points determined by the XR device's built-in SLAM system within a preset time period prior to the current moment. This embodiment of the disclosure, by combining the direction of travel with the construction of the elliptical region, ensures that the user's walking trajectory matches the first region, thereby improving the utilization rate of the first region, avoiding large blank areas in the first region that users cannot reach, and further saving resources.

[0067] 502. For each grid cell in the first region, associate and save image frame data less than or equal to a first preset number.

[0068] Step 502 in this embodiment is similar to step 302 in the above embodiment, and will not be repeated here.

[0069] 503. Construct the first map based on the image frame data associated with each grid cell in the first region, and bind the first anchor point to the first map.

[0070] Specifically, as the associated and saved image frame data increases, existing map building techniques, such as front-end feature point extraction, matching, triangulation, and data analysis (BA), can be used to construct the first map based on the associated and saved image frame data. Once the user leaves the first area and the first map is completed, the first anchor point can be bound to the first map.

[0071] In one embodiment of this disclosure, binding the first anchor point to the first map allows the calculation of the position coordinates of the first anchor point in the map coordinate system of the first map based on the transformation relationship between the world coordinate system corresponding to the SLAM system and the map coordinate system of the first map.

[0072] As can be seen from the above description, the map construction method provided in this disclosure can save resources and improve map construction efficiency by dividing the area for map construction into grids and filtering images based on the grids, and constructing a map using fewer filtered images.

[0073] In one embodiment of this disclosure, in the above Figure 5Based on the embodiment, step 503 may further include: during the positioning process, if the first map is located, the pose transformation relationship between the map coordinate system of the first map and the map coordinate system of the second map is obtained, and the first position coordinates of at least one anchor point bound to the second map in the map coordinate system of the first map are determined according to the pose transformation relationship, and the at least one anchor point bound to the second map is displayed according to the first position coordinates.

[0074] For example, such as Figure 6 As shown, assuming a first map 601, a second map 602, and a third map 603 are constructed, when the device restarts or after a preset time has elapsed since mapping was completed, if the user re-enters the area of ​​the first map 601, the user can be located in the first map 601. The anchor points bound to the first map 601 can be accurately displayed based on the first map's coordinate system. To enhance the realism of the experience, the anchor points bound to the second map 602 and the third map 603 will also be displayed as the user's line of sight moves. Considering that the second map 602 and the third map 603 are some distance from the user, they can also be displayed based on the first map 601's coordinate system. Similarly, when the user moves to the second map 602, the anchor points bound to the first map 601, the second map 602, and the third map 603 will be displayed based on the second map 602's coordinate system.

[0075] As can be seen from the above description, the map construction method provided in this disclosure displays all anchor points based on the map coordinate system of the located map during the positioning process, which takes into account both the accuracy of anchor point display and the realism of user experience.

[0076] refer to Figure 7 , Figure 7 Flowchart of the map construction method provided in this embodiment of the disclosure Figure 3 Based on the above-disclosed embodiments, for example in Figure 5 Based on the disclosed embodiments, this embodiment comprehensively and exemplaryly illustrates the map construction and positioning process, and the map construction method includes:

[0077] 701. In response to receiving a second anchor point placement request, determine a second region based on the current location; construct a second map based on the second region, and bind the second anchor point corresponding to the second anchor point placement request to the second map.

[0078] Specifically, when a user wearing an XR device walks in a certain area, the mapping thread is not activated if there is no need to place anchor points from the start of walking. The mapping thread is activated when the first anchor point is placed. Starting from the moment the user places the anchor point, the mapping thread records the current position of the XR device as determined by the SLAM system. Multiple grids (e.g., 5x5) are divided around this current position as a second region. Each grid can be further divided according to its orientation angle, with each preset angle (e.g., 30 degrees) forming a sector. If the user continues to move within the second region, the captured images are associated with and saved along with the corresponding grids and sector partitions. To balance map accuracy and resource consumption, each sector partition is limited to a maximum of 3 frames; no more frames are added after 3.

[0079] 702. In response to receiving the first anchor point placement request, determine whether the current location is outside the second area. If yes, proceed to step 703; otherwise, proceed to step 706.

[0080] 703. Obtain the current location and determine a first region based on the current location; the first region is divided into multiple grids; the second region is determined in response to receiving a second anchor point placement request.

[0081] 704. For each grid cell in the first region, associate and save image frame data less than or equal to a first preset number.

[0082] 705. Construct the first map based on the image frame data associated with each grid cell in the first region, and bind the first anchor point to the first map.

[0083] Steps 702 to 705 in this embodiment are similar to steps 501 to 503 in the above embodiment, and will not be repeated here.

[0084] 706. Bind the first anchor point to the second map; the second map is constructed based on the second region.

[0085] Specifically, when a user is walking in the second area, all received anchor point placement requests can bind the corresponding anchor point to the second map. However, once the user leaves the second area and receives a new anchor point placement request, a new map needs to be constructed to bind the corresponding anchor point.

[0086] 707. During the positioning process, the target map is expanded based on the target image frame data used for positioning; the target map is a map from all historically constructed maps, including the first map and the second map.

[0087] Specifically, considering that each map is constructed with corresponding grid areas (e.g., a 5x5 grid), users may have traversed some grids but not others. Let's assume that grid 'a' out of these 25 grids was not visited by the user. Then, when the user visits grid 'a' and the image frame data is obtained, it can be associated and saved in grid 'a'. Similarly, if the user has visited grid 'b', but the number of associated and saved images for it hasn't reached a threshold, then when the user visits it again, the image acquired during positioning can be associated and saved with grid 'b'. This allows for the reconstruction of a new map based on the associated and saved image frame data for each grid, thus updating the previously constructed map.

[0088] In this embodiment of the disclosure, existing positioning technologies can be used in the positioning process, such as one or more steps including feature point extraction, generating a bag of words, finding the most similar frame based on the bag of words in the map, feature point matching, PNP, and multi-frame consistency judgment.

[0089] In one embodiment of this disclosure, the step of expanding the located target map based on the target image frame data used for positioning may include: if a target map is located, determining the target grid corresponding to the target image frame data used for positioning in a plurality of grids of the target map; the target map is a map in all historically constructed maps, including the first map; if the image frame data associated with the target grid is less than or equal to a third preset number, then associating and saving the target image frame data with the target grid, and updating the target map based on the target image frame data; the third preset number is less than or equal to the first preset number.

[0090] For example, suppose grid c is limited to storing a maximum of 30 images. If, when locating this grid, only 10 images are stored, then the image used for location can be associated with and stored with this grid to expand the map based on this image. As another example, grid d is divided into 12 sector partitions with a step size of 30 degrees. Each sector partition is limited to storing a maximum of 3 images. If, when locating sector partition e, only 2 images are associated with it, then the image used for location can be associated with and stored with sector partition e to expand the map based on this image.

[0091] As can be seen from the above description, the map construction method provided in this disclosure can save resources and improve mapping efficiency by dividing the area for map construction into grids and filtering images based on the grids, and constructing a map using fewer filtered images. Furthermore, the accuracy of the map can be improved by updating and expanding the map based on the images used for positioning during the positioning process.

[0092] Corresponding to the map construction method in the above embodiments, Figure 8This is a structural block diagram of an information display device provided in an embodiment of this disclosure. For ease of explanation, only the parts relevant to the embodiments of this disclosure are shown. (Refer to...) Figure 8 The device includes: a determination module 801, a storage module 802, and a construction module 803.

[0093] The determining module 801 is used to, in response to receiving a first anchor point placement request, obtain the current position if it is currently outside the second region, and determine the first region based on the current position; the first region is divided into multiple grids; the second region is determined in response to receiving a second anchor point placement request.

[0094] The storage module 802 is used to associate and store image frame data less than or equal to a first preset number for each grid.

[0095] The construction module 803 is used to construct the first map based on the image frame data associated with each grid of the first region, and to bind the first anchor point to the first map.

[0096] As can be seen from the above description, the map building device provided in this embodiment of the present disclosure can save resources and improve map building efficiency by dividing the area for map building into grids and filtering images based on the grids, and building a map using fewer filtered images.

[0097] In one embodiment of this disclosure, the storage module 802 is specifically configured to: allocate multiple orientation ranges for each grid; the orientation ranges represent the range of orientation angles corresponding to image frame data; and associate and store less than or equal to a second preset number of image frame data for each orientation range in each grid; the second preset number is less than the first preset number.

[0098] In one embodiment of this disclosure, each grid in the first map includes at least one sector with the center of the grid as the center and the central angle as a preset angle, and different sector partitions correspond to different orientation angles; the storage module 802 is further configured to: for each sector partition, distinguish and display the sector partition according to the number of image frame data associated with the sector partition.

[0099] In one embodiment of this disclosure, the determining module 801 is specifically used to: construct a first region centered on the current location.

[0100] In one embodiment of this disclosure, the determining module 801 is further configured to: if the current location is within a second region, bind the first anchor point to a second map; the second map is constructed based on the second region.

[0101] In one embodiment of this disclosure, the determining module 801 is further configured to: during the positioning process, if the first map is located, obtain the pose transformation relationship between the map coordinate system of the first map and the map coordinate system of the second map, determine the first position coordinates of at least one anchor point bound to the second map in the map coordinate system of the first map according to the pose transformation relationship, and display at least one anchor point bound to the second map according to the first position coordinates.

[0102] In one embodiment of this disclosure, the determining module 801 is further configured to: during the positioning process, expand the located target map based on the target image frame data used for positioning; the target map is a map among all historically constructed maps, including the first map.

[0103] In one embodiment of this disclosure, the determining module 801 is specifically configured to: if a target map is located, determine the target grid corresponding to the target image frame data used for positioning in a plurality of grids of the target map; the target map is a map in all historically constructed maps, including the first map;

[0104] If the image frame data associated with the target raster is less than or equal to a third preset number, then the target image frame data is associated with the target raster and the target map is updated according to the target image frame data; the third preset number is less than or equal to the first preset number.

[0105] The device provided in this embodiment can be used to execute the technical solutions of the above method embodiments. Its implementation principle and technical effect are similar, and will not be described again here.

[0106] To implement the above embodiments, this disclosure also provides an electronic device.

[0107] refer to Figure 9 The diagram illustrates a structural schematic of an electronic device 900 suitable for implementing embodiments of the present disclosure. The electronic device 900 can be a terminal device or a server. The terminal device can include, but is not limited to, mobile terminals such as mobile phones, laptops, digital radio receivers, personal digital assistants (PDAs), portable Android devices (PADs), portable media players (PMPs), and in-vehicle terminals (e.g., in-vehicle navigation terminals), as well as fixed terminals such as digital TVs and desktop computers. Figure 9 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments disclosed herein.

[0108] like Figure 9 As shown, the electronic device 900 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 901, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 902 or a program loaded from a storage device 908 into a random access memory (RAM) 903. The RAM 903 also stores various programs and data required for the operation of the electronic device 900. The processing unit 901, ROM 902, and RAM 903 are interconnected via a bus 904. An input / output (I / O) interface 905 is also connected to the bus 904.

[0109] Typically, the following devices can be connected to I / O interface 905: input devices 906 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 907 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 908 including, for example, magnetic tapes, hard disks, etc.; and communication devices 909. Communication device 909 allows electronic device 900 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 9 An electronic device 900 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.

[0110] In particular, according to embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device 909, or installed from a storage device 908, or installed from a ROM 902. When the computer program is executed by a processing device 901, it performs the functions defined in the methods of embodiments of this disclosure.

[0111] It should be noted that the computer-readable medium described in this disclosure can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium can be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this disclosure, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in connection with an instruction execution system, apparatus, or device. In this disclosure, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0112] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.

[0113] The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to perform the methods shown in the above embodiments.

[0114] Computer program code for performing the operations of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0115] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0116] The units described in the embodiments of this disclosure can be implemented in software or in hardware. The name of a unit does not necessarily limit the unit itself; for example, the first acquisition unit can also be described as "a unit that acquires at least two Internet Protocol addresses".

[0117] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0118] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0119] The above description is merely a preferred embodiment of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed in this disclosure that have similar functions.

[0120] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this disclosure. Certain features described in the context of individual embodiments may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.

[0121] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.

Claims

1. A map construction method, characterized in that, include: Obtain the target area of ​​the map to be built; The target area is divided into multiple grids; For each grid cell, image frame data less than or equal to a first preset number is associated and stored; The target map is constructed by associating the image frame data of each raster with the data stored separately.

2. The method according to claim 1, characterized in that, The step of associating and storing image frame data less than or equal to a first preset number for each grid cell includes: For each grid cell, multiple orientation ranges are assigned to the grid cell; the orientation range represents the range of orientation angles corresponding to the image frame data. For each orientation range in each grid, image frame data less than or equal to a second preset number is associated and stored; the second preset number is less than the first preset number.

3. The method according to claim 2, characterized in that, Each grid in the target map includes at least one sector with the center of the grid as the center and the central angle as a preset angle, and different sector partitions correspond to different orientation angles; after constructing the target map based on the image frame data associated with each grid, the method further includes: For each sector, the sector is displayed differently based on the number of image frame data associated with it.

4. The method according to any one of claims 1-3, characterized in that, The target area includes a first area and a second area; the target map includes a first map; obtaining the target area of ​​the map to be constructed includes: In response to receiving a first anchor point placement request, if the current location is outside the second region, the current position is obtained, and the first region is determined based on the current position; the first region is divided into multiple grids; the second region is determined in response to receiving a second anchor point placement request; The step of constructing a target map based on the image frame data associated with each raster and stored separately includes: The first map is constructed by associating the image frame data with each grid cell in the first region, and the first anchor point is bound to the first map.

5. The method according to claim 4, characterized in that, Determining the first region based on the current location includes: Construct the first region centered on the current location.

6. The method according to claim 4, characterized in that, The response after receiving the first anchor placement request also includes: If the current location is within the second region, the first anchor point is bound to the second map; the second map is constructed based on the second region.

7. The method according to claim 6, characterized in that, After constructing the first map based on the image frame data associated and stored in each partition, the method further includes: During the positioning process, if the first map is located, the pose transformation relationship between the map coordinate system of the first map and the map coordinate system of the second map is obtained. Based on the pose transformation relationship, the first position coordinates of at least one anchor point bound to the second map in the map coordinate system of the first map are determined, and the at least one anchor point bound to the second map is displayed based on the first position coordinates.

8. The method according to claim 4, characterized in that, After constructing the first map based on the image frame data associated and stored in each partition, the method further includes: During the localization process, the localized target map is expanded based on the target image frame data used for localization; the target map is a map from all historically constructed maps, including the first map.

9. The method according to claim 8, characterized in that, The step of expanding the located target map based on the target image frame data used for positioning includes: If a target map is located, the target raster corresponding to the target image frame data used for location is determined in multiple rasters of the target map; the target map is a map in all historically constructed maps, including the first map; If the image frame data associated with the target raster is less than or equal to a third preset number, then the target image frame data is associated with the target raster and the target map is updated according to the target image frame data; the third preset number is less than or equal to the first preset number.

10. A map building device, characterized in that, include: The acquisition module is used to acquire the target area of ​​the map to be built. The target area is divided into multiple grids; The storage module is used to associate and save image frame data less than or equal to a first preset number for each grid cell; The construction module is used to build the target map based on the image frame data associated with each raster and stored separately.

11. An electronic device, characterized in that, include: Processor and memory; The memory stores computer-executed instructions; The processor executes computer execution instructions stored in the memory, causing the processor to perform the map construction method as described in any one of claims 1 to 9.

12. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer-executable instructions, which, when executed by a processor, implement the map construction method as described in any one of claims 1 to 9.

13. A computer program product, comprising a computer program, characterized in that, When the computer program is executed by a processor, it implements the map construction method as described in any one of claims 1 to 9.

Images