System for map operations

Abstract

Embodiments of the present application disclose a system for map operation. The system comprises: a map generation device configured to determine whether to initiate a material collection task according to acquired first map information, generate a target map, store the target map, and generate a map distribution request corresponding to the target map when the determination result is yes; a map distribution device configured to receive the map distribution request and update second map information corresponding to the target map to a server according to the map distribution request; and a server configured to acquire the second map information, store the second map information, and call corresponding data according to a data processing requirement of an application terminal. The scheme provided by the present application can achieve the technical effect of long-term effective updating and management of an AR map while meeting the function of generating the AR map.

Description

Technical Field

[0001] This invention relates to the field of wearable device technology applications, and more particularly to a map operation system. Background Technology

[0002] AR (Augmented Reality) is a technology that blends virtual information with real-world information. Through AR glasses, mobile phones, and other devices, users can observe pre-defined virtual content in the real environment and even interact with it naturally, making the virtual content feel real. Typical AR applications include Niantic's AR mobile game Pokemon Go and AR navigation from Gaode Maps.

[0003] To enable users to experience a seamless integration of virtual content and the real environment, the core challenge is to replicate and simulate elements of the real environment at a 1:1 scale. These elements include, but are not limited to, geometric spatial sense, lighting, and other physical laws (such as tactile sensation). Taking geometric spatial sense as an example, to simulate a virtual-real overlay effect that conforms to the spatial laws of the real physical world during an AR experience, the following needs to be accomplished: digitizing the 3D space of the real environment, digitizing the 3D space of virtual objects, arranging the 3D poses of virtual objects, determining the 3D pose of the user's viewing angle, and rendering the 3D visuals.

[0004] The output of this digitalization is called an AR map (Augmented Reality Map, which will be referred to as "AR map" for simplicity). An AR map is a digital product that contains three-dimensional digital information about the real environment, provides a three-dimensional spatial reference for the placement of virtual objects in AR applications, and helps determine the three-dimensional spatial pose of the experience device (referred to as three-dimensional positioning or positioning).

[0005] The lack of complete AR-based map generation and maintenance technologies in related fields limits the application of AR technology, and no effective solution has yet been proposed. Summary of the Invention

[0006] To address the aforementioned technical problems, embodiments of the present invention aim to provide a map operation system that at least solves the problem of limited AR technology applications due to the lack of complete AR-based map generation and maintenance technologies in related technologies.

[0007] The technical solution of this invention is implemented as follows:

[0008] This invention provides a map operation system, comprising: a map generation device, a map distribution device, a map monitoring device, a server, and an application terminal. The map generation device is used to determine whether to initiate a material collection task based on acquired first map information. If the determination is yes, it generates a target map, stores the target map, and generates a map distribution request corresponding to the target map. The map distribution device, connected to the map generation device, is used to receive the map distribution request and update the second map information corresponding to the target map to the server according to the map distribution request. The server, connected to both the map distribution device and the application terminal, is used to acquire and store the second map information, and to call corresponding data according to the data processing needs of the application terminal. The application terminal, connected to both the server and the map monitoring device, is used to obtain corresponding operations from the server according to map usage request instructions, and to feed back the data in the map usage request instructions to the map monitoring device, so that the map generation device, map distribution device, and server execute corresponding operations based on the data. The map monitoring device, connected to the map generation device, application terminal, and server, is used to monitor the status of the map used by the application terminal and / or the server, and update the map data with changing status to the map generation device.

[0009] Optionally, the first map information includes: original materials, map ID, geographic location information, specific production requirements, and other external resources that assist in map generation.

[0010] Optionally, the map generation device is also used to reconstruct or update the map according to the received instructions if the determination result is negative.

[0011] Optionally, the map generation device includes: a map generation instruction analysis module, a material collection module, a generation task execution module, and a map archiving module. The map generation instruction analysis module determines the source of the map generation instruction, parses the instruction after determining its source, and obtains the instruction content. The material collection module, connected to the map generation instruction analysis module, determines whether to collect materials based on the instruction content of the map generation instruction, and collects material data if it is determined to do so. The generation task execution module, connected to the material collection module, executes map generation sub-tasks based on the map generation instruction and the corresponding task configuration template. The map archiving module, connected to the generation task execution module, uploads the generated map to the target directory after the map generation sub-tasks are completed, and synchronizes the change information of the target directory in the map database to the map distribution device.

[0012] Further, optionally, the instruction content includes: the target map ID to be processed, wherein the target map ID is used to indicate whether the target map to be processed is a newly created map; a local range in the target map to be processed, wherein the local range is specified by a list of material files or a pre-acquired coordinate range; wherein, when processing a historical map, the local range is obtained; additional material information, wherein the additional material information includes: whether original material is attached; the additional material information is used to indicate whether the original material data is provided in advance for this map generation task, and the access address of the original material data; wherein, the original material data is sent by the map monitoring device and will be automatically uploaded to the server, and the access address is recorded; generation requirements, wherein the generation requirements include: indicating the operations to be performed on the map, and related parameters; the operations are a combination of map generation sub-tasks.

[0013] Optionally, uploading the generated map to the target directory includes: uploading the corresponding information of the generated map to the target directory, wherein the information includes: target map ID, the latest map organization information, and a brief update description; or, the latest output of the leaf node map involved, wherein the latest output of the leaf node map includes: output data, as well as version information of each output and a brief update description of the output data.

[0014] Further, optionally, the change information includes: the target map ID, the latest map organization information, a brief update description; or, the access address of the latest output of the leaf node map involved.

[0015] Optionally, the map distribution device includes: a map service aggregation module, an information acquisition module, and a map distribution module. The map service aggregation module aggregates map server information, including server network address, geographic location, and current load. The information acquisition module acquires map update information, including target map ID, geographic location information, map organization information, leaf node map artifact archive address, and brief map update information. The map distribution module distributes map data, including: determining the target server to which the map will be distributed; and determining the map to be updated.

[0016] Optionally, the map monitoring device includes: a detection module, a collection module, a map status analysis module, and a map status feedback module. The detection module initiates map detection; the collection module collects detection data, including: target map ID, coordinate areas where the target map is functioning normally, coordinate areas where the target map is functioning abnormally, anomaly classification, and additional materials; the map status analysis module evaluates the detection data to obtain status scores and anomaly classifications for different map areas; and the map status feedback module, when the status score is less than a preset value, summarizes the areas to be updated and feeds back the corresponding data of the summarized areas to the map generation device. The corresponding data of the summarized areas to be updated includes: coordinate areas where the target map is abnormal, anomaly classification, and additional materials.

[0017] Further, optionally, the detection module includes: a first detection unit and a second detection unit, wherein the first detection unit is used to receive map feedback information sent by the application terminal in passive detection mode and initiate map detection based on the map feedback information; the second detection unit is used to generate a map detection task based on the scene change characteristics and specific requirements of the map in active detection mode and initiate map detection based on the map detection task.

[0018] This invention provides a map operation system. A map generation device is used to determine whether to initiate a material collection task based on acquired first map information. If the determination is yes, it generates a target map, stores the target map, and generates a map distribution request corresponding to the target map. A map distribution device, connected to the map generation device, is used to receive the map distribution request and update the second map information corresponding to the target map to a server according to the map distribution request. A server, connected to both the map distribution device and an application terminal, is used to acquire and store the second map information and call corresponding data according to the data processing needs of the application terminal. The application terminal, connected to both the server and a map monitoring device, is used to obtain corresponding operations from the server according to map usage request instructions and to feed back the data in the map usage request instructions to the map monitoring device, so that the map generation device, map distribution device, and server execute corresponding operations based on the data. The map monitoring device, connected to the map generation device, application terminal, and server, is used to monitor the status of the map used by the application terminal and / or the server, and update the map data with changing status to the map generation device. This achieves the technical effect of both generating AR maps and providing long-term effective updates and management of AR maps. Attached Figure Description

[0019] The accompanying drawings, which are included to provide a further understanding of the invention and form part of this application, illustrate exemplary embodiments of the invention and, together with their description, serve to explain the invention and do not constitute an undue limitation thereof. In the drawings:

[0020] Figure 1 A schematic diagram of a map operation system provided in an embodiment of the present invention;

[0021] Figure 2 This is a schematic diagram illustrating the relationships between nodes in a map operation system provided by an embodiment of the present invention;

[0022] Figure 3 This is a schematic diagram illustrating map generation in a map operation system provided by an embodiment of the present invention;

[0023] Figure 4 This is a schematic diagram of map monitoring in a map operation system provided in an embodiment of the present invention. Detailed Implementation

[0024] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0025] It should be noted that the terms "first," "second," etc., in the specification, claims, and drawings of this invention are used to distinguish different objects, rather than to limit a specific order.

[0026] It should also be noted that the various embodiments of the present invention described below can be executed individually or in combination with each other, and the embodiments of the present invention do not impose specific limitations in this regard.

[0027] Technology related to the embodiments of this application:

[0028] AR maps contain digital content including, but not limited to: 3D point clouds, 3D meshes (a gridded representation of 3D space), scene feature libraries, and the association information between features and point clouds. The scene feature library is primarily used for device localization and takes various forms, generally depending on the device's localization method. For example, commonly used vision-based localization methods perform 3D localization based on environmental images captured by the device. In this case, the AR map's feature library must contain: global features from several 2D images (used to determine which images in the feature library are most similar to the images captured by the current device), and local features from the aforementioned 2D images (used to determine which 2D feature points on which images match which feature points in the current device's captured images, i.e., corresponding to the same spatial point in the physical world; finally, through the matching relationships of the aforementioned 2D feature points and the correspondence between 2D feature points in the feature library and 3D map points, the matching relationship between the 2D feature points in the current device's captured image and the 3D spatial points in the map can be determined, thereby calculating the current device's 3D pose). Additionally, 3D point clouds / meshes are mainly used for content editing, helping content creators determine the spatial relationship between virtual objects and real-world scenes, and providing realistic object occlusion relationships during AR rendering.

[0029] Based on the above, embodiments of the present invention provide a map operation system. Figure 1 This is a schematic diagram of a map operation system provided in an embodiment of the present invention; as shown. Figure 1 As shown, the map operation system provided in this application embodiment includes:

[0030] The system comprises a map generation device 10, a map distribution device 12, a map monitoring device 14, a server 16, and an application terminal 18, wherein...

[0031] The map generation device 10 is used to determine whether to initiate a material collection task based on the acquired first map information. If the determination result is yes, it generates a target map, stores the target map, and generates a map distribution request corresponding to the target map.

[0032] Specifically, the first map information received during the map generation process may include: original materials, map ID, geographic location information, specific production requirements, and other external resources that assist in map generation.

[0033] The original materials can be a series of scene images taken on site;

[0034] Geographic location information can be used to define the electronic fence of the scene corresponding to the target map;

[0035] Specific production requirements may include updating only the map point cloud or only the map positioning resources;

[0036] Other external resources for auxiliary map production can be on-site CAD drawings;

[0037] Based on the first map information, the map generation device 10 determines whether a material collection task needs to be initiated during the map production process. After obtaining the required materials, it performs map generation operations such as initial reconstruction, secondary update, or coordinate system alignment of the relevant map blocks. Finally, it archives the map and sends a map distribution request to the map distribution device 12.

[0038] The map generation device 10 is also used to reconstruct or update the map according to the received instructions if the determination result is negative.

[0039] Specifically, without initiating a material collection task, the map may be regenerated, such as by adjusting parameters and rebuilding the map, or by the map monitoring device 14 having already provided enough material, so there is no need to collect it separately.

[0040] The map distribution device 12 is connected to the map generation device 10 and is used to receive map distribution requests and update the second map information corresponding to the target map to the server 16 according to the map distribution requests.

[0041] Specifically, after receiving a map distribution request, the map distribution device 12 will parse out the specific distribution information, namely, the second map information in this embodiment of the application. The second map information includes: the ID of the map block to be distributed, the geographical location, the map data archive address, etc., and update the second map information to the corresponding server 16 according to specific rules.

[0042] Server 16 is connected to map distribution device 12 and application terminal 18 respectively, and is used to obtain second map information, store the second map information, and call corresponding data according to the data processing needs of application terminal 18.

[0043] Among them, such as Figure 1 As shown, the server 16 in this embodiment includes a central server and several edge servers.

[0044] Application terminal 18 is connected to server 16 and map monitoring device 14 respectively, and is used to obtain corresponding operations from server 16 according to map usage request instructions, and to feed back the data in map usage request instructions to map monitoring device 14 so that map generation device 10, map distribution device 12 and server 16 perform corresponding operations according to the data.

[0045] Specifically, in common applications of AR maps in this embodiment, the main user of the map (i.e., the application terminal 18 of this application) may include a cloud positioning module and a content editing module. The cloud positioning module receives images captured by the on-site terminal device, referred to as a query image; and transmits the query image to the cloud positioning server. The cloud positioning server then performs pose calculation on the query image based on existing map positioning resources and sends the pose calculation result back to the on-site terminal device. The content editing platform is used primarily to place and animate virtual content based on the map's 3D point cloud (a dataset of 3D coordinate points arranged in a regular grid) or Mesh (a gridded 3D model representation), and deploys the generated content resources to the corresponding server for the terminal to load and use.

[0046] In this embodiment, the content editing platform may include: editor software or webpage.

[0047] In this embodiment, the cloud positioning module needs to be deployed on both the server and the mobile device. Specifically, after the positioning service on the mobile device is activated, it receives the terminal image and uploads it to the server. The server then completes the pose calculation before sending the image to the mobile device's positioning service.

[0048] In this embodiment of the application, in order to realize dynamic monitoring of the map, the cloud positioning server and the content editing platform are configured to communicate with the map monitoring device 14, so as to transmit map feedback information according to the established protocol.

[0049] It should be noted that the application terminal 18 is only described using the content editing platform and cloud positioning module as examples, to implement the map operation system provided in this application embodiment, and is not specifically limited. The map monitoring device 14 is connected to the map generation device 10, the application terminal 18 and the server 16 respectively, and is used to monitor the status of the map used by the application terminal 18 and / or the server 16, and update the map data with status changes to the map generation device 10.

[0050] Specifically, in this embodiment, the task of the map monitoring device 14 is to monitor the status of the deployed map, for example, to determine if there are map defects in map tiles or locations, and to feed back the map anomalies and useful raw materials to the map generation device 10, which then updates the map and starts the next cycle. The map monitoring device 14 performs map monitoring through two channels:

[0051] First, by agreeing on a data interface with map users, map users (such as cloud positioning modules and content editing modules) can automatically provide feedback on map operation information (e.g., positioning failure in a certain area, missing point cloud in a certain area);

[0052] Second, the map monitoring device 14 can initiate map detection tasks in a timely manner. For example, the operation map of a shopping mall can be set to be checked regularly every month.

[0053] In summary, as Figure 1 As shown, the map operation system in this application embodiment can be applied to AR / VR technology, especially to the generation and management of AR / VR maps. The map operation system in this application embodiment is illustrated using AR technology as an example:

[0054] In the augmented reality application, the map operation system in this application embodiment, in order to achieve efficient operation of large-scale, long-term AR maps, may include the following specific forms of AR maps, the required raw data (or raw materials), and specific production methods: the production of 3D point clouds of maps based on laser equipment, the construction of 3D point clouds based on pure visual image capture, or a hybrid form of laser and visual imaging.

[0055] The map operation system in this application embodiment is mainly based on visual image capture, and the main components of the AR map (i.e., the target map in this application embodiment) are: sparse map, dense point cloud / mesh and positioning resources, wherein the positioning resources include: global features and local feature library of the image, and three-dimensional map points corresponding to each local feature.

[0056] The map organization logic of the map operation system in this application embodiment is as follows:

[0057] Map organization logic includes map partitioning and map hierarchy. Each map tile represents a minimum independent production unit, containing its own map output: a sparse map, a dense point cloud / mesh, and location resources, corresponding to a region of the real-world scene; this is called a leaf node map. Furthermore, several minimum map tiles can be merged into a larger map node, and multiple larger nodes can be merged into even higher-level larger nodes; these can all be called parent node maps. Both parent node maps and leaf node maps have a unique map ID as an identifier and can be appended with corresponding geographic location information (such as GPS coordinate system-based geographic location information): including the geographic location of the map origin and the electronic fences of the map boundaries. Figure 2 As shown, Figure 2 This is a schematic diagram illustrating the relationships between nodes in a map operation system provided by an embodiment of the present invention;

[0058] It should be noted that only leaf node maps have their own independent map resources. The map resources of parent nodes are hierarchical nested combinations of the corresponding child node resources, and the coordinate systems between child maps may or may not be aligned. For example... Figure 2As shown, Seq1 to Seq44 are multiple leaf node map tiles, while the other nodes represent the parent node map at the upper level.

[0059] based on Figure 2 For each map node, the map production subtask can include:

[0060] Sparse map reconstruction: For leaf node maps, a sparse point cloud map of the current map tile can be reconstructed based on the input original image material. Common methods include the SFM algorithm (Structure From Motion).

[0061] Dense map reconstruction: For leaf node maps, dense point cloud maps can be generated based on sparse point cloud map data. Common methods include the MVS algorithm (Multi-View-Stereo).

[0062] Mesh reconstruction: For leaf node maps, a 3D mesh map can be generated based on dense map data. Common methods include Poisson reconstruction and Delaunay subdivision. In addition, texture mapping can be performed on the mesh to obtain a 3D mesh with clearer and more visible texture. Common methods include the open-source tool MVS-Texturing.

[0063] Location map reconstruction: For leaf node maps, location map resources with specific features can be generated based on sparse map data. The general process includes: global image feature extraction, local image feature extraction, local feature matching, and 3D point cloud triangulation.

[0064] Sparse map supplement: For leaf node maps, a new sparse map can be created based on newly added original material images and existing sparse maps, which contains both new and old materials, and the coordinate system of the new map can remain unchanged.

[0065] Sparse Map Removal: For leaf node maps, this feature allows you to delete specified portions of sparse map data from an existing sparse map without affecting the remaining map data. There are several ways to specify the portion to be deleted. For example, you can specify a coordinate region and delete the sparse map data within that region; or you can directly specify a list of original source files and delete the sparse map data corresponding to the original images in that list (generally, the sparse map will always maintain the correspondence between each original image and the sparse map data).

[0066] Map Transformation: For leaf node maps, all relevant products of the map can be transformed to another coordinate system based on the input transformation matrix.

[0067] Map alignment: For maps with multiple leaf nodes, one leaf node can be used as the base map. The coordinate transformation matrix from the base map to the other leaf node maps can be automatically calculated. Based on these matrices, multiple maps can be aligned to a unified coordinate system.

[0068] Map Tile Decomposition: A single leaf node map can be divided into multiple leaf node map tiles.

[0069] Map merging: For multiple leaf node maps, they can be merged back into a single leaf node map tile.

[0070] Map organization: Based on the hierarchical structure of the entire map database, complete the initialization, adjustment and update of the organizational structure of relevant map nodes, such as adding leaf nodes when creating a new map, changing parent nodes midway, adding parent nodes, etc.

[0071] Geographic location information annotation:

[0072] For leaf node maps, if the original collected material contains the geographic location information (such as GPS coordinates) of each image, the geographic location information of the map tile can be automatically determined after the sparse map is reconstructed; otherwise, the above geographic location information needs to be provided manually after the map tile is reconstructed.

[0073] For a parent map node, if its child maps are aligned, the origin of the reference child map becomes the origin of the current map, thus determining its geographic coordinates. If the child maps are not aligned, a child map is randomly selected as the reference child map, inheriting its geographic coordinates. In both cases, the geofence information is obtained by taking the union of the geofences of the corresponding child maps.

[0074] Based on the aforementioned map organization logic, the map operation system in this embodiment, building upon the map organization logic and callable map production sub-tasks, performs the following map generation process: Figure 3 As shown, Figure 3 This is a schematic diagram illustrating map generation in a map operation system provided by an embodiment of the present invention;

[0075] Optional, such as Figure 3 As shown, based on the map generation process, the map generation device 10 may include: a map generation instruction analysis module, a material collection module, a generation task execution module, and a map archiving module, wherein,

[0076] The map generation instruction analysis module is used to determine the source of the map generation instruction, parse the map generation instruction after the source is determined, and obtain the instruction content in the map generation instruction;

[0077] Specifically, the sources of map generation instructions include: first, the map monitoring module will send production instructions from time to time; second, system administrators or other authorized external users can directly initiate production instructions.

[0078] Optionally, the instruction content includes: the target map ID to be processed, where the target map ID indicates whether the target map to be processed is a newly created map; a local range in the target map to be processed, which is specified by a list of material files or a pre-acquired coordinate range; wherein, when processing a historical map, the local range is obtained; additional material information, where the additional material information includes: whether original material is attached; the additional material information is used to indicate whether the original material data is provided in advance for this map generation task, and the access address of the original material data; wherein, the original material data is sent by the map monitoring device and will be automatically uploaded to the server 16, recording the access address; generation requirements, where the generation requirements include: indicating the operations to be performed on the map, and related parameters; the operations are a combination of map generation sub-tasks.

[0079] Specifically, the instructions include:

[0080] The target map ID to be processed may not exist in the map database, in which case it indicates a newly created map. That is, if the target map ID does not exist, it means a new map is being created; if the target map ID exists, it means a historical map is being processed.

[0081] The local area in the target map to be processed may be specified through a list of source files or by directly providing the coordinate range. This information is generally only available when processing historical maps.

[0082] Additional Material Information: Whether original materials are included. This only indicates whether original material data was provided in advance for this task, and the access address of that data, without including the data itself. Generally, this data is provided by map monitoring devices and is automatically uploaded to the system server, recording the access address.

[0083] Production requirements: This specifies the operations to be performed on the map and the relevant parameters. These operations may be different combinations of the basic subtasks mentioned above. For example, a command to create a new map can be initiated, and the system will execute a series of subtasks such as map tiling, sparse reconstruction, and dense reconstruction. Alternatively, a single subtask—dense reconstruction—can be initiated. The system provides templates for various production commands, and the associated subtasks behind each template are pre-configured.

[0084] The material collection module is connected to the map generation instruction analysis module. It is used to determine whether to collect materials based on the content of the map generation instruction. If it is determined that material collection should be carried out, material data is collected.

[0085] Specifically, the execution of the material collection module depends on the selection and input of the map generation instructions. For example, if a dense reconstruction of a historical map is required, no new materials need to be collected. However, if a new map is being created, a material collection task needs to be created in this stage, and the data collector will complete the collection and uploading of the materials. If the map monitoring device 14 has provided relevant materials in advance, the material data can be obtained directly based on the material access address.

[0086] The task execution module is connected to the material collection module and is used to execute map generation sub-tasks based on map generation instructions and the task configuration templates corresponding to the map generation instructions.

[0087] Optionally, the map generation subtasks include: creating a new map, supplementing a map, and aligning a map. When creating a new map, sparse map reconstruction, map tiling, and map organization are performed sequentially. For each sub-map tile, dense map reconstruction, mesh reconstruction, localization map reconstruction, and geographic location information annotation are performed sequentially to obtain the target map. When supplementing a map, sparse map supplementation, dense map reconstruction, mesh reconstruction, localization map reconstruction, and geographic location information annotation are performed sequentially to obtain the supplemented map. When aligning a map, map alignment, map transformation, and geographic location information annotation are performed sequentially to obtain the aligned map.

[0088] This process involves executing a series of map generation sub-tasks based on different map generation instructions and corresponding task configuration templates. The following are examples of the execution process for several map generation instructions.

[0089] When creating a new map, the map operation system in this application embodiment will sequentially perform: sparse map reconstruction, map segmentation, and map organization. For each sub-map block, the following will be performed sequentially: dense map reconstruction, mesh reconstruction, location map reconstruction, and geographic location information annotation.

[0090] To supplement the map, the system sequentially performs the following steps for the target map tile: sparse map supplementation, dense map reconstruction, mesh reconstruction, location map reconstruction, and geographic location information annotation.

[0091] Map alignment, in this embodiment of the application, the map operation system will perform: map alignment, map transformation, and geographic location information annotation.

[0092] The map archiving module, connected to the task generation execution module, is used to upload the generated map to the target directory after the map generation subtask is completed, and to synchronize the change information of the target directory of the map database to the map distribution device.

[0093] Uploading the generated map to the target directory includes uploading the corresponding information of the generated map to the target directory. This information includes: the target map ID, the latest map organization information, and a brief update description; or, the latest output of the leaf node map involved. The latest output of the leaf node map includes: output data, version information of each output, and a brief update description of the output data.

[0094] The change information includes: the target map ID, the latest map organization information, a brief update description; or, the access address of the latest output of the affected leaf node map.

[0095] In this embodiment, the map generation device 10 automatically divides and hierarchically manages the map, and provides a variety of key functions such as map building, updating, and alignment, so as to realize the organized management and efficient updating of the map.

[0096] Optionally, the map distribution device 12 includes: a map service aggregation module, an information acquisition module, and a map distribution module, wherein,

[0097] The map service aggregation module is used to aggregate map server information, which includes: server network address, geographical location, and current load.

[0098] This includes aggregated map server information, such as: each server's network address, geographical location, and current load.

[0099] The information acquisition module is used to acquire map update information, which includes: target map ID, geographic location information, map organization information, leaf node map artifact archive address, and brief map update information;

[0100] This includes obtaining map update information, which includes the target map ID, geographic location information, map organization information, leaf node map artifact archive address, and brief map update information (newly created maps can also be considered as a type of update from scratch).

[0101] The map distribution module is used for map data distribution, which includes: determining the target server to which the map will be distributed; and determining the map to be updated.

[0102] Specifically, map data distribution mainly considers two aspects:

[0103] Determine which server the map is distributed to (i.e., determine the target server for map distribution in this embodiment): If a map has already been deployed on the corresponding server, the map continues to be updated to the corresponding server; otherwise, obtain the network latency information from the geographical location of the target map to each server, select the server with network latency less than a preset threshold as a candidate, and proceed to the next step; if the server load is less than the preset threshold, the central server is selected first, otherwise the server with the load less than the preset threshold and the smallest network latency is selected from the edge servers.

[0104] Determine which map products need to be updated (i.e., determine the map to be updated in this embodiment): Based on the brief map update information, determine whether all products of the target map need to be updated, or only some products need to be updated. For example, for a newly created map, all products need to be updated; for a map that has only updated the location resources, only the location resource data needs to be updated; for a parent node map, if only a leaf node map has been added this time, only the corresponding child map products need to be updated.

[0105] In the map distribution device 12 of this application embodiment, the map can be automatically distributed to the best server according to the map geographic information, and the map data can be partially replaced according to the map update results.

[0106] Optional, such as Figure 4 As shown, Figure 4 This is a schematic diagram of map monitoring in a map operation system provided by an embodiment of the present invention; the map monitoring device 14 includes: a detection module, a collection module, a map status analysis module, and a map status feedback module, wherein,

[0107] The detection module is used to initiate map detection;

[0108] The detection module includes a first detection unit and a second detection unit. The first detection unit is used to receive map feedback information sent by the application terminal in passive detection mode and initiate map detection based on the map feedback information. The second detection unit is used to generate map detection tasks based on the scene change characteristics and specific requirements of the map in active detection mode and initiate map detection based on the map detection tasks.

[0109] Specifically, the detection module initiates the detection. The detection timing of the map monitoring device 14 can be configured in several ways; the following two types of detection timings can be used concurrently:

[0110] On the one hand, the passive detection mode of the map operation system provided in this embodiment of the invention is activated. In this mode, the map monitoring device 14 can periodically and automatically receive map feedback information transmitted from the cloud positioning server and the content editing platform.

[0111] On the other hand, based on the scene change characteristics of different maps and the requirements of the owner, proactive map inspection tasks can be initiated periodically. At this time, testers can go to the site and use test software to perform specific map inspections.

[0112] The collection module is used to collect detection data, which includes: target map ID, coordinate areas of the target map that are functioning normally, coordinate areas of the target map that are functioning abnormally, anomaly classification, and additional materials.

[0113] Specifically, detection data is collected through a data collection module. Whether in passive or active detection mode, the map monitoring device 14 can collect specific detection data from the corresponding detection party (cloud positioning module, content editing platform, or detection software), mainly including:

[0114] Target map ID, coordinate area where the target map is functioning normally, coordinate area where the target map is malfunctioning, specific malfunction classification: missing point cloud / mesh structure, incorrect point cloud / mesh structure, insufficient point cloud / mesh structure accuracy, cloud positioning failure, etc.; Additional materials: mainly including on-site images returned during cloud positioning or active map detection.

[0115] Among them, the failure or success of cloud positioning can be automatically identified and the relevant data can be saved by the cloud positioning server. Anomalies related to point cloud / mesh can be discovered and marked by content editors during use, or the point cloud or mesh can be rendered and superimposed onto the images captured by the field equipment by the testing software, and then identified and marked by the testers.

[0116] The map status analysis module is used to evaluate the data and obtain status scores and anomaly classifications for different areas of the map.

[0117] Specifically, map status analysis is performed through a map status analysis module. Based on the collected normal or abnormal map information, this module executes a map status evaluation algorithm and outputs status scores and anomaly classifications for different map regions (missing point cloud / mesh structure, incorrect point cloud / mesh structure, insufficient point cloud / mesh structure accuracy, incomplete location map features, etc.). Figure 3 (Insufficient number of points D, etc.)

[0118] The map status feedback module is used to summarize the areas to be updated when the status score is less than the preset value, and then feed back the corresponding data of the summarized areas to the map generation device.

[0119] The corresponding data for the area to be updated (i.e., the map data of the state changes in this application embodiment) includes: the coordinate area of ​​the target map where anomalies occur, the anomaly classification, and additional materials.

[0120] Specifically, map status feedback is provided through the map status feedback module. When the status score of certain map areas falls below a preset threshold, the map monitoring device 14 will aggregate the areas to be updated and then feed back the relevant data (i.e., the map data showing status changes in this embodiment) to the map generation device 10. This data includes: the target map ID; the coordinates of the target map where anomalies occur; and specific anomaly classifications: missing point cloud / Mesh structure, incorrect point cloud / Mesh structure, insufficient point cloud / Mesh structure accuracy, incomplete positioning map features, and positioning location... Figure 3 Insufficient number of D points, etc.; Additional materials: mainly including on-site images returned during cloud positioning or active map detection.

[0121] In summary, the map monitoring device 14 in this embodiment can automatically and irregularly collect map usage problems and necessary data from map users, or it can collect problems and data through dedicated map detection, automatically update and warn of map status, and send relevant data back to the map production module.

[0122] The map operation system provided in this embodiment of the invention enables automated, efficient production and long-term maintenance of AR maps.

[0123] This invention provides a map operation system. A map generation device is used to determine whether to initiate a material collection task based on acquired first map information. If the determination is yes, it generates a target map, stores the target map, and generates a map distribution request corresponding to the target map. A map distribution device, connected to the map generation device, is used to receive the map distribution request and update the second map information corresponding to the target map to a server according to the map distribution request. A server, connected to both the map distribution device and an application terminal, is used to acquire and store the second map information and call corresponding data according to the data processing needs of the application terminal. The application terminal, connected to both the server and a map monitoring device, is used to obtain corresponding operations from the server according to map usage request instructions and to feed back the data in the map usage request instructions to the map monitoring device, so that the map generation device, map distribution device, and server execute corresponding operations based on the data. The map monitoring device, connected to the map generation device, application terminal, and server, is used to monitor the status of the map used by the application terminal and / or the server, and update the map data with changing status to the map generation device. This achieves the technical effect of both generating AR maps and providing long-term effective updates and management of AR maps.

[0124] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0125] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0126] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0127] The above description is merely a preferred embodiment of the present invention and is not intended to limit the scope of protection of the present invention.

Claims

1. A map operation system, characterized in that, include: Map generation device, map distribution device, map monitoring device, server, and application terminal, among which, The map generation device is used to determine whether to initiate a material collection task based on the acquired first map information. If the determination result is yes, it generates a target map, stores the target map, and generates a map distribution request corresponding to the target map. The map distribution device is connected to the map generation device and is used to receive the map distribution request and update the second map information corresponding to the target map to the server according to the map distribution request. The server is connected to the map distribution device and the application terminal respectively, and is used to obtain the second map information, store the second map information, and call the corresponding data according to the data processing needs of the application terminal. The application terminal is connected to the server and the map monitoring device respectively, and is used to obtain the corresponding operation from the server according to the map usage request instruction, and to feed back the data in the map usage request instruction to the map monitoring device, so that the map generation device, the map distribution device and the server perform the corresponding operation according to the data; The map monitoring device is connected to the map generating device, the application terminal, and the server respectively, and is used to monitor the status of the map used by the application terminal and / or the server, and update the map data with the changed status to the map generating device. The map distribution device includes: a map service aggregation module, an information acquisition module, and a map distribution module. The map service aggregation module aggregates map server information, including server network address, geographical location, and current load. The information acquisition module acquires map update information, including target map ID, geographical location information, map organization information, leaf node map artifact archive address, and brief map update information. The map distribution module distributes map data, including: determining the target server to which the map will be distributed; and determining the map to be updated. The first map information includes: original materials, map ID, geographic location information, specific production requirements, and other external resources that assist in map generation; the geographic location information is the electronic fence of the scene corresponding to the target map; the specific production requirements are to update only the map point cloud or only the map positioning resources; the other external resources that assist in map generation are on-site CAD drawings; The second map information includes: the ID of the map tile to be distributed, its geographical location, and the map data archive address.

2. The system according to claim 1, characterized in that, The map generation device is also used to reconstruct or update the map according to the received instructions if the determination result is negative.

3. The system according to claim 2, characterized in that, The map generation device includes: The system comprises a map generation instruction analysis module, a material collection module, a generation task execution module, and a map archiving module. The map generation instruction analysis module is used to determine the source of the map generation instruction and parse the instructions after determining the source. The map generation instruction is used to obtain the instruction content within the map generation instruction. The material collection module is connected to the map generation instruction analysis module and is used to determine whether to collect materials based on the instruction content of the map generation instruction. If it is determined that material collection should be carried out, material data is collected. The task execution module is connected to the material collection module and is used to execute map generation sub-tasks according to the map generation instruction and the task configuration template corresponding to the map generation instruction. The map archiving module is connected to the task generation execution module and is used to upload the generated map to the target directory after the map generation sub-task is completed, and to synchronize the change information of the target directory in the map database to the map distribution device.

4. The system according to claim 3, characterized in that, The instructions include: The target map ID to be processed is used to indicate whether the target map to be processed is a newly created map; The local area in the target map to be processed is specified by a list of source files or a pre-acquired coordinate range; wherein, the local area is obtained when processing a historical map; Additional material information, including whether original materials are attached; the additional material information is used to indicate whether original material data is provided in advance for this map generation task, and the access address of the original material data; wherein, the original material data is sent by the map monitoring device, and the original material data is automatically uploaded to the server, and the access address is recorded; The generation requirements include: indicating the operations performed on the map and related parameters; the operations are a combination of the map generation sub-tasks.

5. The system according to claim 3, characterized in that, Uploading the generated map to the target directory includes: The generated map information is uploaded to the target directory, wherein the information includes: Target map ID, latest map organization information, brief update notes; or, The latest output of the leaf node map involved includes: output data, version information of each output, and a brief update description of the output data.

6. The system according to claim 5, characterized in that, The change information includes: The target map ID, the latest map organization information, and the brief update description; or, The access address of the latest output of the leaf node map involved.

7. The system according to claim 3, characterized in that, The map monitoring device includes: The system comprises a detection module, a data collection module, a map status analysis module, and a map status feedback module. The detection module is used to initiate map detection; The collection module is used to collect detection data, which includes: target map ID, coordinate areas of the target map that are operating normally, coordinate areas of the target map that are operating abnormally, anomaly classification, and additional materials. The map status analysis module is used to evaluate the detection data and obtain status scores and anomaly classifications for different areas of the map. The map status feedback module is used to summarize the areas to be updated when the status score is less than a preset value, and feed back the corresponding data of the summarized areas to be updated to the map generation device; wherein, the corresponding data of the summarized areas to be updated includes: the coordinate areas of the target map where anomalies occur, the anomaly category, and additional materials.

8. The system according to claim 7, characterized in that, The detection module includes: a first detection unit and a second detection unit, wherein... The first detection unit is used to receive map feedback information sent by the application terminal in passive detection mode, and initiate map detection based on the map feedback information; The second detection unit is used to generate a map detection task based on the scene change characteristics and specific requirements of the map in active detection mode, and to initiate the map detection based on the map detection task.

Images