Indoor and outdoor GIS real-time cooperation and position matching method based on MR equipment

Abstract

The invention relates to the technical field of mixed reality (MR) and geographic information system (GIS), and provides an MR equipment-based indoor and outdoor GIS real-time cooperation and position matching method, which integrates real geographic space information and virtual geographic space information, combines space positioning and object identification technologies, and utilizes a geographic coordinate conversion method to realize real-time cooperation and position matching of the GIS. Therefore, GIS indoor and outdoor cooperation based on MR equipment is supported. It is ensured that the three-dimensional GIS data can be accurately displayed in MR equipment in the indoor and outdoor environment, geographic information in the MR indoor and outdoor environment has consistency, and therefore accurate matching of indoor and outdoor geographic positions is achieved. According to the method, cooperative work of GIS data in MR indoor and outdoor environments is supported, and the immersive experience and interactivity of a user are improved.

Description

Technical Field

[0001] The present invention relates to the technical field of mixed reality (MR) and geographic information system (GIS), and in particular to a method for real-time indoor and outdoor GIS collaboration and position matching based on MR equipment. Background Art

[0002] Mixed Reality (MR) technology is a new interaction and visualization technology that is a fusion of Virtual Reality (VR) and Augmented Reality (AR). It inherits the human-computer interaction method of VR and the environmental fusion method of AR, and enhances the perception of environmental space. Through depth perception and spatial mapping of the user's surroundings, it combines virtual information and the real-world environment to achieve a highly interactive and immersive user experience.

[0003] MR technology is revolutionizing multiple industries by seamlessly combining digital content with the real world. In terms of geographic information system (GIS) data visualization, MR technology can break through the limitations of two-dimensional screens, overlay GIS data into the real world in three dimensions, and directly interact with geospatial data in the real environment, deepening users' understanding and perception of geographic information. Existing GIS collaborative designs based on MR devices include indoor sandbox collaborative work systems and outdoor virtual battlefield drill systems, but in terms of real-time collaboration in indoor and outdoor scenes, existing technologies are difficult to provide a truly immersive and interactive user experience.

[0004] In the display of 3D GIS data by MR, the indoor environment generates a miniature model by reducing the scale of the 3D geographic scene model and the auxiliary model at the same time, so as to provide users with a 3D macroscopic observation and operation perspective of geographic information through the MR device in the indoor environment. The above auxiliary model refers to a virtual or real object model that will be used to support the completion of a collaboration when working indoors and outdoors. It is not necessarily a necessary component of the established 3D geographic scene; while the MR device of the collaborator in the outdoor environment displays the real geographic scene, and the auxiliary model in the MR indoor terminal can be correctly superimposed with the real geographic scene in proportion to display in the MR outdoor terminal. The above technical difficulty lies in that since the MR device in the indoor working environment displays a scaled 3D geographic scene but it has a certain geographic coordinate, and the MR device in the outdoor working environment displays the real geographic world, then how to accurately display the auxiliary model displayed by the MR device in the indoor working environment in the outdoor environment MR device during collaborative work? It is necessary to accurately match the auxiliary model in the indoor MR with the real geographic environment displayed by the outdoor MR, and ensure the accurate correspondence between the auxiliary model and the real position in the MR outdoor terminal, so as to help users successfully complete the collaboration in the outdoor environment.

[0005] However, the development environment of MR applications is different from that of GIS. MR devices usually use a Cartesian coordinate system with (0, 0, 0) as the origin, and its origin is usually determined by the user's location when starting the MR application. GIS data usually uses geographic coordinate systems such as WGS84, which results in the auxiliary model correctly displayed in the indoor environment MR not being correctly loaded at the corresponding position in the outdoor environment MR, because the two coordinate systems cannot be directly adapted and need to be converted. How to convert the geographic coordinates of GIS data into a coordinate system suitable for the MR environment, and ensure that the auxiliary model in the MR indoor terminal is correctly displayed and the auxiliary model in the corresponding MR outdoor terminal can be accurately matched with the real geographic scene to support the collaborative work of indoor and outdoor scenes is a key issue to be solved. Summary of the invention

[0006] The present invention is an innovative invention involving MR indoor terminal three-dimensional GIS and MR outdoor terminal three-dimensional GIS, and their collaborative work. It specifically proposes a method for real-time collaboration and position matching of indoor and outdoor GIS based on MR devices, aiming to solve the problem of position matching and mixed superposition display of virtual bodies in outdoor real environments under indoor and outdoor collaborative environments based on MR devices. By integrating real geographic space information and virtual geographic space information, combining spatial positioning and object recognition technology, and utilizing geographic coordinate conversion methods, GIS indoor and outdoor collaboration based on MR devices is supported. It ensures that three-dimensional GIS data can be accurately displayed in MR devices in indoor and outdoor environments, so that the geographic information in MR indoor and outdoor environments is consistent, thereby achieving accurate matching of indoor and outdoor geographic locations. This method supports the collaborative work of GIS data in MR indoor and outdoor environments, and enhances the user's immersive experience and interactivity.

[0007] A method for real-time indoor and outdoor GIS collaboration and location matching based on MR equipment mainly includes the design of MR indoor terminals, the design of MR outdoor terminals, and accurate geographic location matching based on MR indoor and outdoor terminals, wherein:

[0008] (1) For MR indoor terminals:

[0009] In the present invention, the MR indoor terminal loads the established three-dimensional geographic scene data and auxiliary model data of the research area. However, since the data format supported by the current MR device is incompatible with the traditional GIS data format, it is necessary to perform format conversion, coordinate conversion, and model integration on the established three-dimensional geographic scene data and auxiliary model data so as to correctly display and perform GIS operations in the MR environment. The technical steps involved are: data format conversion, coordinate conversion, model integration, and model scaling.

[0010] (2) For MR outdoor terminals:

[0011] The MR device of the outdoor environment collaborator displays the real geographical scene, and can correctly overlay the auxiliary model in the MR indoor terminal with the real geographical scene in equal proportion on the MR outdoor terminal to help users perform tasks in the outdoor environment. The key technical steps involved are: format conversion, feature map acquisition and processing, spatial positioning and coordinate conversion, model integration and position adjustment, and direction calibration.

[0012] (3) Accurate indoor and outdoor geographic location matching based on MR:

[0013] When using MR devices for indoor and outdoor collaborative work, the position of the auxiliary model in the MR indoor terminal in the three-dimensional geographic scene needs to be accurately matched with the position of the auxiliary model in the MR outdoor terminal in the real geographic scene to support the collaborative work of the MR devices in indoor and outdoor environments. When the auxiliary model is moved or marked in the three-dimensional geographic scene using the MR terminal indoors, the auxiliary model of the MR outdoor terminal can also synchronously reflect the operation and match it to the accurate position in the real geographic scene for display, so as to achieve efficient collaboration across indoor and outdoor environments.

[0014] The beneficial effects achieved by the present invention are:

[0015] (1) The present invention significantly improves the consistency of 3D GIS data in indoor and outdoor scenes by achieving accurate geographic location matching of GIS data in MR environments. Through the coordinate conversion method, the present invention ensures the accurate display of the model location of GIS data in indoor and outdoor environments, allowing users to intuitively operate and analyze geographic information in miniature models and real geographic scenes. In particular, the real-time collaborative method of indoor and outdoor geographic locations can achieve seamless and accurate geographic location matching, and the present invention realizes efficient MR device collaboration across indoor and outdoor scenes.

[0016] (2) The user can edit and adjust the model in the miniature model of the indoor terminal in real time and synchronize it to the real scene overlay model of the outdoor terminal, so that it can be accurately displayed in the actual environment. The present invention extends the real-time collaborative work based on MR to the collaboration of indoor scenes and outdoor large geographical scenes, which not only improves the collaborative work efficiency and experience of users in indoor and outdoor environments based on MR devices, but also expands the application scenarios of MR, making the collaborative application scenarios based on MR more extensive and reliable. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 It is a schematic diagram of the display effect of the MR indoor terminal in an embodiment of the present invention.

[0018] Figure 2 It is a schematic diagram of the spatial positioning of the MR outdoor terminal and the model superposition effect in an embodiment of the present invention.

[0019] Figure 3 It is a schematic diagram of precise indoor and outdoor geographic location matching based on MR in an embodiment of the present invention.

[0020] Figure 4 Schematic diagram of earth coordinate system conversion parameters in an embodiment of the present invention.

[0021] Figure 5 Schematic diagram of the ENU coordinate system in the MR application in the embodiment of the present invention.

[0022] Figure 6Schematic diagram of multi-source GIS data model integration in an embodiment of the present invention.

[0023] Figure 7 Schematic diagram of the influence of different reference point positions on the model scaling operation in an embodiment of the present invention.

[0024] Figure 8 Schematic diagram of the model scaling operation in an embodiment of the present invention.

[0025] Fig. 9 1 is a partial outdoor feature diagram in an embodiment of the present invention.

[0026] Fig.10 Schematic diagram of the calculation of the real-world coordinates of the feature graph in an embodiment of the present invention.

[0027] Fig.11 Schematic diagram of calculation of the characteristic graph in the northeast celestial coordinates in an embodiment of the present invention.

[0028] Fig.12 The figure is a flow chart of spatial positioning based on augmented reality marking in an embodiment of the present invention. DETAILED DESCRIPTION

[0029] The technical solution of the present invention is further described in detail below in conjunction with the accompanying drawings.

[0030] Figure 1 A detailed indoor miniature model is displayed to show the effect of the MR indoor terminal. The model includes multiple buildings and the pipe systems connected to them. Through precise geographic location matching and scale processing, these complex three-dimensional entities are clearly displayed in the MR environment. Users can interact with these models intuitively through MR devices in the indoor environment, observe the spatial relationship between buildings and pipes, and perform related operations and analysis.

[0031] Figure 2The spatial positioning and model superposition effect of the MR outdoor terminal are shown. The figure includes two parts: (a) and (b) respectively show two outdoor landmarks used for spatial positioning. These landmarks help the MR device determine its position in the actual environment through their clear latitude, longitude and altitude information. The precise positioning of landmarks 1 and 2 provides the basis for the subsequent virtual model rendering. Figure (c) shows the superposition effect of the pipeline model pre-created in the outdoor environment on the real world. In the actual operation process, the MR device first captures the landmark image in the environment through the camera, and uses the pre-recorded landmark latitude, longitude and altitude data for location recognition and matching. Then, the system aligns the coordinates of the virtual pipeline model with the geographic coordinates of the real world through the spatial positioning algorithm to ensure that the scale, position and direction of the model are highly consistent with the actual environment. Finally, the virtual pipeline model is accurately mapped and rendered in the real road scene, allowing users to intuitively view the location and layout of the pipeline in the MR environment, realizing the accurate display of three-dimensional geographic entities in the outdoor environment.

[0032] Figure 3 The task execution process and effect of precise indoor and outdoor geographic location matching based on MR technology are demonstrated. In the MR indoor terminal interface above, the user first selects the location that needs to be alerted in the miniature model of the three-dimensional map and places a roadblock model. The task is operated in the miniature scene of the virtual geographic model. The user selects the area suitable for placing the roadblock by viewing the relevant information and publishes the task through the miniature model. After the task is published, the system transmits the detailed information of the task, including the specific latitude and longitude and other geographic coordinates, to the outdoor terminal.

[0033] The MR outdoor terminal below shows the execution of the task. The task panel on the left shows the detailed information of the task, including data such as the geographic coordinates of the task and the task type. The right side shows the actual outdoor scene, where the virtual roadblock model is accurately superimposed on the real road. It is worth noting that the position of the roadblock in the outdoor environment is exactly the same as the position where the user places the miniature model in the indoor terminal, ensuring the precise match of the virtual model between the two. Through precise geographic coordinate conversion and spatial matching, the consistency of the virtual model in indoor and outdoor terminals is ensured, thereby achieving seamless collaboration from indoors to outdoors.

[0034] The diagram demonstrates the efficiency of the present invention in cross-scenario collaboration, ensuring that the data operated or edited by users indoors can be accurately transmitted and mapped to the actual outdoor scene, achieving seamless connection between the real environment and virtual information, and significantly improving the user's work efficiency and interactive experience.

[0035] The present invention aims to solve the problem of how to convert the geographic coordinates of GIS data into a coordinate system suitable for the MR environment, and ensure that the auxiliary model in the MR indoor terminal is correctly displayed, and the auxiliary model in the MR outdoor terminal can accurately match the position with the real geographic scene, so as to support the collaborative work of indoor and outdoor scenes. Therefore, the present invention relates to the construction of the MR indoor terminal and the MR outdoor terminal and the accurate matching of the geographic location data between the two terminals.

[0036] The MR indoor terminal loads the established 3D geographic scene data and auxiliary model data of the research area. However, since the data format supported by the current MR device is incompatible with the traditional GIS data format, it is necessary to convert the format and coordinates of the established 3D geographic scene data and auxiliary model data, and integrate the models so that they can be correctly displayed and operated in the GIS environment.

[0037] Step 1: Format conversion.

[0038] Traditional 3D GIS data (such as Shapefile or OSGB format) is converted to a format supported by the MR environment (such as FBX or OBJ). Since the geographic coordinates may be lost during the format conversion of 3D GIS data, the geographic coordinate information of the data needs to be saved before conversion, including the longitude and latitude (lat, lon) and altitude (alt) in the WGS84 coordinate system.

[0039] Step 2: Coordinate transformation.

[0040] MR applications use a Cartesian coordinate system, with XYZ coordinates representing width, depth, and height, respectively, while 3D GIS data usually uses a geographic coordinate system, such as WGS84, in which the coordinate values ​​are longitude, latitude, and elevation. In order to convert 3D GIS data into a coordinate system compatible with MR applications, it is usually necessary to use the East-North-Up (ENU) coordinate system, which is compatible with the Cartesian coordinate system and can convert geographic coordinates into a local rectangular coordinate system. In order to convert 3D GIS data into an ENU coordinate system compatible with MR applications, such as Figure 4 As shown, the following steps are required:

[0041] a. Select a reference point (reference point): Before conversion, first select a reference point P 0 Its coordinates are expressed in longitude, latitude and altitude under the WGS84 system, denoted as: (λ 0 ,φ 0 ,h 0 ). In the MR indoor terminal, the reference point P 0Usually the center of the 3D geographic scene to maintain the geometric symmetry and spatial relative position of the model. This reference point will be used as the origin of the ENU coordinate system, and the relative coordinates of all positions will be calculated based on this.

[0042] b. WGS84 to ECEF coordinate conversion: First, you need to convert the longitude, latitude and elevation of WGS84 to the ECEF (Earth-Centered Earth-Fixed) coordinate system. The origin of the ECEF coordinate system is the center of the earth, and the directions of the coordinate axes are: X-axis: the plane passing through the prime meridian and the equator. Y-axis: perpendicular to the X-axis, passing through the equator. Z-axis: along the rotation axis of the earth (North Pole).

[0043] The conversion formula of ECEF coordinate system is:

[0044]

[0045] In the above formula, a and b represent the major and minor axes of the earth ellipsoid, respectively, and e 2 is the eccentricity of the earth ellipsoid, N is the radius of curvature. (λ, φ, h) are the geographic coordinates to be converted of the three-dimensional GIS data, and (X, Y, Z) are the ECEF coordinates of the converted three-dimensional GIS data.

[0046] c. Calculate the ECEF coordinates of the reference point: Use the reference point P 0 The longitude, latitude and altitude (λ 0 ,φ 0 ,h 0 ), according to the above formula, the WGS84 coordinates of the reference point are converted to ECEF coordinates, recorded as (X 0 ,Y 0 ,Z 0 ).

[0047] d. Convert WGS84 coordinates to ECEF coordinates (target point): Similarly, convert the longitude, latitude and elevation (λ, φ, h) of the target point to ECEF coordinates, recorded as (X, Y, Z).

[0048] e. Conversion from ECEF to ENU coordinate system: After obtaining the ECEF coordinates of the reference point and the target point, the conversion from the ECEF coordinate system to the ENU coordinate system can be performed. The coordinates (E, N, U) of the target point in the ENU coordinate system can be calculated by the following formula:

[0049]

[0050] Among them, λ 0 and φ 0 are the longitude and latitude of the reference point, (X 0,Y 0 ,Z 0 ) is the ECEF coordinate of the reference point, and (X,Y,Z) is the ECEF coordinate of the target point.

[0051] f. Result of coordinate conversion: Through the above steps, the longitude, latitude and elevation coordinates of the target point are converted into the coordinates of the east, north and up directions based on the ENU coordinate system. The ENU coordinate system is the Cartesian coordinate used in MR applications.

[0052] Step three: model integration.

[0053] In the development environment of MR applications, in order to accurately express and control virtual geographic entities, the present invention creates an Anchor object (anchor point object) as a fixed reference point for the three-dimensional geographic scene. The Anchor object is an empty parent object, and all three-dimensional GIS data are child objects of the Anchor object. Their position, rotation, and scale in the MR environment are defined relative to the local coordinates of the Anchor object. Figure 5 As shown, the reference point P 0 The position is the origin of the Anchor object. The coordinates of the 3D GIS data in the ENU coordinate system correspond to the coordinates in the local coordinate system of the Anchor object.

[0054] When 3D GIS data is imported into MR applications, their respective ENU coordinates must first be calculated through step 2. Convert the WGS84 global coordinate system of each entity to local ENU coordinates with the Anchor object as the origin, and map these coordinates (E, N, U) to the local coordinate system of the Anchor object. By adjusting the local coordinates of the Anchor object to accurately set the position of the virtual geographic entity, ensure that the position, scale, and orientation of the 3D GIS data in the Cartesian coordinate system in the MR environment are the same as those in the geographic coordinate system.

[0055] Step 4: Scale the model.

[0056] Due to the limitations of the indoor environment, 3D models are usually built in proportion, but they need to be scaled for display in an indoor environment. 0 The location selection requires special consideration. Figure 7 As shown, the reference point P 0 Different choices will affect the position of the model when scaling. 0 When the geometric center of the model is selected, the scaling operation will be performed based on the center of the model, ensuring that all parts of the model are scaled evenly relative to the center point during the scaling process. 0If the model is selected outside or at a non-central position, the model will tilt or move in the opposite direction when it is scaled. Therefore, for the construction of the miniature model of the MR indoor terminal, the geometric center of the model is selected as the reference point P 0 This can keep the geometric symmetry and relative spatial position of the model unchanged during the scaling operation.

[0057] Proportional scaling is achieved by adjusting the scaling parameters in the Transform property of the Anchor object. By adjusting the scaling parameters in the Transform property of the Anchor object, the size of all sub-objects is changed to achieve proportional scaling of the model. This not only preserves the details and structural integrity of the model, but also significantly reduces the space occupied by the model, allowing users to interact more naturally and flexibly in limited indoor spaces. The scaling formula in terms of position is:

[0058] (E',N',U')=(E,N,U)×scale<Formula 4>

[0059] Among them, (E, N, U) represents the original coordinate value, scale is the scaling factor, and (E', N', U') is the scaled coordinate value. Through this formula, the position of the model in space can be adjusted according to the specified scaling ratio to adapt to different display requirements and environmental constraints.

[0060] (2) MR outdoor terminal

[0061] The MR outdoor terminal displays the real geographical scene, and provides the auxiliary model in the MR indoor terminal to be correctly superimposed with the real geographical scene in equal proportion and displayed in the MR outdoor terminal, helping users to perform tasks in the outdoor environment. The auxiliary model construction and integration process of the MR outdoor terminal is basically the same as that of the MR indoor terminal.

[0062] Step 1: Format conversion: This is the same as step 1 of building an MR indoor terminal, and will not be repeated here.

[0063] Step 2: Feature map collection and processing.

[0064] The overlay display of 3D GIS data in an outdoor environment needs to be realized through spatial positioning technology to ensure the accurate correspondence between virtual objects and actual geographical locations. The present invention is based on the positioning method of augmented reality markers. This method assists positioning by scanning specific features (such as buildings, landmarks, etc.) in the outdoor environment. The real-world coordinates of the feature points are obtained through preliminary measurements and registered in the system. By identifying these feature points, the system can accurately locate the virtual content.

[0065] The collection of feature maps is a prerequisite for positioning. In order to accurately align virtual content with specific locations and directions in the real world, the principle of collecting feature maps is to select landmarks or objects with obvious and unique visual features. These landmarks or objects should have the following characteristics:

[0066] a. Highly recognizable: The selected feature map should be easily recognizable in the real environment, such as unique buildings, conspicuous signs or special terrain features.

[0067] b. Stability: The objects or places represented by the feature map should have long-term stability to avoid feature failure due to environmental changes.

[0068] c. Wide distribution: Feature maps should be distributed in different locations of the user's activity area to ensure effective spatial positioning at different locations.

[0069] While collecting feature maps, it is necessary to measure and record the detailed location information of each location, including latitude, longitude and altitude. The ORB algorithm is then used to detect key points in the feature maps and generate corresponding feature descriptors for each feature map. These descriptors will be registered in the system, allowing the system to accurately identify and locate feature maps. Fig. 9 These are some examples of outdoor feature maps.

[0070] Step three: spatial positioning and coordinate conversion.

[0071] In the process of outdoor spatial positioning, the MR device first activates its spatial scanning function to capture and analyze key features in the real environment. After identifying the feature map, the system performs the following steps to achieve spatial positioning.

[0072] First, the system obtains the Cartesian coordinates of the feature map in the MR application and its geographic coordinates in the real geographic scene, and calculates the coordinates of the three-dimensional GIS data in the ENU coordinates with the feature map as the origin. The specific calculation method is similar to step 2 in the MR indoor terminal, except that the reference point P 0 The reference point P in the coordinate transformation of the MR outdoor terminal 0 The choice is the WGS84 geographic coordinates of the feature map in the real geographic scene. By substituting the geographic coordinates (longitude, latitude and altitude) of the real geographic scene of the feature map into Formula 1, Formula 2 and Formula 3 in step 2, the WGS84 coordinates of the three-dimensional GIS data in the ENU coordinates with the feature map as the origin can be calculated.

[0073] Step 4: Model integration and position adjustment.

[0074] The model integration of MR outdoor terminals is similar to that of MR indoor terminals. By constructing an Anchor object (anchor point object) as the fixed reference point 0 of the three-dimensional geographic scene, all three-dimensional GIS data are used as child objects of the Anchor object, and the model integration is achieved by setting the ENU coordinates of the three-dimensional GIS data calculated in step 3 as the local coordinates of the Anchor object. After the model integration is completed, the position of the Anchor object is set to the position of the feature map in the MR application to achieve overall position adjustment.

[0075] Step 5: Direction calibration.

[0076] After adjusting the position of the Anchor object, its direction needs to be calibrated. Since a single feature map is not enough to accurately calibrate the direction, this study uses the position data of two feature maps to improve the calibration accuracy. 0 , the position of another feature map is used as a reference for direction calibration. Assume that the reference point P is set 0 The coordinates of the feature map in MR applications are (x 0 ,y 0 ), the corresponding longitude and latitude are (lon 0 ,lat 0 ), and the coordinates of the other feature map in MR applications are (x 1 ,y 1 ), the corresponding longitude and latitude are (lon 1 ,lat 1 ), the radius of the earth is R.

[0077] The formula for calculating the azimuth between two points is:

[0078]

[0079] like Fig.10 As shown in formula 5, d N Represents the distance to the north (in the latitude direction), d E Represents the distance in the east direction (longitude direction), and the azimuth θ is calculated by arctan2 and adjusted to the range of 0 to 2π by mod. This angle indicates the distance from p 0 The angle of the direction to another feature relative to true north.

[0080] After that, adjust the Z-axis rotation angle of the Anchor object according to the azimuth angle θ:

[0081]

[0082] like Fig.11 As shown, R z Represents the reference point p in the ENU coordinate system 0 The azimuth angle from the feature map to another feature map, RAz The rotation that needs to be adjusted about the Z axis of the Anchor object.

[0083] In summary, the positioning process based on augmented reality markers is as follows Fig.12 shown.

[0084] (3) Accurate indoor and outdoor geographic location matching based on MR

[0085] After using MR devices to display three-dimensional geographic scene data and auxiliary model data indoors and outdoors, when working together, it is necessary to accurately match the position of the auxiliary model in the MR indoor terminal in the three-dimensional geographic scene with the position of the auxiliary model in the MR outdoor terminal in the real geographic scene to support collaborative work using MR devices in indoor and outdoor environments. For example, when the auxiliary model is moved or marked in the three-dimensional geographic scene in the MR indoor terminal, the auxiliary model of the outdoor terminal can synchronously reflect the operation and display it at an accurate position in the actual geographic scene, realizing efficient collaboration across indoor and outdoor environments. The key technical method involved is reverse coordinate conversion.

[0086] Although both the MR indoor terminal and the MR outdoor terminal use the ENU (East-North-Up) coordinate system to set the position of the 3D GIS data in the local coordinate system of the Anchor object, the origin (reference point P) corresponding to their Anchor objects 0 ) is different from the scale. In collaborative work, the geographic coordinates need to be converted and matched accordingly to ensure that the location information of the auxiliary model in the 3D geographic scene in the MR indoor terminal can be accurately mapped to the actual geographic location of the outdoor terminal. This allows users to use the MR indoor terminal to perform operations on 3D GIS data indoors, which can be reflected synchronously in the MR outdoor terminal, achieving efficient collaboration across indoor and outdoor environments.

[0087] Reference point P of MR indoor terminal 0 The center of the 3D geographic model is set by the user, and the scale is a scaled ratio. The reference point P of the MR outdoor terminal 0 This is the actual scene position of the first reference image scanned during positioning, and the scale bar is the actual scale.

[0088] a. MR indoor terminal to MR outdoor terminal

[0089] The coordinates of the auxiliary model in the MR indoor terminal to the coordinates of the auxiliary model in the MR outdoor terminal are obtained by converting the ENU coordinates of the indoor terminal to the globally used WGS84 coordinate system, and then mapping it to the ENU coordinate system of the outdoor terminal through corresponding mathematical conversion.

[0090] Specifically, first convert the ENU coordinates of the MR indoor terminal back to WGS84 coordinates. This process involves converting the reference point coordinates (origin) of the ENU coordinate system to the ECEF (Earth-Centered, Earth-Fixed) coordinate system at the center of the earth. The specific steps are as follows:

[0091]

[0092] Among them, (e,n,u) is the ENU coordinate to be converted, scale is the scaling factor, (lon 0 ,lat 0 ,alt 0 ) is the corresponding position of the origin of the ENU coordinate system of the indoor terminal in the WGS84 coordinate system, and X, Y, and Z are the converted ECEF coordinates.

[0093] Then, convert the ECEF coordinates to WGS84 coordinates:

[0094]

[0095] In the above formula, a and b represent the major and minor axes of the earth ellipsoid, respectively, e is the eccentricity of the earth ellipsoid, N is the radius of curvature, and p is the modulus of the plane coordinate.

[0096] Convert the WGS84 coordinates to the ENU coordinate system of the outdoor terminal. This step has been given in step 2 of the construction of the MR outdoor terminal. Through the conversion, the geographic coordinates of the auxiliary model in the 3D geographic scene in the MR indoor terminal can be accurately matched with the geographic location of the real geographic scene in the MR outdoor terminal, ensuring that the geographic data of the two are consistent in different environments.

[0097] b. MR outdoor terminal to MR indoor terminal

[0098] The coordinates of the auxiliary model in the MR outdoor terminal are converted to the coordinates of the auxiliary model in the MR indoor terminal by converting the ENU coordinates of the indoor terminal into the globally used WGS84 coordinate system, and then mapping it to the ENU coordinate system of the indoor terminal through corresponding mathematical conversion.

[0099] Conversion from ENU coordinate system to WGS84 coordinate system:

[0100] First, the ENU coordinates of the outdoor terminal need to be converted to WGS84 coordinates. This process involves converting the reference point coordinates (origin) of the ENU coordinate system to the ECEF (Earth-Centered, Earth-Fixed) coordinate system at the center of the Earth. The specific steps are as follows:

[0101]

[0102] Among them, (E, N, U) is the ENU coordinate to be converted, (lon 0 ,lat 0 ,alt 0 ) is the corresponding position of the origin of the ENU coordinate system of the indoor terminal in the WGS84 coordinate system, and X, Y, and Z are the converted ECEF coordinates.

[0103] Then, convert the ECEF coordinates to WGS84 coordinates:

[0104]

[0105]

[0106] In the above formula, a and b represent the major and minor axes of the earth ellipsoid, respectively, e is the eccentricity of the earth ellipsoid, N is the radius of curvature, and p is the modulus of the plane coordinate.

[0107] Finally, the WGS84 coordinates are converted to the ENU coordinate system of the indoor terminal. This step has been given in step 2 of the construction of the MR indoor terminal. Through the conversion, the geographic location of the real geographic scene in the MR outdoor terminal can be accurately matched with the geographic coordinates of the auxiliary model in the three-dimensional geographic scene in the MR indoor terminal, ensuring that the geographic data of the two are consistent in different environments.

[0108] The above description is only a preferred embodiment of the present invention, and the protection scope of the present invention is not limited to the above embodiment. Any equivalent modifications or changes made by ordinary technicians in this field based on the contents disclosed by the present invention should be included in the protection scope recorded in the claims.

Claims

1. A method for real-time indoor and outdoor GIS collaboration and location matching based on MR equipment, characterized by: The method comprises the following steps: Step 1: The MR indoor terminal loads the 3D geographic scene data and auxiliary model data; In the MR indoor terminal, the established 3D geographic scene data and auxiliary model data are converted into formats, coordinates, integrated, and scaled to enable correct display and GIS operations in the mixed reality MR environment. Step 2: Display the real geographical scene in the MR outdoor terminal, and correctly overlay the auxiliary model in the MR indoor terminal with the real geographical scene in equal proportion and display them in the MR outdoor terminal, so as to help the user perform tasks in the outdoor environment; In the MR outdoor terminal, the construction of the auxiliary model includes format conversion, feature map acquisition and processing, spatial positioning and coordinate conversion, model integration and position adjustment, and direction calibration; Step 3, after using the MR indoor terminal and the MR outdoor terminal to display the three-dimensional geographic scene data and the auxiliary model data indoors and outdoors, when working collaboratively, through reverse coordinate conversion, the position of the auxiliary model in the MR indoor terminal in the three-dimensional geographic scene is accurately matched with the position of the auxiliary model in the MR outdoor terminal in the real geographic scene to support collaborative work using MR devices in indoor and outdoor environments.

2. The method for indoor and outdoor GIS real-time collaboration and location matching based on MR equipment according to claim 1, characterized in that: The step 1 specifically includes the following steps: Step 1-1, format conversion, converting the 3D GIS data into a format supported by the MR environment, and saving the geographic coordinate information of the data before conversion, including the latitude, longitude and altitude in the WGS84 coordinate system; Step 1-2, coordinate conversion, converting the coordinate data in the WGS84 coordinate system obtained in step 1-1 into coordinates in the earth center ECEF coordinate system, and then converting into coordinates in the east-north-sky ENU coordinate system; Steps 1-3, model integration is to create an anchor object as a fixed reference point for the 3D geographic scene; when the 3D GIS data is imported into the software built into the MR indoor and outdoor terminal, first calculate their respective ENU coordinates through the above coordinate conversion steps; convert the WGS84 global coordinate system of each entity into local ENU coordinates with the Anchor object as the origin, and map these coordinates (E, N, U) to the local coordinate system of the Anchor object; set the position of the virtual geographic entity by adjusting the local coordinates of the Anchor object, ensuring that the position, scale and orientation of the 3D GIS data in the Cartesian coordinate system in the MR environment are the same as those in the geographic coordinate system; Step 1-4, model scaling, for the construction of the auxiliary model of the MR indoor terminal, select the geometric center of the model as the position of the reference point P0, and perform the scaling operation based on the center of the model. Keep the geometric symmetry and spatial relative position of the model unchanged during the scaling operation.

3. The method for indoor and outdoor GIS real-time collaboration and location matching based on MR equipment according to claim 2, characterized in that: The coordinate transformation of step 1-2 includes the following steps: Step 1-2-1, select the reference point or the base point: before the conversion, first select a reference point P0, whose coordinates are expressed in longitude, latitude and altitude in the WGS84 coordinate system, recorded as (λ0, φ0, h0); in the MR indoor terminal, the reference point P0 is the center position of the three-dimensional geographic scene to keep the geometric symmetry and spatial relative position of the model unchanged; this reference point will be used as the origin of the ENU coordinate system, and the relative coordinates of all positions will be calculated based on this; Step 1-2-2, WGS84 to ECEF coordinate conversion: First, you need to convert the longitude, latitude and elevation of WGS84 to the earth-centered ECEF coordinate system. The origin of the ECEF coordinate system is the center of the earth, the plane passing through the prime meridian and the equator is the X-axis; the Y-axis is perpendicular to the X-axis and passes through the equator; the Z-axis is along the rotation axis of the earth; The conversion formula of ECEF coordinate system is: In the above formula, a and b represent the major and minor axes of the earth ellipsoid, respectively, and e 2 is the eccentricity of the earth ellipsoid, N is the radius of curvature; (λ, φ, h) is the geographic coordinates to be converted of the three-dimensional GIS data, (X, Y, Z) is the ECEF coordinates after conversion of the three-dimensional GIS data; Step 1-2-3, calculate the ECEF coordinates of the reference point: using the longitude, latitude and elevation (λ0, φ0, h0) of the reference point P0, convert the WGS84 coordinates of the reference point to ECEF coordinates according to the above formula, recorded as (X0, Y0, Z0); Step 1-2-4, convert WGS84 coordinates to ECEF coordinates: Similarly, convert the longitude, latitude and elevation (λ, φ, h) of the target point to ECEF coordinates, recorded as (X, Y, Z); Step 1-2-5, conversion from ECEF to ENU coordinate system: After obtaining the ECEF coordinates of the reference point and the target point, perform conversion from the ECEF coordinate system to the east-north-sky ENU coordinate system; the coordinates (E, N, U) of the target point in the ENU coordinate system are calculated using the following formula: Where λ0 and φ0 are the longitude and latitude of the reference point, (X0, Y0, Z0) are the ECEF coordinates of the reference point, and (X, Y, Z) are the ECEF coordinates of the target point. Step 1-2-6, result of coordinate conversion: Through the above steps, the longitude, latitude, and elevation coordinates of the target point are converted into coordinates in the east, north, and up directions based on the ENU coordinate system; the ENU coordinate system is the Cartesian coordinate used in MR applications.

4. The method for indoor and outdoor GIS real-time collaboration and location matching based on MR equipment according to claim 2, characterized in that: In steps 1-3, the Anchor object is an empty parent object, and all 3D GIS data are child objects of the Anchor object. Their positions, rotations, and scales in the MR environment are defined relative to the local coordinates of the Anchor object. The position of the reference point P0 is the origin of the Anchor object, and the coordinates of the 3D GIS data in the ENU coordinate system correspond to the coordinates in the local coordinate system of the Anchor object.

5. The method for indoor and outdoor GIS real-time collaboration and location matching based on MR equipment according to claim 2, characterized in that: In steps 1-4, proportional scaling is achieved by adjusting the scaling parameters in the Transform property of the Anchor object. By adjusting the scaling parameters in the Transform property of the Anchor object, the sizes of all sub-objects are changed to achieve proportional scaling of the model; the scaling formula in position is: (E',N',U')=(E,N,U)×scale<Formula 4> Among them, (E, N, U) represents the original coordinate value, scale is the scaling factor, and (E', N', U') is the scaled coordinate value; through this formula, the position of the model in space is adjusted according to the specified scaling ratio to adapt to different display requirements and environmental constraints.

6. The method for indoor and outdoor GIS real-time collaboration and location matching based on MR equipment according to claim 1, characterized in that: The step 2 specifically includes the following steps: Step 2-1, format conversion, is the same as the steps for building an MR indoor terminal; Step 2-2, feature map collection and processing is to realize the superimposed display of three-dimensional GIS data in outdoor environment based on the positioning method of augmented reality marking, and assist positioning by scanning specific features in outdoor environment; the real-world coordinates of feature points are obtained through preliminary measurement and registered; by identifying these feature points, accurate positioning is performed; wherein, feature points are specific locations in the real world, which have specific and detailed geographic coordinates, and the locations should have landmarks or objects with obvious and unique visual features to facilitate feature extraction; feature maps are image photos of feature points, and specific features are features extracted from feature maps. Feature points are identified by features extracted from feature maps, thereby locating real-world locations; Step 2-3, spatial positioning and coordinate conversion: In the process of outdoor spatial positioning, the MR outdoor terminal first activates its spatial scanning function to capture and analyze key features in the real environment; after identifying the feature map, the system obtains the Cartesian coordinates of the feature map in the MR application and its geographic coordinates in the real geographic scene, and based on the coordinate conversion step in the MR indoor terminal, selects the WGS84 geographic coordinates of the feature map in the real geographic scene as the reference point P0 for calculation, and obtains the coordinates of the three-dimensional GIS data in the ENU coordinates with the feature map as the origin; Step 2-4, in model integration and position adjustment, an Anchor object is constructed as a fixed reference point P0 of the 3D geographic scene, all 3D GIS data are taken as child objects of the Anchor object, and the model integration is realized by calculating the ENU coordinates of the 3D GIS data as the local coordinates of the Anchor object; after the model integration is completed, the position of the Anchor object is set to the position of the feature map in the MR application to realize the overall position adjustment; In step 2-5, in direction calibration, the position data of the two feature maps are used to define the reference point P0 with the position of one feature map, and the position of the other feature map is used as a reference for direction calibration, and the azimuth angle between the two is calculated to determine the rotation angle.

7. The method for indoor and outdoor GIS real-time collaboration and location matching based on MR equipment according to claim 6, characterized in that: In step 2-2, for feature points, while collecting their feature maps, it is necessary to measure and record the detailed location information of each location, including longitude, latitude and altitude. Then, the ORB algorithm is used to detect the key points in the feature map and generate corresponding feature descriptors for each feature map. These descriptors will be registered as key features.

8. The method for indoor and outdoor GIS real-time collaboration and location matching based on MR equipment according to claim 6, characterized in that: In step 2-5, assume that the coordinates of the feature map of the reference point P0 in the MR application are (x0, y0), and the corresponding longitude and latitude are (lon0, lat0), and the coordinates of the other feature map in the MR application are (x1, y1), and the corresponding longitude and latitude are lon1, lat1), and the radius of the earth is R; The formula for calculating the azimuth between two points is: d N Represents the distance in the north direction, i.e. in the direction of latitude, d E Represents the distance in the east direction, that is, the longitude direction. The azimuth angle θ is calculated by arctan2, and the angle is adjusted to the range of 0 to 2π by mod; this angle indicates the angle from P0 to another feature map relative to due north; After that, adjust the Z-axis rotation angle of the Anchor object according to the azimuth angle θ: R z Represents the azimuth from the feature map of the reference point P0 to another feature map in the ENU coordinate system, R Az The rotation that needs to be adjusted about the Z axis of the Anchor object.

9. The method for indoor and outdoor GIS real-time collaboration and location matching based on MR equipment according to claim 1, characterized in that: In the reverse coordinate conversion of step 3, for the conversion of the coordinates of the auxiliary model in the MR indoor terminal to the coordinates of the auxiliary model in the MR outdoor terminal, the ENU coordinates of the MR indoor terminal are converted to the globally used WGS84 coordinate system, and then mapped to the ENU coordinate system of the MR outdoor terminal through corresponding mathematical conversion; Specifically, first convert the ENU coordinates of the MR indoor terminal back to WGS84 coordinates; this process involves converting the reference point coordinates of the ENU coordinate system to the earth center ECEF coordinate system. The specific steps are as follows: Among them, (e, n, u) is the ENU coordinate to be converted, sacle is the scaling factor, (lon0, lat0, alt0) is the corresponding position of the origin of the ENU coordinate system of the MR indoor terminal in the WGS84 coordinate system, and X, Y, Z are the converted ECEF coordinates; Then, convert the ECEF coordinates to WGS84 coordinates: In the above formula, a and b represent the major and minor axes of the earth ellipsoid, respectively, e is the eccentricity of the earth ellipsoid, N is the radius of curvature, and p is the modulus of the plane coordinates; Based on the coordinate conversion steps of the MR outdoor terminal, the WGS84 coordinates are converted into the ENU coordinate system of the MR outdoor terminal; Through conversion, the geographic coordinates of the auxiliary model in the MR indoor terminal in the three-dimensional geographic scene are matched with the geographic location of the real geographic scene in the MR outdoor terminal to ensure that the geographic data of the two in different environments are consistent.

10. The method for indoor and outdoor GIS real-time collaboration and location matching based on MR equipment according to claim 1, characterized in that: In the reverse coordinate conversion of step 3, for the conversion of the coordinates of the auxiliary model in the MR outdoor terminal to the coordinates of the auxiliary model in the MR indoor terminal, the ENU coordinates of the MR indoor terminal are converted to the WGS84 coordinate system, and then mapped to the ENU coordinate system of the MR indoor terminal through corresponding mathematical conversion; Specifically, first, the ENU coordinates of the MR outdoor terminal need to be converted to WGS84 coordinates. This process involves converting the reference point coordinates of the ENU coordinate system to the ECEF coordinate system at the center of the earth. The specific steps are as follows: Among them, (E, N, U) is the ENU coordinate to be converted, (lon0, lat0, alt0) is the corresponding position of the origin of the ENU coordinate system of the MR indoor terminal in the WGS84 coordinate system, and X, Y, Z are the ECEF coordinates after conversion; Then, convert the ECEF coordinates to WGS84 coordinates: In the above formula, a and b represent the major and minor axes of the earth ellipsoid, respectively, e is the eccentricity of the earth ellipsoid, N is the radius of curvature, and p is the modulus of the plane coordinates; Finally, based on the coordinate conversion step of the MR indoor terminal, the WGS84 coordinates are converted into the ENU coordinate system of the indoor terminal. Through the conversion, the geographic location of the real geographic scene in the MR outdoor terminal is matched with the geographic coordinates of the auxiliary model in the three-dimensional geographic scene in the MR indoor terminal to ensure that the geographic data of the two in different environments are consistent.

Images