A method for indoor and outdoor GIS real-time collaboration and position matching based on MR equipment

Abstract

The present application relates to the technical field of mixed reality (MR) and geographic information system (GIS), and proposes a method for indoor and outdoor GIS real-time collaboration and position matching based on MR equipment, which integrates real geographic space information and virtual geographic space information, combines spatial positioning and object recognition technology, and uses a geographic coordinate conversion method to support indoor and outdoor GIS collaboration based on MR equipment. The method ensures that three-dimensional GIS data can be accurately displayed in indoor and outdoor MR equipment, and makes the geographic information in indoor and outdoor MR environments consistent, thereby realizing accurate matching of indoor and outdoor geographic positions. The present application supports the collaborative work of GIS data in indoor and outdoor MR environments, and improves the immersive experience and interactivity of users.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of mixed reality (MR) and geographic information system (GIS) technologies, and particularly relates to a method for indoor and outdoor GIS real-time collaboration and position matching based on an MR device. BACKGROUND

[0002] Mixed reality (MR) technology is a new interactive and visual technology formed by the fusion of virtual reality (VR) and augmented reality (AR). It inherits the human-computer interaction of virtual reality and the environment fusion method of augmented reality, and enhances the perception of the environment space. Through deep perception and spatial mapping of the environment around the user, virtual information and real-world environment are combined to achieve highly interactive and immersive user experience.

[0003] MR technology is revolutionizing multiple industries by seamlessly combining digital content with the real world. In the field of geographic information system (GIS) data visualization, MR technology can break through the limitations of two-dimensional screens and superimpose GIS data in three-dimensional form onto the real world. Users can directly interact with geospatial data in the real environment, deepening their understanding and perception of geographic information. Existing GIS collaborative design based on MR devices includes indoor sand table collaborative work systems and outdoor virtual battlefield training systems. However, in terms of real-time collaboration in indoor and outdoor scenarios, existing technologies are difficult to provide a truly immersive and highly interactive user experience.

[0004] In the display of MR to three-dimensional GIS data, the indoor environment is generated by simultaneously reducing the scale of the three-dimensional geographic scene model and the auxiliary model, to generate a miniature model, to provide a perspective for users to observe and operate three-dimensional macroscopic geographic information in the indoor environment through the MR device. The above auxiliary model refers to the virtual or real object model that will be used to support the completion of a collaboration when working indoors and outdoors. It is not necessarily a necessary component of the established three-dimensional geographic scene. The real geographic scene is displayed in the MR device of the outdoor environment collaborator, which can correctly overlay the auxiliary model in the MR indoor terminal with the real geographic scene in the MR outdoor terminal. The technical difficulty lies in that the MR device in the indoor working environment displays a scaled three-dimensional geographic scene, but it has a certain geographic coordinate. The MR device in the outdoor working environment displays the real geographic world. Therefore, how to accurately display the auxiliary model in the indoor working environment MR device in the outdoor environment MR device during collaborative work needs to accurately match the auxiliary model in the indoor MR with the real geographic environment displayed in the outdoor MR, to ensure the accurate correspondence between the auxiliary model and the real location in the MR outdoor terminal, thereby helping users to successfully complete the collaboration in the outdoor environment.

[0005] However, the development environment of MR application is different from GIS. The MR device usually uses a Cartesian coordinate system with (0, 0, 0) as the origin, and the origin is usually determined by the user's location when starting the MR application. The GIS data usually uses a geographic coordinate system such as WGS84, which leads to the fact that the auxiliary model correctly displayed in the indoor environment MR cannot be correctly loaded in the corresponding position in the outdoor environment MR, because the two sets of coordinate systems cannot be directly adapted and need to be converted. How to convert the geographic coordinates of GIS data into a coordinate system suitable for the MR environment, and ensure that the auxiliary model in the MR indoor terminal is correctly displayed, and the auxiliary model in the corresponding MR outdoor terminal and the real geographic scene can be accurately matched in position, to support the collaborative work of indoor and outdoor scenes, is a key problem to be solved. SUMMARY

[0006] The application is an innovative application related to MR indoor terminal three-dimensional GIS and MR outdoor terminal three-dimensional GIS and their collaborative work, and specifically proposes an indoor and outdoor GIS real-time collaboration and position matching method based on MR equipment, aiming to solve the problem of virtual body position matching and mixed superimposed display in outdoor real environment in the indoor and outdoor collaboration environment based on MR equipment. By integrating real geographic space information and virtual geographic space information, combining spatial positioning and object recognition technology, and using geographic coordinate conversion method, GIS indoor and outdoor collaboration based on MR equipment is supported. Ensure that three-dimensional GIS data can be accurately displayed in indoor and outdoor MR equipment, so that the geographic information in the indoor and outdoor MR environment has consistency, thereby realizing accurate matching of indoor and outdoor geographic positions. The method supports the collaborative work of GIS data in the indoor and outdoor MR environment, and improves the immersive experience and interactivity of users.

[0007] An indoor and outdoor GIS real-time collaboration and position matching method based on MR equipment mainly includes the design of MR indoor terminal, the design of MR outdoor terminal, and accurate geographic position matching based on MR indoor and outdoor terminals, wherein:

[0008] (1) For MR indoor terminal:

[0009] In the application, the MR indoor terminal loads the three-dimensional geographic scene data and auxiliary model data of the established research area. However, since the data format supported by the current MR equipment for loading is incompatible with the traditional GIS data format, it is necessary to perform format conversion and coordinate conversion, model integration on the three-dimensional geographic scene data and auxiliary model data that have been established, so as to correctly display and perform GIS operation in the MR environment. The technical steps involved are: data format conversion, coordinate conversion, model integration, model scaling.

[0010] (2) For MR outdoor terminal:

[0011] The MR equipment of the outdoor environment collaborator displays the real geographic scene, can correctly superimpose the auxiliary model in the MR indoor terminal and the real geographic scene in the MR outdoor terminal in equal proportion, and helps the user to assist in performing tasks in the outdoor environment. The key technical steps involved are: format conversion, feature map acquisition and processing, spatial positioning and coordinate conversion, model integration and position adjustment, direction calibration.

[0012] (3) Accurate matching of indoor and outdoor geographic positions based on MR:

[0013] When using the MR device to perform the indoor and outdoor collaborative work, the position of the auxiliary model in the MR indoor terminal in the three-dimensional geographic scene needs to be accurately matched with the position of the auxiliary model in the MR outdoor terminal in the real geographic scene, so as to support the collaborative work in the indoor and outdoor environments using the MR device. After the auxiliary model is moved or labeled in the three-dimensional geographic scene by using the MR terminal indoors, the auxiliary model of the MR outdoor terminal can also synchronously reflect the operation and be displayed in the accurate position in the real geographic scene, so as to realize the efficient cooperation across the indoor and outdoor environments.

[0014] The present application has the following beneficial effects:

[0015] (1) The present application significantly improves the consistency of 3D GIS data in indoor and outdoor scenes by realizing the accurate geographic position matching of GIS data in the MR environment. Through the coordinate conversion method, the present application ensures the accurate display of the model position of GIS data in indoor and outdoor environments, so that the user can intuitively operate and analyze geographic information in the micro model and the real geographic scene. Especially, the real-time collaborative method of indoor and outdoor geographic positions can be seamlessly and accurately matched, and the present application realizes efficient MR device cooperation across indoor and outdoor scenes.

[0016] (2) The editing and adjustment of the model by the user in the micro model of the indoor terminal can be synchronously reflected in the outdoor terminal real scene superposition model and accurately displayed in the actual environment. The present application extends the real-time collaborative work based on MR to the cooperation between indoor scenes and outdoor large geographic scenes, not only improves the collaborative work efficiency and experience of the user in the indoor and outdoor environments based on the MR device, but also expands the application scenarios of MR, so that the application scenarios of MR are more extensive and reliable. BRIEF DESCRIPTION OF DRAWINGS

[0017] Figure 1 It is a display effect schematic diagram of the MR indoor terminal in the embodiment of the present application.

[0018] Figure 2 It is a space positioning and model superposition effect schematic diagram of the MR outdoor terminal in the embodiment of the present application.

[0019] Figure 3 It is a schematic diagram of accurate position matching of indoor and outdoor geographic positions based on MR in the embodiment of the present application.

[0020] Figure 4 It is a schematic diagram of the earth coordinate system conversion parameter in the embodiment of the present application.

[0021] Figure 5 It is a schematic diagram of the ENU coordinate system in the MR application in the embodiment of the present application.

[0022] Figure 6A diagram of multi-source GIS data model integration in an embodiment of the present application.

[0023] Figure 7 A diagram of the effect of different reference point locations on model scaling in an embodiment of the present application.

[0024] Figure 8 A diagram of model scaling in an embodiment of the present application.

[0025] Figure 9 A partial outdoor feature map in an embodiment of the present application.

[0026] Figure 10 A diagram of feature map calculation in real world coordinates in an embodiment of the present application.

[0027] Figure 11 A diagram of feature map calculation in northeast celestial coordinates in an embodiment of the present application.

[0028] Figure 12 A flowchart of spatial positioning based on augmented reality markers in an embodiment of the present application. DETAILED DESCRIPTION

[0029] The technical solutions of the present application are further described below in conjunction with the accompanying drawings.

[0030] Figure 1 A detailed indoor micro model is shown for the display effect of the MR indoor terminal. The model contains multiple buildings and the pipeline system connected thereto. Through accurate geographic position matching and scaling, these complex three-dimensional entities are clearly displayed in the MR environment. The user can intuitively interact with these models in the indoor environment through the MR device, observe the spatial relationship between the buildings and the pipelines, and perform related operations and analysis.

[0031] Figure 2The spatial positioning and model superimposition effects of the MR outdoor terminal are demonstrated. The figure includes two parts: (a) and (b) respectively show two outdoor landmarks for spatial positioning, which help the MR device determine its position in the actual environment through their explicit latitude, longitude and height information. The precise positioning of landmark 1 and landmark 2 provides the basis for subsequent virtual model rendering. Figure (c) demonstrates the superimposition effect of the pre-created pipeline model in the real world in an outdoor environment. In actual operation, the MR device first captures the landmark images in the environment through the camera and uses the pre-recorded landmark latitude, longitude and height data for position recognition and matching. Then, the system aligns the coordinates of the virtual pipeline model with the geographic coordinates of the real world through spatial positioning algorithms, ensuring that the scale, position and direction of the model are highly consistent with the actual environment. Finally, the virtual pipeline model is accurately mapped and rendered in the real road scene, enabling users to intuitively view the position and layout of the pipeline in the MR environment, achieving precise display of three-dimensional geographic entities in outdoor environments.

[0032] Figure 3 The task execution process and effect of precise matching of indoor and outdoor geographic positions based on MR technology are demonstrated. In the MR indoor terminal interface above, the user first selects the location that needs to be alerted in the three-dimensional map micro model and places a roadblock model. The task is operated in the micro scene of the virtual geographic model, and the user selects the appropriate area for placing the roadblock by viewing the relevant information and publishes this task through the micro model. After the task is published, the system transmits the detailed information of the task, including the specific latitude, longitude and other geographic coordinates, to the outdoor terminal.

[0033] The MR outdoor terminal below demonstrates the execution of the task. The left task panel shows the detailed information of the task, including the geographic coordinates of the task and the type of the task, etc. The right side demonstrates that the virtual roadblock model is accurately superimposed on the real road in the outdoor actual scene. It is worth noting that the position of the roadblock in the outdoor environment is completely consistent with the position placed by the user on the micro model in the indoor terminal, ensuring the precise matching of the virtual model between the two. Through accurate geographic coordinate conversion and spatial matching, the consistency of the virtual model in indoor and outdoor terminals is ensured, realizing seamless cooperation from indoor to outdoor.

[0034] This figure demonstrates the efficiency of the present application in cross-scene cooperation, ensuring that the data edited or operated by the user indoors can be accurately transmitted and mapped to the actual scene outdoors, realizing seamless connection between the real environment and virtual information, significantly improving the work efficiency and interaction experience of the user.

[0035] This invention aims to solve the problems of converting geographic coordinates of GIS data into a coordinate system suitable for MR environments, ensuring the correct display of auxiliary models in indoor MR terminals, and enabling accurate location matching of auxiliary models with real geographic scenes in outdoor MR terminals, thereby supporting collaborative work between indoor and outdoor scenes. Therefore, this invention relates to the construction of indoor and outdoor MR terminals and the problem of accurate matching of geographic location data between these two terminals.

[0036] The MR indoor terminal loads the established 3D geographic scene data and auxiliary model data of the study area. However, since the data format supported by current MR devices is incompatible with traditional GIS data formats, it is necessary to perform format conversion, coordinate transformation, and model integration on the established 3D geographic scene data and auxiliary model data in order to display them correctly in the MR environment and perform GIS operations.

[0037] Step 1: Format conversion.

[0038] Traditional 3D GIS data (such as Shapefile or OSGB format) is converted to a format supported by the MR environment (such as FBX or OBJ). Since the conversion process may result in the loss of geographic coordinates, it is necessary to save the geographic coordinate information of the data beforehand, including latitude and longitude (lat, lon) and altitude (alt) in the WGS84 coordinate system.

[0039] Step 2, coordinate transformation.

[0040] MR applications use the Cartesian coordinate system, with XYZ coordinates representing width, depth, and height, respectively. 3D GIS data typically uses geographic coordinate systems, such as WGS84, where coordinate values ​​are longitude, latitude, and elevation. To convert 3D GIS data to a coordinate system compatible with MR applications, the East-North-Top (ENU) coordinate system is usually used. This system is compatible with the Cartesian coordinate system and can convert geographic coordinates to a local Cartesian coordinate system. To convert 3D GIS data to an ENU coordinate system compatible with MR applications, such as… Figure 4 As shown, the following steps are required:

[0041] a. Selecting a Reference Point (Base Point): Before the transformation, first select a reference point P0, whose coordinates are represented by longitude, latitude, and elevation in the WGS84 system, denoted as (λ0, φ0, h0). In the MR indoor terminal, the reference point P0 is usually the center of the 3D geographic scene to maintain the geometric symmetry of the model and the spatial relative position. This reference point will serve as the origin of the ENU coordinate system, and the relative coordinates of all locations will be calculated based on this.

[0042] b. WGS84 to ECEF coordinate conversion: First, the longitude, latitude, and altitude of WGS84 need to be converted to the ECEF (Earth-Centered Earth-Fixed) coordinate system. The origin of the ECEF coordinate system is the center of the Earth, and the coordinate axes are oriented as follows: X-axis: through the plane of the prime meridian and the equator. Y-axis: perpendicular to the X-axis, through the equator. Z-axis: along the Earth's axis of rotation (North Pole).

[0043] The conversion formula for the ECEF coordinate system is:

[0044]

[0045] In the above formula, a and b represent the long and short semi-axes of the Earth's ellipsoid, respectively, e 2 is the eccentricity of the Earth's ellipsoid, and N is the radius of curvature. (λ, φ, h) are the geographic coordinates of the three-dimensional GIS data to be converted, and (X, Y, Z) are the converted ECEF coordinates of the three-dimensional GIS data.

[0046] c. Calculate the ECEF coordinates of the reference point: Using the longitude, latitude, and altitude of the reference point P0 (λ0, φ0, h0), convert the WGS84 coordinates of the reference point to ECEF coordinates according to the above formula, denoted as (X0, Y0, Z0).

[0047] d. WGS84 to ECEF coordinate conversion (target point): Similarly, convert the longitude, latitude, and altitude of the target point (λ, φ, h) to ECEF coordinates, denoted as (X, Y, Z).

[0048] e. ECEF to ENU coordinate system conversion: After obtaining the ECEF coordinates of the reference point and the target point, the conversion from the ECEF coordinate system to the ENU coordinate system can be performed. The coordinates of the target point in the ENU coordinate system (E, N, U) can be calculated by the following formula:

[0049]

[0050] where λ0 and φ0 are the longitude and latitude of the reference point, respectively, (X0, Y0, Z0) is the ECEF coordinate of the reference point, and (X, Y, Z) is the ECEF coordinate of the target point.

[0051] f. Result of coordinate conversion: Through the above steps, the longitude, latitude, and altitude coordinates of the target point are converted to the east, north, and up directions based on the ENU coordinate system. The ENU coordinate system is used in the MR application.

[0052] Step three, model integration.

[0053] In the development environment of MR applications, to accurately represent and control virtual geographic entities, this invention creates an Anchor object as a fixed reference point for the 3D geographic scene. The Anchor object is an empty parent object, and all 3D GIS data are child objects of the Anchor object. Their position, rotation, and scaling in the MR environment are defined relative to the local coordinates of the Anchor object. Figure 5 As shown, the location of reference point P0 is the origin of the Anchor object. The coordinates of the 3D GIS data in the ENU coordinate system correspond to the coordinates of the Anchor object in the local coordinate system.

[0054] When 3D GIS data is imported into an MR application, the first step is to calculate the individual ENU coordinates of each entity using step two. The WGS84 global coordinate system of each entity is converted to local ENU coordinates with the Anchor object as the origin, and these coordinates (E, N, U) are mapped to the local coordinate system of the Anchor object. By adjusting the local coordinates of the Anchor object, the position of the virtual geographic entities is precisely set, ensuring that the position, scale, and orientation of the 3D GIS data in the Cartesian coordinate system within the MR environment are identical to those in the geographic coordinate system.

[0055] Step 4: Scale down the model.

[0056] Due to the limitations of indoor environments, 3D models are usually built to scale, but scaling is necessary for display in an indoor setting. Special consideration must be given to the selection of the reference point P0 in the ENU coordinate system during coordinate transformation. For example... Figure 7 As shown, the choice of reference point P0 affects the model's position during scaling. When reference point P0 is selected at the geometric center of the model, the scaling operation is performed with the model's center as the reference point, ensuring that all parts of the model scale uniformly relative to the center point during scaling. If reference point P0 is selected outside the model or at a non-central location, the model will tilt or move in the opposite direction during scaling. Therefore, for constructing a miniature model of an MR indoor terminal, choosing the geometric center of the model as the reference point P0 can maintain the geometric symmetry and spatial relative position of the model during scaling.

[0057] Scaling is achieved by adjusting the scaling parameter in the Anchor object's Transform property. By adjusting this parameter, the size of all child objects is changed, thus scaling the model proportionally. This preserves the model's detail and structural integrity while significantly reducing its footprint, allowing for more natural and flexible interaction in limited indoor spaces. The scaling formula for position is:

[0058] (E',N',U') = (E,N,U) x scale <Equation 4>

[0059] where (E,N,U) represents the original coordinate values, scale is the scaling factor, and (E',N',U') are the scaled coordinate values. Through this equation, the position of the model in space can be adjusted according to the specified scaling ratio, thus adapting to different display requirements and environmental constraints.

[0060] (2) MR outdoor terminal

[0061] The MR outdoor terminal displays a real geographical scene, providing a correct and proportional superposition of the auxiliary model in the MR indoor terminal with the real geographical scene in the MR outdoor terminal, helping users to assist them in performing tasks in outdoor environments. The construction and integration process of the auxiliary model of the MR outdoor terminal is basically the same as that of the MR indoor terminal.

[0062] Step one, format conversion. The same as step one of the MR indoor terminal construction, which will not be repeated here.

[0063] Step two, feature map collection and processing.

[0064] The superimposed display of three-dimensional GIS data in outdoor environments needs to be realized through spatial positioning technology to ensure the accurate correspondence of virtual objects and actual geographical positions. The positioning method based on augmented reality markers is realized in this invention. This method assists positioning by scanning specific features in the outdoor environment, such as buildings, landmarks, etc. The real-world coordinates of feature points are obtained through pre-measurement and registered in the system. By recognizing these feature points, the system can accurately position virtual content.

[0065] The collection of feature maps is the premise of positioning. In order to accurately align virtual content with specific positions and directions in the real world, the collection principle of feature maps is to select landmarks or objects with obvious and unique visual features. These landmarks or objects should have the following characteristics:

[0066] a. High recognition: The selected feature map should be easily recognized in the real environment, such as unique buildings, eye-catching signs, or special topographic features.

[0067] b. Stability: The object or place represented by the feature map should have long-term stability to avoid feature failure due to environmental changes.

[0068] c. Wide distribution: Feature maps should be distributed in different locations of the user's activity area to ensure effective spatial positioning in different places.

[0069] While collecting the feature map, detailed location information of each site needs to be measured and recorded, including latitude, longitude and altitude. Then, ORB algorithm is used to detect key points in the feature map and generate corresponding feature descriptors for each feature map. These descriptors will be registered in the system, so that the system can accurately identify and locate the feature map. Figure 9 is a part of the outdoor feature map instance.

[0070] Step three, spatial positioning and coordinate conversion.

[0071] In the process of outdoor space positioning, the MR device first starts its space scanning function to capture and analyze key features in the real environment. After identifying the feature map, the system performs the following steps to achieve spatial positioning.

[0072] First, the system obtains the Cartesian coordinates of the feature map in the MR application and its geographic coordinates in the real geographic scene, and calculates the coordinates of the three-dimensional GIS data in the ENU coordinates with the feature map as the origin. The specific calculation method is similar to step two in the MR indoor terminal, the difference is the selection of reference point P0. In the coordinate conversion of the MR outdoor terminal, the selection of reference point P0 is the WGS84 geographic coordinates of the feature map in the real geographic scene. By bringing the geographic coordinates (longitude, latitude and altitude) of the feature map in the real geographic scene into formula 1, formula 2 and formula 3 in step two, the WGS84 coordinates of the three-dimensional GIS data in the ENU coordinates with the feature map as the origin can be calculated.

[0073] Step four, model integration and position adjustment.

[0074] The model integration of the MR outdoor terminal is similar to that of the MR indoor terminal. By constructing an Anchor object (anchor point object) as a fixed reference point 0 of the three-dimensional geographic scene, all three-dimensional GIS data are set as sub-objects of the Anchor object, and the ENU coordinates of the three-dimensional GIS data calculated in step three are set as the local coordinates of the Anchor object to realize model integration. After model integration, the position of the Anchor object is set to the position of the feature map in the MR application to realize overall position adjustment.

[0075] Step five, direction calibration.

[0076] After adjusting the position of the Anchor object, its direction also needs to be calibrated. Since a single feature map is not enough to accurately calibrate the direction, this study uses the position data of two feature maps to improve the calibration accuracy. Take the position of one feature map as the reference point P0, and the position of the other feature map as the reference for direction calibration. Assume that the coordinates of the feature map setting the reference point P0 in the MR application are (x0, y0), and its corresponding longitude and latitude are (lon0, lat0). The coordinates of the other feature map in the MR application are (x1, y1), and its corresponding longitude and latitude are (lon1, lat1). The radius of the Earth is R.

[0077] Then the azimuth angle formula between the two points is:

[0078]

[0079] As shown in Figure 10 , in formula 5, d N represents the distance in the north direction (latitude direction), d E represents the distance in the east direction (longitude direction), and the azimuth angle θ is calculated by arctan2 and adjusted to the range of 0 to 2π by mod. This angle indicates the angle of the direction from p0 to the other feature map relative to the true north.

[0080] Then, adjust the Z-axis rotation angle of the Anchor object according to the azimuth angle θ:

[0081]

[0082] As shown in Figure 11 , R z represents the azimuth angle of the feature map of the reference point p0 in the ENU coordinate system to the other feature map, and R Az is the rotation that needs to be adjusted for the Z-axis of the Anchor object.

[0083] In summary, the flow of positioning based on augmented reality markers is shown in Figure 12 .

[0084] (3) Accurate matching of indoor and outdoor geographic positions based on MR

[0085] After the three-dimensional geographic scene data and auxiliary model data are displayed in indoor and outdoor using MR devices, the positions of the auxiliary models in the three-dimensional geographic scene in the MR indoor terminal and the positions of the auxiliary models in the real geographic scene in the MR outdoor terminal need to be accurately matched when working together to support the collaborative work in indoor and outdoor environments using MR devices. For example, after the auxiliary models in the three-dimensional geographic scene are moved or labeled in the MR indoor terminal, the auxiliary models in the outdoor terminal can reflect the operation synchronously and are displayed at the accurate positions in the actual geographic scene, realizing efficient collaboration across indoor and outdoor environments. The key technical method involved is reverse coordinate conversion.

[0086] Although both the MR indoor terminal and the MR outdoor terminal use the ENU (East-North-Up) coordinate system to set the positions of the three-dimensional GIS data in the local coordinate system of the Anchor object, because the origins (reference point P0) and scales of the Anchor objects corresponding to them are different, the geographic coordinates need to be converted and matched accordingly in collaborative work to ensure that the position information of the auxiliary models in the three-dimensional geographic scene in the MR indoor terminal can be accurately mapped to the actual geographic positions of the outdoor terminal. So that the operations on the three-dimensional GIS data using the MR indoor terminal in indoor can be reflected synchronously in the MR outdoor terminal, realizing efficient collaboration across indoor and outdoor environments.

[0087] The reference point P0 of the MR indoor terminal is the center of the three-dimensional geographic model, which is set by the user, and the scale is the scale ratio. The reference point P0 of the MR outdoor terminal is the real scene position of the first reference map scanned when positioning, and the scale is the actual scale.

[0088] a. MR indoor terminal to MR outdoor terminal

[0089] The coordinates of the auxiliary models in the MR indoor terminal to the coordinates of the auxiliary models in the MR outdoor terminal are converted from the ENU coordinates of the indoor terminal to the globally used WGS84 coordinate system, and then mapped to the ENU coordinate system of the outdoor terminal through corresponding mathematical conversion.

[0090] Specifically, the ENU coordinates of the MR indoor terminal are first converted back to the WGS84 coordinates. This process involves converting the reference point coordinates (origin) of the ENU coordinate system to the ECEF (Earth-Centered, Earth-Fixed) coordinate system of the center of the earth, and the specific steps are as follows:

[0091]

[0092] Wherein, (e, n, u) is the ENU coordinate to be converted, scale is the scaling factor, (lon0, lat0, alt0) is the corresponding position of the ENU coordinate system origin of the indoor terminal in the WGS84 coordinate system, X, Y, Z are the converted ECEF coordinates.

[0093] Then, convert the ECEF coordinates into WGS84 coordinates:

[0094]

[0095] In the above formula, a and b represent the long semi-axis and short semi-axis of the earth ellipsoid respectively, e is the eccentricity of the earth ellipsoid, N is the radius curvature, and p is the module length of the plane coordinate.

[0096] Convert the WGS84 coordinates into the ENU coordinate system of the outdoor terminal. This step has been given in step two of the construction of the MR outdoor terminal. Through the conversion, the geographic coordinates of the auxiliary model in the MR indoor terminal in the three-dimensional geographic scene can be accurately matched with the geographic position of the real geographic scene in the MR outdoor terminal, ensuring that the geographic data in different environments are consistent.

[0097] b. MR outdoor terminal to MR indoor terminal

[0098] The coordinates of the auxiliary model in the MR outdoor terminal to the coordinates of the auxiliary model in the MR indoor terminal are also converted by converting the ENU coordinates of the indoor terminal into the globally common WGS84 coordinate system, and then mapping them into the ENU coordinate system of the indoor terminal through corresponding mathematical conversion.

[0099] Conversion of ENU coordinate system to WGS84 coordinate system:

[0100] First, the ENU coordinates of the outdoor terminal need to be converted into WGS84 coordinates. This process involves converting the reference point coordinates (origin) of the ENU coordinate system into the ECEF (Earth-Centered, Earth-Fixed) coordinate system of the center of the earth, and the specific steps are as follows:

[0101]

[0102] Wherein, (E, N, U) is the ENU coordinate to be converted, (lon0, lat0, alt0) is the corresponding position of the ENU coordinate system origin of the indoor terminal in the WGS84 coordinate system, X, Y, Z are the converted ECEF coordinates.

[0103] Then, convert the ECEF coordinates into WGS84 coordinates:

[0104]

[0105]

[0106] In the above formula, a and b represent the long semi-axis and the short semi-axis of the earth ellipsoid respectively, e is the eccentricity of the earth ellipsoid, N is the radius curvature, and p is the module length of the plane coordinate.

[0107] Finally, the WGS84 coordinates are converted into the ENU coordinate system of the indoor terminal. This step has been given in the second step of the construction of the MR indoor terminal. Through the conversion, the geographical position of the real geographical scene in the MR outdoor terminal can be accurately matched with the geographical coordinates of the auxiliary model in the three-dimensional geographical scene in the MR indoor terminal, ensuring that the geographical data in different environments are consistent.

[0108] The above only describes the preferred embodiments of the present application, and the protection scope of the present application is not limited to the above embodiments, but any equivalent modifications or changes made by those skilled in the art according to the disclosed content of the present application shall be included in the protection scope recorded in the claims.

Claims

1. A method for real-time indoor and outdoor GIS collaboration and location matching based on MR devices, characterized in that: The method includes the following steps: Step 1: The MR indoor terminal loads 3D geographic scene data and auxiliary model data; In the MR indoor terminal, the established 3D geographic scene data and auxiliary model data are converted in format, converted in coordinate, integrated in model and scaled down in order to be correctly displayed and operated in the mixed reality (MR) environment; Step 1 specifically includes the following steps: Step 1-1, Format Conversion: Convert the 3D GIS data to a format supported by the MR environment, and save the geographic coordinate information of the data before conversion, including latitude, longitude and altitude in the WGS84 coordinate system; Step 1-2, coordinate conversion: convert the coordinate data obtained in step 1-1 in the WGS84 coordinate system to coordinates in the Earth-centered ECEF coordinate system, and then to coordinates in the East-North-Sky ENU coordinate system. Steps 1-3 involve model integration, specifically creating an anchor object as a fixed reference point for the 3D geographic scene. When the 3D GIS data is imported into the software built into the MR indoor / outdoor terminal, the ENU coordinates of each entity are first calculated using the coordinate transformation steps described above. The WGS84 global coordinate system of each entity is then converted to local ENU coordinates with the anchor object as the origin, and these coordinates (E, N, U) are mapped to the local coordinate system of the anchor object. The position of the virtual geographic entities is set by adjusting the local coordinates of the anchor object, ensuring that the position, scale, and orientation of the 3D GIS data in the Cartesian coordinate system within the MR environment are the same as those in the geographic coordinate system. Steps 1-4: Model scaling. For the construction of the auxiliary model of the MR indoor terminal, the geometric center of the model is selected as the reference point P0. The scaling operation is performed with the center of the model as the base point. The geometric symmetry and spatial relative position of the model remain unchanged during the scaling operation. Step 2: Display the real geographical scene in the MR outdoor terminal. The auxiliary model in the MR indoor terminal is correctly superimposed on the real geographical scene in the MR outdoor terminal in proportion to help users perform tasks in the outdoor environment. In MR outdoor terminals, the construction of auxiliary models includes format conversion, feature map acquisition and processing, spatial positioning and coordinate transformation, model integration and position adjustment, and orientation calibration. Step 3: After displaying the 3D geographic scene data and auxiliary model data indoor and outdoors using MR indoor and MR outdoor terminals, during collaborative work, the position of the auxiliary model in the 3D geographic scene in the MR indoor terminal is accurately matched with the position of the auxiliary model in the real geographic scene in the MR outdoor terminal through reverse coordinate transformation, so as to support collaborative work using MR devices in indoor and outdoor environments.

2. The method for real-time indoor and outdoor GIS collaboration and location matching based on MR devices according to claim 1, characterized in that: The coordinate transformation in step 1-2 includes the following steps: Step 1-2-1, Selecting a Reference Point: Before the transformation, first select a reference point P0, whose coordinates are represented by longitude, latitude, and elevation in the WGS84 coordinate system, denoted as (λ0, φ0, h0). In the MR indoor terminal, the reference point P0 is the center position of the 3D geographic scene to maintain the geometric symmetry of the model and the spatial relative position. This reference point will serve as the origin of the ENU coordinate system, and the relative coordinates of all positions will be calculated based on this. Step 1-2-2, WGS84 to ECEF coordinate transformation: First, the longitude, latitude, and elevation of WGS84 need to be converted to the Earth-centered ECEF coordinate system. The origin of the ECEF coordinate system is the center of the Earth, and the plane passing through the Prime Meridian and the equator is the X-axis; the Y-axis is perpendicular to the X-axis and passes through the equator; the Z-axis is along the Earth's rotation axis. The transformation formula for the ECEF coordinate system is: In the above formula, a and b represent the semi-major axis and semi-minor axis of the Earth's ellipsoid, respectively, and e 2 λ is the eccentricity of the Earth ellipsoid, N is the radius curvature; (λ,φ,h) are the geographic coordinates to be converted from the 3D GIS data, and (X,Y,Z) are the converted ECEF coordinates of the 3D GIS data; Steps 1-2-3: Calculate the ECEF coordinates of the reference point: Using the longitude, latitude, and elevation (λ0, φ0, h0) of the reference point P0, convert the WGS84 coordinates of the reference point to ECEF coordinates according to the formula above, denoted as (X0, Y0, Z0). Steps 1-2-4, converting WGS84 coordinates to ECEF coordinates: Similarly, convert the longitude, latitude, and elevation (λ,φ,h) of the target point to ECEF coordinates, denoted as (X,Y,Z); Step 1-2-5, ECEF to ENU coordinate system transformation: After obtaining the ECEF coordinates of the reference point and the target point, perform the transformation from the ECEF coordinate system to the East-North-Sky ENU coordinate system; the target point's coordinates (E, N, U) in the ENU coordinate system are calculated using the following formula: Where λ0 and φ0 are the longitude and latitude of the reference point, respectively, (X0,Y0,Z0) are the ECEF coordinates of the reference point, and (X,Y,Z) are the ECEF coordinates of the target point; Steps 1-2-6, results of coordinate transformation: Through the above steps, the longitude, latitude, and elevation coordinates of the target point are transformed into coordinates in the east, north, and up directions based on the ENU coordinate system; the ENU coordinate system is the same as the Cartesian coordinate system used in MR applications.

3. The method for real-time indoor and outdoor GIS collaboration and location matching based on MR devices according to claim 1, characterized in that: In steps 1-3, the Anchor object is an empty parent object, and all 3D GIS data are child objects of the Anchor object. Their position, rotation, and scaling in the MR environment are defined relative to the local coordinates of the Anchor object. The position of the reference point P0 is the origin of the Anchor object, and the coordinates of the 3D GIS data in the ENU coordinate system correspond to the coordinates in the local coordinate system of the Anchor object.

4. The method for real-time indoor and outdoor GIS collaboration and location matching based on MR devices according to claim 1, characterized in that: In steps 1-4, scaling is achieved by adjusting the scaling parameter in the Anchor object's Transform property. By adjusting the scaling parameter in the Anchor object's Transform property, the size of all child objects is changed, thus achieving the scaling of the model. The scaling formula with the model's center as the base point is: (E',N',U')=(E,N,U)×scale<Formula 4> Where (E,N,U) represents the original coordinate values, scale is the scaling factor, and (E',N',U') is the scaled coordinate value; through this formula, the position of the model in space is adjusted according to the specified scaling ratio to adapt to different display requirements and environmental constraints.

5. The method for real-time indoor and outdoor GIS collaboration and location matching based on MR devices according to claim 1, characterized in that: Step 2 specifically includes the following steps: Step 2-1, format conversion, is the same as the steps for building an MR indoor terminal; Step 2-2, Feature Map Acquisition and Processing, involves using an augmented reality marker-based positioning method to overlay and display 3D GIS data in an outdoor environment. This is achieved by scanning specific features in the outdoor environment to assist in positioning. The real-world coordinates of the feature points are obtained through prior measurements and registered. Accurate positioning is then achieved by identifying these feature points. Each feature point represents a specific location in the real world with detailed geographic coordinates and should possess a landmark or object with distinct and unique visual characteristics to facilitate feature extraction. The feature map is an image photograph of the feature point, and specific features are those extracted from the feature map. These extracted features are used to identify the feature point, thereby pinpointing its real-world location. Steps 2-3, Spatial Positioning and Coordinate Conversion: During outdoor spatial positioning, the MR outdoor terminal first activates its spatial scanning function to capture and analyze key features in the real environment. After recognizing the feature map, the system obtains the Cartesian coordinates of the feature map in the MR application and its geographic coordinates in the real geographic scene. Based on the coordinate conversion steps in the MR indoor terminal, the WGS84 geographic coordinates of the feature map in the real geographic scene are selected as the reference point P0 for calculation to obtain the coordinates of the three-dimensional GIS data under the ENU coordinates with the feature map as the origin. In steps 2-4, during model integration and position adjustment, an Anchor object is constructed as a fixed reference point P0 for the 3D geographic scene. All 3D GIS data are treated as child objects of the Anchor object. The ENU coordinates of the calculated 3D GIS data are used as the local coordinates of the Anchor object to achieve model integration. After model integration is completed, the position of the Anchor object is set to the position of the feature map in the MR application to achieve overall position adjustment. In steps 2-5, during orientation calibration, the position data of two feature maps are used. One feature map position is used to define a reference point P0, and the position of the other feature map is used as a reference for orientation calibration. The azimuth angle between the two is calculated to determine the rotation angle.

6. The method for real-time indoor and outdoor GIS collaboration and location matching based on MR devices according to claim 5, characterized in that: In step 2-2, for feature points, while collecting their feature maps, it is necessary to measure and record the detailed location information of each location, including latitude, longitude and altitude. Then, the ORB algorithm is used to detect key points in the feature maps and generate corresponding feature descriptors for each feature map. These descriptors will be registered as key features.

7. The method for real-time indoor and outdoor GIS collaboration and location matching based on MR devices according to claim 5, characterized in that: In steps 2-5, let the coordinates of the feature map of reference point P0 in the MR application be (x0, y0), and its corresponding latitude and longitude be (lon0, lat0), while the coordinates of another feature map in the MR application are (x1, y1), and its corresponding latitude and longitude are (lon1, lat1), and the Earth radius is R. The formula for calculating the azimuth between two points is: d N d represents the distance in the northward direction, i.e., in the latitude direction. E Represents the distance in the eastward direction, i.e., the longitude direction. The azimuth angle θ is calculated by arctan2 and adjusted to the range of 0 to 2π using mod. This angle indicates the direction from P0 to another feature map relative to true north. Next, adjust the Z-axis rotation angle of the Anchor object according to the azimuth angle θ: R z R represents the azimuth angle from one feature map to another in the ENU coordinate system, representing the reference point P0. Az The rotation that needs to be adjusted for the Z-axis of the Anchor object.

8. The method for real-time indoor and outdoor GIS collaboration and location matching based on MR devices according to claim 1, characterized in that: In the reverse coordinate transformation of step 3, for the transformation from the coordinates of the auxiliary model in the MR indoor terminal to the coordinates of the auxiliary model in the MR outdoor terminal, the ENU coordinates of the MR indoor terminal are converted to the globally universal WGS84 coordinate system, and then mapped to the ENU coordinate system of the MR outdoor terminal through the corresponding mathematical transformation. Specifically, the first step is to convert the ENU coordinates of the MR indoor terminal back to WGS84 coordinates. This process involves converting the reference point coordinates of the ENU coordinate system to the Earth's central ECEF coordinate system. The specific steps are as follows: Where (e,n,u) are the ENU coordinates to be converted, scale is the scaling factor, (lon0,lat0,alt0) is the corresponding position of the ENU coordinate system origin of the MR indoor terminal in the WGS84 coordinate system, and X, Y, Z are the converted ECEF coordinates. Then, convert the ECEF coordinates to WGS84 coordinates: In the above formula, a and b represent the semi-major axis and semi-minor axis of the Earth ellipsoid, respectively, e is the eccentricity of the Earth ellipsoid, N is the radius curvature, and p is the modulus of the plane coordinate. Based on the coordinate transformation steps of the MR outdoor terminal, the WGS84 coordinates are converted to the ENU coordinate system of the MR outdoor terminal; By converting the geographic coordinates of the auxiliary model in the 3D geographic scene in the MR indoor terminal to match the geographic location of the real geographic scene in the MR outdoor terminal, the geographic data of the two are consistent in different environments.

Images