A method and device for realizing virtual model and real scene superposition based on orientation

By obtaining the geographical coordinates of mobile devices and using ARKit in the Unity3d engine, the placement of virtual models is solved, and the problem of lack of spatial positioning of virtual models and real scenes is solved in the existing technology, the operation convenience and intelligence are improved, and the virtualization is realized The precise placement of the model in real scenes.

CN113781657BActive Publication Date: 2024-11-19北京视据科技有限公司
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110901066.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-06
Publication Date
2024-11-19
Estimated Expiration
2041-08-06

AI Technical Summary

Technical Problem

In the preview of real-life buildings, the virtual model and real-life superposition lack spatial positioning function, resulting in complex operations and low intelligence, and requires sticking marks on the ground and scanning them on foot.

Method used

By obtaining the real-time geographic coordinates of mobile devices, converting the geographic coordinates of target points on the virtual model into virtual reality software, calculating the world coordinates, determining the scaling value and placement direction, and using the Unity3d engine and ARKit to realize the placement of the virtual model without ground stickers The logo can place a virtual model in the real scene.

Benefits of technology

It realizes the superposition of virtual models based on orientation and real scenes, which are convenient to operate and highly intelligent, and has spatial orientation memory functions. It can lay virtual models on the real scene ground, improving the visualization effect of urban construction space.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113781657B_ABST
    Figure CN113781657B_ABST
Patent Text Reader

Abstract

The present application discloses a method and device for realizing the superposition of virtual models and real scenes based on orientation. The method uses ARkit and geographic positioning technology in the Unity3d engine. By using geographic coordinates as a reference for the placement of virtual models, the virtual model combined with the real scene has a spatial orientation memory function. It is not necessary to attach markers to the ground and scan the real scene. The virtual model can be laid on the ground of the real scene according to the geographic coordinates, that is, the virtual model is placed in the real scene. The operation is convenient and intelligent, and the urban construction space combined with the real scene is visualized. A method for converting geographic coordinates to world coordinates of virtual reality software is also provided, so that the position of the virtual model superimposed on the real scene is closer to the preset position, which can reduce the error.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of virtual reality technology, and in particular to a method and device for realizing the superposition of a virtual model and a real scene based on an orientation. Background Art

[0002] Currently, when previewing a real-life building, it is necessary to superimpose the virtual model with the real scene.

[0003] The existing method does not have a spatial positioning function, so it is necessary to post markers on the ground for identification and tracking. For example, when a developer develops a new community and wants to preview the real-life buildings in the community, it is necessary to post a marker in the center of an open space of 50,000 square meters in the community. Then, when you arrive at the planned preview area, you need to walk for 30 minutes to scan the real scene, and then superimpose the virtual model with the scanned real scene to preview the real building, which makes the operation complicated and the degree of intelligence low. Summary of the invention

[0004] The present application provides a method and device for realizing the superposition of a virtual model and a real scene based on an orientation, which is easy to operate and highly intelligent.

[0005] In a first aspect, the present application provides a method for realizing the superposition of a virtual model and a real scene based on an orientation, comprising:

[0006] Acquire the real-time geographic coordinates of the mobile device, and transform the geographic coordinates of two target points on the virtual model into the virtual reality software installed on the mobile device according to the real-time geographic coordinates to obtain the world coordinates of the two target points in the virtual reality software;

[0007] Determine a first distance between the two target points on the virtual model according to the geographic coordinates of the two target points, determine a second distance between the two target points in the virtual reality software according to the world coordinates of the two target points, and determine a scaling value of the virtual model according to the first distance and the second distance;

[0008] The virtual model is scaled according to the scaling value to obtain a scaled virtual model, a placement direction and a placement orientation of the scaled virtual model are determined, and the scaled virtual model is placed in a real scene captured by a main perspective camera in the virtual reality software according to the placement direction and the placement orientation.

[0009] Optionally, the step of converting the geographic coordinates of the two target points on the virtual model into the virtual reality software installed on the mobile device according to the real-time geographic coordinates to obtain the world coordinates of the two target points in the virtual reality software includes:

[0010] Determine the target world coordinates of the mobile device in the virtual reality software according to the real-time geographic coordinates;

[0011] For each target point, the longitude span and latitude span from the point where the mobile device is located on the virtual model to the target point are calculated, a first product of the length of the north direction indicated by the compass in the virtual reality software and the latitude span of the target point is calculated, a second product of the length of the east direction indicated by the compass in the virtual reality software and the longitude span of the target point is calculated, the sum of the target world coordinates, the first product and the second product is calculated, and the sum is used as the world coordinates of the target point in the virtual reality software.

[0012] Optionally, the step of calculating, for each target point, the longitude span from the point on the virtual model of the mobile device to the target point includes:

[0013] For each target point, the longitude span from the point where the mobile device is located on the virtual model to the target point is calculated according to the following formula:

[0014] locross(n)=(lon-lon(n))*Gn'O*π

[0015] Gn'O=GnO*cos(lat(n))

[0016] Wherein, locross(n) is the longitude span from the point where the mobile device is located on the virtual model to the target point, n is the identifier of the target point, lon is the degree of the angle QOG', lon(n) is the degree of the angle QOGn', Q is the intersection of the prime meridian and the equator, O is the center point of the equator, G' is the first intersection point between the first perpendicular line from the point where the mobile device is located on the virtual model to the equatorial plane and the equatorial plane, Gn' is the second intersection point between the second perpendicular line from the target point to the equatorial plane on the virtual model and the equatorial plane, Gn'O is the line between the center point of the equator and the second intersection point, π is pi, GnO is the line between the target point on the virtual model and the center point of the equator, lat(n) is the degree of the angle GnOGn', Gn is the target point on the virtual model, the range of lon and lon(n) are both [-180°, 180°], and the range of lat(n) is [-90°, 90°].

[0017] Optionally, the step of calculating the latitude span from the point where the mobile device is located on the virtual model to the target point includes:

[0018] For each target point, the latitude span from the point where the mobile device is located on the virtual model to the target point is calculated according to the following formula:

[0019] lacross(n)=(lat-lat(n))*GnO*π

[0020] Among them, lacross(n) is the longitude span from the point where the mobile device is located on the virtual model to the target point, n is the identifier of the target point, lat is the degree of angle GOG', G is the point where the mobile device is located on the virtual model, O is the center point of the equator, G' is the first intersection point between the first perpendicular line from the point where the mobile device is located on the virtual model to the equatorial plane and the equatorial plane, lat(n) is the degree of angle GnOGn', Gn is the target point on the virtual model, Gn' is the second intersection point between the second perpendicular line from the target point to the equatorial plane on the virtual model and the equatorial plane, GnO is the line between the target point and the center point of the equator on the virtual model, π is pi, and the range of lat and lat(n) are both [-90°, 90°].

[0021] Optionally, the step of determining a scaling value of the virtual model according to the first distance and the second distance includes:

[0022] A quotient between the first distance and the second distance is calculated as a scaling value of the virtual model.

[0023] Optionally, the step of determining the placement direction and orientation of the scaled virtual model includes:

[0024] When the geographical coordinates of the two target points in the scaled virtual model coincide with the world coordinates of the two target points in the virtual reality software, the direction and orientation of the virtual model are used as the placement direction and placement orientation respectively.

[0025] Optionally, the method is implemented based on Unity3d engine, ARKit and satellite positioning.

[0026] In a second aspect, the present application provides a device for implementing a virtual model and a real scene superposition based on an orientation, comprising:

[0027] An acquisition module, used to acquire the real-time geographic coordinates of the mobile device, and transform the geographic coordinates of two target points on the virtual model into the virtual reality software installed on the mobile device according to the real-time geographic coordinates, so as to obtain the world coordinates of the two target points in the virtual reality software;

[0028] a scaling value determination module, configured to determine a first distance between the two target points on the virtual model according to the geographic coordinates of the two target points, determine a second distance between the two target points in the virtual reality software according to the world coordinates of the two target points, and determine a scaling value of the virtual model according to the first distance and the second distance;

[0029] The superposition module is used to scale the virtual model according to the scaling value to obtain the scaled virtual model, determine the placement direction and placement orientation of the scaled virtual model, and place the scaled virtual model in the real scene captured by the main perspective camera in the virtual reality software according to the placement direction and the placement orientation.

[0030] Optionally, the acquisition module includes:

[0031] A target world coordinate determination submodule, used to determine the target world coordinates of the mobile device in the virtual reality software according to the real-time geographic coordinates;

[0032] The determination submodule is used to calculate, for each target point, the longitude span and latitude span from the point where the mobile device is located on the virtual model to the target point, calculate a first product of the length of the north direction indicated by the compass in the virtual reality software and the latitude span of the target point, calculate a second product of the length of the east direction indicated by the compass in the virtual reality software and the longitude span of the target point, calculate the sum of the target world coordinates, the first product and the second product, and use the sum as the world coordinates of the target point in the virtual reality software.

[0033] Optionally, the determining submodule is specifically used for:

[0034] For each target point, the longitude span from the point where the mobile device is located on the virtual model to the target point is calculated according to the following formula:

[0035] locross(n)=(lon-lon(n))*Gn'O*π

[0036] Gn'O=GnO*cos(lat(n))

[0037] Wherein, locross(n) is the longitude span from the point where the mobile device is located on the virtual model to the target point, n is the identifier of the target point, lon is the degree of the angle QOG', lon(n) is the degree of the angle QOGn', Q is the intersection of the prime meridian and the equator, O is the center point of the equator, G' is the first intersection point between the first perpendicular line from the point where the mobile device is located on the virtual model to the equatorial plane and the equatorial plane, Gn' is the second intersection point between the second perpendicular line from the target point to the equatorial plane on the virtual model and the equatorial plane, Gn'O is the line between the center point of the equator and the second intersection point, π is pi, GnO is the line between the target point on the virtual model and the center point of the equator, lat(n) is the degree of the angle GnOGn', Gn is the target point on the virtual model, the range of lon and lon(n) are both [-180°, 180°], and the range of lat(n) is [-90°, 90°].

[0038] Optionally, the determining submodule is specifically used for:

[0039] For each target point, the latitude span from the point where the mobile device is located on the virtual model to the target point is calculated according to the following formula:

[0040] lacross(n)=(lat-lat(n))*GnO*π

[0041] Among them, lacross(n) is the longitude span from the point where the mobile device is located on the virtual model to the target point, n is the identifier of the target point, lat is the degree of angle GOG', G is the point where the mobile device is located on the virtual model, O is the center point of the equator, G' is the first intersection point between the first perpendicular line from the point where the mobile device is located on the virtual model to the equatorial plane and the equatorial plane, lat(n) is the degree of angle GnOGn', Gn is the target point on the virtual model, Gn' is the second intersection point between the second perpendicular line from the target point to the equatorial plane on the virtual model and the equatorial plane, GnO is the line between the target point and the center point of the equator on the virtual model, π is pi, and the range of lat and lat(n) are both [-90°, 90°].

[0042] Optionally, the scaling value determination module is specifically used to:

[0043] A quotient between the first distance and the second distance is calculated as a scaling value of the virtual model.

[0044] Optionally, the superposition module is specifically used for:

[0045] When the geographical coordinates of the two target points in the scaled virtual model coincide with the world coordinates of the two target points in the virtual reality software, the direction and orientation of the virtual model are used as the placement direction and placement orientation respectively.

[0046] In a third aspect, the present application provides a readable medium comprising execution instructions. When a processor of an electronic device executes the execution instructions, the electronic device executes any method as described in the first aspect.

[0047] In a fourth aspect, the present application provides an electronic device, comprising a processor and a memory storing execution instructions, wherein when the processor executes the execution instructions stored in the memory, the processor executes any method described in the first aspect.

[0048] It can be seen from the above technical scheme that the present application provides a method for realizing the superposition of a virtual model and a real scene based on orientation. In the method, the real-time geographic coordinates of the mobile device are obtained, and the geographic coordinates of two target points on the virtual model are converted into the virtual reality software installed on the mobile device according to the real-time geographic coordinates to obtain the world coordinates of the two target points in the virtual reality software; the first distance between the two target points on the virtual model is determined according to the geographic coordinates of the two target points, and the second distance between the two target points in the virtual reality software is determined according to the world coordinates of the two target points, and the scaling value of the virtual model is determined according to the first distance and the second distance; the virtual model is scaled according to the scaling value to obtain the scaled virtual model, the placement direction and placement orientation of the scaled virtual model are determined, and the scaled virtual model is placed in the real scene captured by the main visual camera in the virtual reality software according to the placement direction and placement orientation. In the technical solution of the present application, ARkit and geo-positioning technologies are used in the Unity3d engine. By taking the geographic coordinates as a reference for the placement of the virtual model, the virtual model combined with the real scene has a spatial orientation memory function. There is no need to stick markers on the ground and scan the real scene. The virtual model can be laid on the real scene ground according to the geographic coordinates, that is, the virtual model is placed in the real scene. The operation is convenient and highly intelligent, and the urban construction space combined with the real scene can be visualized.

[0049] The further effects of the above-mentioned non-conventional preferred manner will be described below in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] In order to more clearly illustrate the embodiments of the present application or the existing technical solutions, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments recorded in the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0051] Figure 1 A schematic diagram of a flow chart of a method for implementing a virtual model based on orientation and a real scene superposition method in an embodiment of the present application;

[0052] Figure 2 Another flowchart of a method for implementing a virtual model based on an orientation and superimposing a real scene in an embodiment of the present application is shown;

[0053] Figure 3 A schematic diagram of the geometrization of the Earth;

[0054] Figure 4 This is a schematic diagram showing the variation of ec with latitude;

[0055] Figure 5A schematic diagram of the orientation relationship between the geographical coordinates converted into the world coordinates in the virtual reality software and the camera coordinates assembled by the virtual reality software;

[0056] Figure 6 A schematic diagram of coinciding the geographical coordinates of two target points in the scaled virtual model with the world coordinates of the two target points in the virtual reality software;

[0057] Figure 7 A schematic diagram of a structure of a device for implementing a virtual model based on orientation and a real scene superposition in an embodiment of the present application;

[0058] Figure 8 A schematic diagram of the structure of an electronic device. DETAILED DESCRIPTION

[0059] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work are within the scope of protection of the present application.

[0060] In order to solve the problem in the prior art that global context information cannot be effectively utilized and that multi-phase image data cannot be effectively fused due to inconsistencies between the multi-phase image data, resulting in low accuracy of organ segmentation.

[0061] The present application provides a method for realizing the superposition of a virtual model and a real scene based on an orientation. In the method, the real-time geographic coordinates of a mobile device are obtained, and the geographic coordinates of two target points on the virtual model are converted into virtual reality software installed on the mobile device according to the real-time geographic coordinates to obtain the world coordinates of the two target points in the virtual reality software; a first distance between the two target points on the virtual model is determined according to the geographic coordinates of the two target points, and a second distance between the two target points in the virtual reality software is determined according to the world coordinates of the two target points, and a scaling value of the virtual model is determined according to the first distance and the second distance; the virtual model is scaled according to the scaling value to obtain a scaled virtual model, a placement direction and a placement orientation of the scaled virtual model are determined, and the scaled virtual model is placed in the real scene captured by a main visual camera in the virtual reality software according to the placement direction and the placement orientation. In the technical solution of the present application, ARkit and geo-positioning technologies are used in the Unity3d engine. By taking the geographic coordinates as a reference for the placement of the virtual model, the virtual model combined with the real scene has a spatial orientation memory function. There is no need to stick markers on the ground and scan the real scene. The virtual model can be laid on the real scene ground according to the geographic coordinates, that is, the virtual model is placed in the real scene. The operation is convenient and highly intelligent, and the urban construction space combined with the real scene can be visualized.

[0062] Various non-limiting implementations of the present application are described in detail below in conjunction with the accompanying drawings.

[0063] See also Figure 1 , shows a flow chart of a method for implementing a virtual model based on orientation and a real scene superposition method in an embodiment of the present application. In this embodiment, the method is applied to an electronic device, and the method may include the following steps:

[0064] S101: Acquire the real-time geographic coordinates of the mobile device, and convert the geographic coordinates of two target points on the virtual model into the virtual reality software installed on the mobile device according to the real-time geographic coordinates to obtain the world coordinates of the two target points in the virtual reality software.

[0065] In terms of AR (Augmented Reality) overlay, this application selects the ARKit technology that can spatially reconstruct the scene and has a memory function for the relative position in space. In terms of geographic positioning, the GPS protocol is used to assist ARKit in extracting and referencing the absolute position in space, and the compass data of mobile devices such as mobile phones is obtained to control the direction when superimposing virtual objects. In terms of the engine, the Unity3d engine is selected, which has relatively complete stereo geometry operations and 3D information management mechanisms. It is also the core processing carrier for integrating, converting and corresponding to the position data obtained by ARKit and GPS positioning.

[0066] The following is an introduction to the unity3d engine, ARkit and geolocation involved in this application:

[0067] When ARKit is used alone, it is mostly used in spatial interactive games. Although it can be played directly on a mobile phone, the original intention of ARKit is to serve the immersive experience of wearable devices. ARKit's position in the AR world is on par with Hololens in the VR (Virtual Reality) world. HoloLens is a mixed reality head-mounted display developed by Microsoft. ARKit is an AR (Augmented Reality) development platform. Developers can use ARKit to create augmented reality applications.

[0068] ARKit is a first-generation augmented reality SDK that inherits the image-based recognition and tracking function, and also records the motion changes of the physical sensing components of the phone. Specifically, it uses the gyroscope to determine the orientation of the phone, and then uses the accelerometer to determine the displacement of the phone relative to the viewing angle. Therefore, the biggest difference between ARKit and traditional image-based recognition and tracking is that it can continue to track even if the target disappears from the screen. SDK is a software development kit.

[0069] GPS (Global Positioning System) protocol and compass are mostly used in navigation software. GPS protocol is a required protocol in positioning software, because the geographic coordinates obtained by mobile phone positioning in real time and fed back to the positioning software are GPS coordinates. Therefore, no matter what satellite returns the geographic coordinates, they are converted into GPS coordinates for calculating the geographic location relationship in the positioning software.

[0070] The GPS protocol is already the most accurate positioning method currently open to civilian use. Some mobile phone models, such as iPhone 11 and above, can work with Beidou to make the GPS coordinates obtained in the software more accurate, but the conversion method between geographic coordinates and three-dimensional coordinates in the software will not change.

[0071] Unity3d is a comprehensive 3D cross-platform application engine, which is used in virtual roaming or simulation, 3D games, and film and television rendering. Most of the existing 3D independent games are completed through this engine. The interface is relatively comprehensive and the degree of freedom in use is also high. It can not only publish software across platforms but also call functions across technical environments.

[0072] Since the purpose of superimposing virtual models and real scenes in this application is to realize real-scene building preview, motion tracking function is required, so ARkit is used in this application. Specifically, ARkit and geographic positioning technology are used in Unity3d engine to superimpose virtual models and real scenes. Since ARkit is an ios-based sdk, the ios development module is also selected for Unity, in which geographic coordinates are converted into gps coordinates. iOS is a mobile operating system developed by Apple.

[0073] Exemplarily, the implementation environment of this application can be: Window10 operating system + Unity3d 2018 engine + ARKit sdk, and the result operating environment is: ios13+.

[0074] In this application, the purpose of using ARKit is to confirm the plane where the virtual model is placed. The specific placement position is determined by converting geographic coordinates into world coordinates in the virtual reality software, that is, obtaining the real-time geographic coordinates of the mobile device, and converting the geographic coordinates of the two target points on the virtual model into the virtual reality software installed on the mobile device according to the real-time geographic coordinates, and obtaining the world coordinates of the two target points in the virtual reality software. Among them, the virtual model is a 3D model resource artificially made in the virtual reality software and superimposed on the real scene. The two target points are any two points on the virtual model, and the plane where the two target points are located in the virtual reality software is the plane where the virtual model is placed.

[0075] This step involves the transformation of geographic coordinates. As we all know, geographic coordinates are the coordinates that represent the central angle of a spherical cross section on a spherical surface. The earth is a sphere with an irregular radius. The transformation of geographic coordinates is to "tear off" a small part of the surface of an irregular sphere and unfold it flattened into a real scene with a perspective relationship and require that the geographic range corresponds as much as possible.

[0076] Wherein, step S101 may include:

[0077] Determine the target world coordinates of the mobile device in the virtual reality software based on the real-time geographic coordinates;

[0078] For each target point, the longitude span and latitude span from the point where the mobile device is located on the virtual model to the target point are calculated, the first product of the length of the north direction indicated by the compass in the virtual reality software and the latitude span of the target point is calculated, the second product of the length of the east direction indicated by the compass in the virtual reality software and the longitude span of the target point is calculated, the sum of the target world coordinates, the first product and the second product is calculated, and the sum is used as the world coordinates of the target point in the virtual reality software.

[0079] Figure 2Another flowchart of a method for implementing a virtual model based on orientation and superimposing a real scene provided by this application is shown in FIG. Figure 2 , Figure 2 Where G is the point where the mobile device is located on the virtual model, P1 and P2 are two target points on the virtual model M1, the scaled virtual model M2 is the virtual model obtained by enlarging the virtual model M1 by the scaling value, P1' and P2' on M2 correspond to P1 and P2 on M1 respectively, LP is the distance between P1 and P2, LT is the distance between T1 and T2, T1 and T2 are two points on the plane F in the virtual reality software, H1 and H2 are two points in the actual scene, M->MF represents the process of placing the virtual model M1 on the plane F in the virtual reality software and making the two target points P1 and P2 on the virtual model M1 coincide with the two points T1 and T2 on the plane F in the virtual reality software, Gn->Tn represents the process of converting the geographic coordinates in the real scene into the world coordinates in the virtual reality software according to the geographic coordinates of the point G where the mobile device is located on the virtual model, and S is the origin of the central coordinate axis of the virtual model. In this step, the two target points P1 and P2 on the virtual model M1 are corresponded to H1 and H2 in the actual scene.

[0080] It should be noted that Figure 2 This is just a schematic diagram. Figure 2 The two target points P1 and P2 on the virtual model M1 span the east-west longitude only for the convenience of viewing. In fact, the distance between the two points is relatively close, and can be formed as P1 and P2 from a macroscopic perspective of the universe, and the corresponding ones are H1 and H2 on the real ground in the actual scene.

[0081] Specifically, the target world coordinates of the mobile device in the virtual reality software are determined based on the real-time geographic coordinates. Since the world coordinates of the mobile device in the virtual reality software are the world coordinates of the main perspective camera in the virtual reality software, the target world coordinates are the world coordinates of the main perspective camera.

[0082] In order to calculate the world coordinates of each target point in the virtual reality software, it is necessary to calculate the longitude span and latitude span from the point where the mobile device is located on the virtual model to the target point for each target point.

[0083] The above-mentioned calculation of the longitude span from the point where the mobile device is located on the virtual model to the target point may include:

[0084] For each target point, the longitude span from the point where the mobile device is located on the virtual model to the target point is calculated according to the following formula:

[0085] locross(n)=(lon-lon(n))*Gn'O*π

[0086] Gn'O=GnO*cos(lat(n))

[0087] Wherein, locross(n) is the longitude span from the point where the mobile device is located on the virtual model to the target point, n is the identifier of the target point, lon is the degree of the angle QOG', lon(n) is the degree of the angle QOGn', Q is the intersection of the prime meridian and the equator, O is the center point of the equator, G' is the first intersection point between the first perpendicular line from the point where the mobile device is located on the virtual model to the equatorial plane and the equatorial plane, Gn' is the second intersection point between the second perpendicular line from the target point to the equatorial plane on the virtual model and the equatorial plane, Gn'O is the line between the center point of the equator and the second intersection point, π is pi, GnO is the line between the target point on the virtual model and the center point of the equator, lat(n) is the degree of the angle GnOGn', Gn is the target point on the virtual model, the range of lon and lon(n) are both [-180°, 180°], and the range of lat(n) is [-90°, 90°].

[0088] The above-mentioned calculation of the latitude span from the point where the mobile device is located on the virtual model to the target point may include:

[0089] For each target point, the latitude span from the point where the mobile device is located on the virtual model to the target point is calculated according to the following formula:

[0090] lacross(n)=(lat-lat(n))*GnO*π

[0091] Among them, lacross(n) is the longitude span from the point where the mobile device is located on the virtual model to the target point, n is the identifier of the target point, lat is the degree of angle GOG', G is the point where the mobile device is located on the virtual model, O is the center point of the equator, G' is the first intersection point between the first perpendicular line from the point where the mobile device is located on the virtual model to the equatorial plane and the equatorial plane, lat(n) is the degree of angle GnOGn', Gn is the target point on the virtual model, Gn' is the second intersection point between the second perpendicular line from the target point to the equatorial plane on the virtual model and the equatorial plane, GnO is the line between the target point and the center point of the equator on the virtual model, π is pi, and the range of lat and lat(n) are both [-90°, 90°].

[0092] The following is an introduction using G1 as an arbitrary point on the earth's surface as a specific embodiment: Figure 3 For a schematic diagram of the geometrization of the Earth, see Figure 3The sphere in the figure simulates an earth, where Q(0,0) is the intersection of the prime meridian and the equator, the equatorial radius OA = 6378137m, the polar radius OC = 6356725m, and a perpendicular line is drawn from any point G1 (lat1, lon1) on the earth's surface to the equatorial plane QOA, intersecting the plane QOA at point G1'. Lat1 is the degree of the angle G1OG1', and lon1 is the degree of the angle QOG1'.

[0093] If G1 coincides with point C, then lat1 = 90°; if G1 coincides with point D, then lat1 = -90°, where C is the North Pole on the earth and D is the North Pole on the earth. Therefore, for any point G1 on the surface of the earth, the range of lat1 is [-90°, 90°], where south latitude is negative and north latitude is positive. Similarly, the range of lon1 is [-180°, 180°], where west longitude is negative and east longitude is positive.

[0094] Figure 4 This is a schematic diagram showing the variation of ec with latitude, with ec as the ordinate and latitude as the abscissa. Figure 4 , ec represents the length of G1O. When G1 coincides with point P, that is, |lat1|=0°, the length of G1O is equal to the equatorial radius OA, where point P is the intersection of the meridian of G1 and the equator; when G1 coincides with point C or point D, that is, |lat1|=90°, the length of G1O is equal to the polar radius OC. Therefore, in the [-90°, 90°] variation interval of lat1, the value of the monotonic interval of the distance change of G1O is symmetrical with lat1=0°, which can be approximately regarded as a linear change. Therefore, the value of the distance of G1O in the variation interval can be expressed as:

[0095] ec=6356.725+21.412*(90°-|lat1|) / 90, the unit of ec is km.

[0096] Continue to see Figure 3 , G1O=OC+(OA-OC)*(90°-|lat1|) / 90, the projection of G1O on the equatorial plane QOA G1'O=G1O*cos(lat1).

[0097] Similarly, if there is another point G(lat,lon) on the earth, the projection of GO on the equatorial plane QOA is G'O=GO*cos(lat).

[0098] As we all know, the arc length formula is ωr, so the longitude span from G to G1 is locross1=(lon-lon1)*G1'O*π, and the latitude span is lacross1=(lat-lat1)*G1O*π. That is to say, the longitude and latitude span from G to Gn, that is, the longitude span from the point where the mobile device is located on the virtual model to the target point, can be expressed in the following format:

[0099] locross(n)=(lon-lon(n))*Gn'O*π

[0100] lacross(n)=(lat-lat(n))*GnO*π

[0101] Among them, locross(n) and lacross(n) are both vectors.

[0102] Figure 5 This is a schematic diagram of the orientation relationship between the geographic coordinates converted into the world coordinates in the virtual reality software and the camera coordinates assembled by the virtual reality software, as shown in Figure 5 As shown in the figure, T is the ground position corresponding to the mobile device in the virtual reality software, that is, the target world coordinate, N is the north direction indicated by the compass, and E is the east direction indicated by the compass. The lengths of N and E are both 1. Then the ground position Tn corresponding to the target point Gn, that is, the world coordinate of the target point in the virtual reality software, can be expressed as:

[0103] Tn=T+N*lacross(n)+E*locross(n)

[0104] Among them, Tn is the world coordinate of the target point in the virtual reality software, T is the target world coordinate, N*lacross(n) is the first product of the length of the north direction pointed by the compass in the virtual reality software and the latitude span of the target point, and E*locross(n) is the second product of the length of the east direction pointed by the compass in the virtual reality software and the longitude span of the target point.

[0105] S102: Determine a first distance between the two target points on the virtual model according to the geographic coordinates of the two target points, determine a second distance between the two target points in the virtual reality software according to the world coordinates of the two target points, and determine a scaling value of the virtual model according to the first distance and the second distance.

[0106] After obtaining the world coordinates of the two target points in the virtual reality software, the first distance between the two target points on the virtual model can be determined according to the geographic coordinates of the two target points, the second distance between the two target points in the virtual reality software can be determined according to the world coordinates of the two target points, and the scaling value of the virtual model can be determined according to the first distance and the second distance.

[0107] The determining of the scaling value of the virtual model according to the first distance and the second distance may include:

[0108] A quotient between the first distance and the second distance is calculated as a scaling value of the virtual model.

[0109] For example, assuming that the first distance is LP and the second distance is LT, the scaling value Scale is:

[0110] Scale=LP / LT

[0111] S103: Scaling the virtual model according to the scaling value to obtain a scaled virtual model, determining a placement direction and a placement orientation of the scaled virtual model, and placing the scaled virtual model in a real scene captured by a main visual camera in the virtual reality software according to the placement direction and the placement orientation.

[0112] Continue to see Figure 2 After obtaining the scaling value, the virtual model M1 can be scaled according to the scaling value to obtain the scaled virtual model M2. Specifically, the virtual model M1 is magnified by the scaling value times to obtain the scaled virtual model M2, and P1' and P2' on M2 correspond to P1 and P2 on M1 respectively.

[0113] In order to place the scaled virtual model M2 into the real scene, after obtaining the scaled virtual model M2, it is necessary to determine the placement direction and placement orientation of the scaled virtual model, and place the scaled virtual model in the real scene captured by the main visual camera in the virtual reality software according to the placement direction and placement orientation.

[0114] In the present application, the placement direction and placement orientation are obtained by reverse deduction based on the placement result. Specifically, the above-mentioned determination of the placement direction and placement orientation of the scaled virtual model may include:

[0115] When the geographic coordinates of the two target points in the scaled virtual model are overlapped with the world coordinates of the two target points in the virtual reality software, the direction and orientation of the virtual model are used as the placement direction and placement orientation respectively.

[0116] Figure 6 For a schematic diagram of coinciding the geographic coordinates of two target points in the scaled virtual model with the world coordinates of the two target points in the virtual reality software, see Figure 6 In this step, the scaled virtual model M2 is rotated and placed on the plane F in the virtual reality software to obtain the virtual model MF, so that the two target points P1' and P2' on the scaled virtual model M2 coincide with T1 and T2 on the plane F in the virtual reality software, respectively. Figure 6Where f is the z-axis direction of the central coordinate axis, u is the y-axis direction of the central coordinate axis, r is the x-axis direction of the central coordinate axis, pf is the vector from P2 to P1, pu is the vector parallel to the u direction, f' is the placement direction, that is, the direction in which the z-axis of the central coordinate axis of MF is to be aligned, u' is the direction in which the y-axis of the central coordinate axis of MF is to be aligned, r' is the direction in which the x-axis of the central coordinate axis of MF is to be aligned, tf is the vector from T2 to T1, tu is the normal vector perpendicular to plane F, GP->GP' is to place the virtual model The process of moving the empty object GP on M1 to the point T2 on the plane F in the virtual reality software, M->MF is the process of placing the virtual model M1 on the plane F in the virtual reality software, and making the two target points P1 and P2 on the virtual model M1 coincide with the two points T1 and T2 on the plane F in the virtual reality software, S is the origin of the central coordinate axis of the virtual model M1 and the virtual model M2, S' is the origin of the central coordinate axis of MF, GP is the empty object coincident with P2, and GP' is the state of the empty object GP moving to the point T2.

[0117] Since the distance between point P1 and point P2 on the scaled virtual model M2 is equal to the distance between point T1 and point T2 on plane F, it is only necessary to first confirm the orientation of M2 and then infer the position of M2 based on the relative relationship to achieve the fit.

[0118] Because it is placed on plane F, and the placement planes calculated by ARKit are all planes with normals facing the upward direction of the world coordinate, that is, wu, it has been determined that the placement direction u' in the u direction on the model M2 is consistent with wu, so you only need to confirm the placement direction f' of f to determine the placement direction of M2.

[0119] Specifically, f' is determined as follows:

[0120] (1) Create an empty object GP, GP.position = P2,

[0121] GP.rotation=Quaternion.LookRotation(pf,pu), where pf=P2-P1,pu=u. Specifically, GP.position=P2 means that the position of the specified GP coincides with the position of P2. Rotation in the software uses a quaternion Quaternion to express the result of the rotation. The most common method to obtain this result is to use LookRotation to specify the alignment direction. First of all, GP.rotation specifies the direction of the central coordinate axis of the GP. GP.rotation=Quaternion.LookRotation(pf,pu) is to instruct the z-axis direction of the central coordinate axis of the GP to be parallel to pf, and the y-axis direction to be parallel to pu.

[0122] (2) Record the position and orientation of the scaled virtual model M2 relative to the empty object GP,

[0123] inv_u = GP.InverseDirection(u), inv_f = GP.InverseDirection(f), inv_o = GP.InversePoint(O), where Inverse is a matrix operation, which means converting an absolute relationship into a relative relationship. Therefore, InverseDirection means converting an absolute direction into a relative direction, and InversePoint means converting an absolute position into a relative position.

[0124] Because this application is to reversely restore the displacement and orientation changes of the virtual model M2 through the displacement and orientation changes of GP, the relationship between M2 and GP should be recorded.

[0125] inv_u=GP.InverseDirection(u) means converting the y-axis direction of the central coordinate axis of model M2 into the direction relative to the central coordinate axis of GP and recorded as inv_u. The same is true for inv_f. inv_o=GP.InversePoint(O) means converting the origin of the central coordinate axis of model M2 into the position relative to the central coordinate axis of GP and recorded as inv_o.

[0126] (3) The GP state is transformed to GP': pf and tf are aligned in the same direction, pu and tu are aligned in the same direction, and the position of GP coincides with T2, where tf = T2-T1. Therefore, at this time: GP.position = T2, GP.rotation = Quaternion.LookRotation (tf, tu).

[0127] (4) Transformation of M2 to MF state: The position and orientation of M2 relative to GP are transformed into absolute position and orientation through the state transformed by GP, i.e., GP'. Therefore, we have: MF.position = GP'.TransformPoint(inv_o), u' = GP'.TransformDirection(inv_u), f' = GP'.TransformDirection(inv_f), MF.rotation = Quaternion.LookRotation(f',u').

[0128] On the contrary, TransformPoint converts the relative coordinate relationship into the absolute coordinate relationship. What is known here is that the state of the virtual model M2 after the change is recorded as MF, and the state of GP after the change is recorded as GP'. Then the completed process is the transformation from GP to GP'. Now we need to analyze how to transform the M2 state into MF.

[0129] In the previous process, the position of the origin of the central coordinate axis of M2 relative to the central coordinate axis of GP has been recorded and recorded as inv_o, and then GP is transformed to GP' through changes in position and orientation. At this time, to resolve the position of the origin of the central coordinate axis of MF, it is only necessary to transform the position of inv_o in the central coordinate system of GP' into the position in the world coordinate system, that is, MF.position = GP'.TransformPoint(inv_o).

[0130] TransformDirection is to convert the relative direction into the absolute direction. Here, the direction inv_u of the y-axis of the central coordinate axis of the virtual model M2 recorded before relative to the central coordinate axis of GP is converted into the direction of the world coordinate system and recorded as u', that is, u'=GP'.TransformDirection(inv_u). The same applies to TransformDirection(inv_f), MF.rotation=Quaternion.LookRotation(f',u').

[0131] f' and u' respectively represent the directions in which the z-axis and the y-axis in the central coordinate axis of the MF are to be aligned.

[0132] In summary, the present application provides a method for realizing orientation-based superposition of a virtual model and a real scene. In the method, the real-time geographic coordinates of the mobile device are obtained, and the geographic coordinates of two target points on the virtual model are converted into the virtual reality software installed on the mobile device according to the real-time geographic coordinates to obtain the world coordinates of the two target points in the virtual reality software; a first distance between the two target points on the virtual model is determined according to the geographic coordinates of the two target points, and a second distance between the two target points in the virtual reality software is determined according to the world coordinates of the two target points, and a scaling value of the virtual model is determined according to the first distance and the second distance; the virtual model is scaled according to the scaling value to obtain a scaled virtual model, and a placement direction and placement orientation of the scaled virtual model are determined, and the scaled virtual model is placed in the real scene captured by the main visual camera in the virtual reality software according to the placement direction and placement orientation. In the technical solution of the present application, ARkit and geo-positioning technologies are used in the Unity3d engine. By taking the geographic coordinates as a reference for the placement of the virtual model, the virtual model combined with the real scene has a spatial orientation memory function. There is no need to stick markers on the ground and scan the real scene. The virtual model can be laid on the real scene ground according to the geographic coordinates, that is, the virtual model is placed in the real scene. The operation is convenient and highly intelligent, and the urban construction space combined with the real scene can be visualized.

[0133] However, although ARKit has a small-range positioning function, ARKit's memory of the environment and the construction of 3D space from the software are based on images, but the environment, especially the public environment, changes frequently, so the existing technology that relies on ARKit to reset memory is unreliable. In addition, the disadvantages of positioning technology are accuracy problems and deviations caused by signals, which leads to large errors in the existing technology when superimposing virtual models and real scenes.

[0134] The present application provides a method for converting geographic coordinates to world coordinates of virtual reality software, so that the position where the virtual model is superimposed on the real scene is closer to the preset position, which can reduce the error.

[0135] And compared with the prior art, after Pokemon go reaches the specified geographic coordinate range, all AR interactive links are no longer related to the geographic coordinates. The compass is only used for user orientation guidance, and the object of technical service is game entertainment. The range of geographic coordinates in this application is not used for entry triggering, but visual triggering, that is, it is triggered when the geographic coordinates of the pointed area are within the visible range of the current camera. Moreover, the geographic coordinates are not only used to trigger the AR module. After triggering the entry into the AR module, the range of geographic coordinates is used to specify the overall scaling and placement position of the model. The compass is not only used for user orientation guidance, but also participates in the calculation of the placement orientation when the virtual model is superimposed. Therefore, the application field of this application is more inclined to the practical construction planning field.

[0136] Based on the above method of realizing the virtual model based on orientation and superimposing the real scene, see Figure 7 The present application also provides a device for realizing the superposition of a virtual model and a real scene based on an orientation, the device comprising:

[0137] The acquisition module 701 is used to acquire the real-time geographic coordinates of the mobile device, and transform the geographic coordinates of the two target points on the virtual model into the virtual reality software installed on the mobile device according to the real-time geographic coordinates to obtain the world coordinates of the two target points in the virtual reality software;

[0138] A scaling value determining module 702 is used to determine a first distance between the two target points on the virtual model according to the geographic coordinates of the two target points, determine a second distance between the two target points in the virtual reality software according to the world coordinates of the two target points, and determine a scaling value of the virtual model according to the first distance and the second distance;

[0139] The overlay module 703 is used to scale the virtual model according to the scaling value to obtain a scaled virtual model, determine the placement direction and placement orientation of the scaled virtual model, and place the scaled virtual model in the real scene captured by the main perspective camera in the virtual reality software according to the placement direction and the placement orientation.

[0140] It can be seen that the device can obtain the real-time geographic coordinates of the mobile device, convert the geographic coordinates of the two target points on the virtual model into the virtual reality software installed on the mobile device according to the real-time geographic coordinates, and obtain the world coordinates of the two target points in the virtual reality software; determine the first distance between the two target points on the virtual model according to the geographic coordinates of the two target points, determine the second distance between the two target points in the virtual reality software according to the world coordinates of the two target points, and determine the scaling value of the virtual model according to the first distance and the second distance; scale the virtual model according to the scaling value to obtain the scaled virtual model, determine the placement direction and placement orientation of the scaled virtual model, and place the scaled virtual model in the real scene captured by the main visual camera in the virtual reality software according to the placement direction and placement orientation. In the technical solution of the present application, ARkit and geo-positioning technologies are used in the Unity3d engine. By taking the geographic coordinates as a reference for the placement of the virtual model, the virtual model combined with the real scene has a spatial orientation memory function. There is no need to stick markers on the ground and scan the real scene. The virtual model can be laid on the real scene ground according to the geographic coordinates, that is, the virtual model is placed in the real scene. The operation is convenient and highly intelligent, and the urban construction space combined with the real scene can be visualized.

[0141] In one implementation, the acquisition module 701 may include:

[0142] A target world coordinate determination submodule, used to determine the target world coordinates of the mobile device in the virtual reality software according to the real-time geographic coordinates;

[0143] The determination submodule is used to calculate, for each target point, the longitude span and latitude span from the point where the mobile device is located on the virtual model to the target point, calculate a first product of the length of the north direction indicated by the compass in the virtual reality software and the latitude span of the target point, calculate a second product of the length of the east direction indicated by the compass in the virtual reality software and the longitude span of the target point, calculate the sum of the target world coordinates, the first product and the second product, and use the sum as the world coordinates of the target point in the virtual reality software.

[0144] In one implementation, the determining submodule may be specifically used for:

[0145] For each target point, the longitude span from the point where the mobile device is located on the virtual model to the target point is calculated according to the following formula:

[0146] locross(n)=(lon-lon(n))*Gn'O*π

[0147] Gn'O=GnO*cos(lat(n))

[0148] Wherein, locross(n) is the longitude span from the point where the mobile device is located on the virtual model to the target point, n is the identifier of the target point, lon is the degree of the angle QOG', lon(n) is the degree of the angle QOGn', Q is the intersection of the prime meridian and the equator, O is the center point of the equator, G' is the first intersection point between the first perpendicular line from the point where the mobile device is located on the virtual model to the equatorial plane and the equatorial plane, Gn' is the second intersection point between the second perpendicular line from the target point to the equatorial plane on the virtual model and the equatorial plane, Gn'O is the line between the center point of the equator and the second intersection point, π is pi, GnO is the line between the target point on the virtual model and the center point of the equator, lat(n) is the degree of the angle GnOGn', Gn is the target point on the virtual model, the range of lon and lon(n) are both [-180°, 180°], and the range of lat(n) is [-90°, 90°].

[0149] In one implementation, the determining submodule may be specifically used for:

[0150] For each target point, the latitude span from the point where the mobile device is located on the virtual model to the target point is calculated according to the following formula:

[0151] lacross(n)=(lat-lat(n))*GnO*π

[0152] Among them, lacross(n) is the longitude span from the point where the mobile device is located on the virtual model to the target point, n is the identifier of the target point, lat is the degree of angle GOG', G is the point where the mobile device is located on the virtual model, O is the center point of the equator, G' is the first intersection point between the first perpendicular line from the point where the mobile device is located on the virtual model to the equatorial plane and the equatorial plane, lat(n) is the degree of angle GnOGn', Gn is the target point on the virtual model, Gn' is the second intersection point between the second perpendicular line from the target point to the equatorial plane on the virtual model and the equatorial plane, GnO is the line between the target point and the center point of the equator on the virtual model, π is pi, and the range of lat and lat(n) are both [-90°, 90°].

[0153] In one implementation, the scaling value determination module 702 may be specifically configured to:

[0154] A quotient between the first distance and the second distance is calculated as a scaling value of the virtual model.

[0155] In one implementation, the superposition module 703 may be specifically used for:

[0156] When the geographical coordinates of the two target points in the scaled virtual model coincide with the world coordinates of the two target points in the virtual reality software, the direction and orientation of the virtual model are used as the placement direction and placement orientation respectively.

[0157] Figure 8 It is a structural diagram of an electronic device provided in an embodiment of the present application. At the hardware level, the electronic device includes a processor, and optionally also includes an internal bus, a network interface, and a memory. Among them, the memory may include a memory, such as a high-speed random access memory (Random-Access Memory, RAM), and may also include a non-volatile memory (non-volatile memory), such as at least one disk storage, etc. Of course, the electronic device may also include hardware required for other services.

[0158] The processor, network interface and memory can be interconnected through an internal bus, which can be an ISA (Industry Standard Architecture) bus, a PCI (Peripheral Component Interconnect) bus or an EISA (Extended Industry Standard Architecture) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 8 Only one bidirectional arrow is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0159] The memory is used to store execution instructions. Specifically, the execution instructions are computer programs that can be executed. The memory may include internal memory and non-volatile memory, and provides execution instructions and data to the processor.

[0160] In one possible implementation, the processor reads the corresponding execution instructions from the non-volatile memory into the memory and then runs them, and can also obtain the corresponding execution instructions from other devices to form a virtual model based on the position and a real scene superposition device at the logical level. The processor executes the execution instructions stored in the memory to implement the method for superimposing a virtual model based on the position and a real scene provided in any embodiment of the present application through the execution of the execution instructions.

[0161] The above application Figure 1The method for implementing the virtual model based on orientation and the real scene superposition provided by the illustrated embodiment can be applied to a processor or implemented by a processor. The processor may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method can be completed by an integrated logic circuit of hardware in the processor or instructions in the form of software. The above processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The various methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor, etc.

[0162] The steps of the method disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware decoding processor, or can be executed by a combination of hardware and software modules in the decoding processor. The software module can be located in a storage medium mature in the art such as a random access memory, a flash memory, a read-only memory, a programmable read-only memory, or an electrically erasable programmable memory, a register, etc. The storage medium is located in a memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0163] An embodiment of the present application further proposes a readable storage medium, which stores execution instructions. When the stored execution instructions are executed by a processor of an electronic device, the electronic device can execute the method for implementing the orientation-based virtual model and real scene overlay provided in any embodiment of the present application.

[0164] The electronic device described in the above embodiments may be a computer.

[0165] Those skilled in the art should understand that the embodiments of the present application can be provided as methods or computer program products. Therefore, the present application can adopt a complete hardware embodiment, a complete software embodiment, or a combination of software and hardware.

[0166] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0167] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0168] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A method for realizing the superposition of virtual model and real scene based on orientation, It is characterized in that include: Acquire the real-time geographic coordinates of the mobile device, and transform the geographic coordinates of two target points on the virtual model into the virtual reality software installed on the mobile device according to the real-time geographic coordinates to obtain the world coordinates of the two target points in the virtual reality software; Determine a first distance between the two target points on the virtual model according to the geographic coordinates of the two target points, determine a second distance between the two target points in the virtual reality software according to the world coordinates of the two target points, and determine a scaling value of the virtual model according to the first distance and the second distance; Scaling the virtual model according to the scaling value to obtain a scaled virtual model, determining a placement direction and a placement orientation of the scaled virtual model, and placing the scaled virtual model in a real scene captured by a primary viewing angle camera in the virtual reality software according to the placement direction and the placement orientation; The step of converting the geographic coordinates of the two target points on the virtual model into the virtual reality software installed on the mobile device according to the real-time geographic coordinates to obtain the world coordinates of the two target points in the virtual reality software includes: Determine the target world coordinates of the mobile device in the virtual reality software according to the real-time geographic coordinates; For each target point, calculate the longitude span and latitude span from the point where the mobile device is located on the virtual model to the target point, calculate a first product of the length of the north direction indicated by the compass in the virtual reality software and the latitude span of the target point, calculate a second product of the length of the east direction indicated by the compass in the virtual reality software and the longitude span of the target point, calculate the sum of the target world coordinates, the first product and the second product, and use the sum as the world coordinates of the target point in the virtual reality software; The step of calculating the longitude span from the point on the virtual model of the mobile device to the target point for each target point includes: For each target point, the longitude span from the point where the mobile device is located on the virtual model to the target point is calculated according to the following formula: locross(n)=(lon-lon(n))*Gn'O*π Gn'O=GnO*cos(lat(n)) Wherein, locross(n) is the longitude span from the point where the mobile device is located on the virtual model to the target point, n is the identifier of the target point, lon is the degree of the angle QOG', lon(n) is the degree of the angle QOGn', Q is the intersection of the prime meridian and the equator, O is the center point of the equator, G' is the first intersection point between the first perpendicular line from the point where the mobile device is located on the virtual model to the equatorial plane and the equatorial plane, Gn' is the second intersection point between the second perpendicular line from the target point to the equatorial plane on the virtual model and the equatorial plane, Gn'O is the line between the center point of the equator and the second intersection point, π is pi, GnO is the line between the target point on the virtual model and the center point of the equator, lat(n) is the degree of the angle GnOGn', Gn is the target point on the virtual model, the ranges of lon and lon(n) are both [-180°, 180°], and the range of lat(n) is [-90°, 90°]; The step of calculating the latitude span from the point where the mobile device is located on the virtual model to the target point includes: For each target point, the latitude span from the point where the mobile device is located on the virtual model to the target point is calculated according to the following formula: lacross(n)=(lat-lat(n))*GnO*π Wherein, lacross(n) is the latitude span from the point where the mobile device is located on the virtual model to the target point, n is the identifier of the target point, lat is the degree of the angle GOG', G is the point where the mobile device is located on the virtual model, O is the center point of the equator, G' is the first intersection point between the first perpendicular line from the point where the mobile device is located on the virtual model to the equatorial plane and the equatorial plane, lat(n) is the degree of the angle GnOGn', Gn is the target point on the virtual model, Gn' is the second intersection point between the second perpendicular line from the target point to the equatorial plane on the virtual model and the equatorial plane, GnO is the line between the target point and the center point of the equator on the virtual model, π is pi, and the ranges of lat and lat(n) are both [-90°, 90°].

2. The method according to claim 1, It is characterized in that The step of determining the scaling value of the virtual model according to the first distance and the second distance comprises: A quotient between the first distance and the second distance is calculated as a scaling value of the virtual model.

3. The method according to claim 1, It is characterized in that The step of determining the placement direction and orientation of the scaled virtual model includes: When the geographical coordinates of the two target points in the scaled virtual model coincide with the world coordinates of the two target points in the virtual reality software, the direction and orientation of the virtual model are used as the placement direction and placement orientation respectively.

4. The method according to claim 1, It is characterized in that The method is implemented based on Unity3d engine, ARKit and satellite positioning.

5. A device for realizing virtual model and real scene superposition based on orientation, It is characterized in that include: An acquisition module, used to acquire the real-time geographic coordinates of the mobile device, and transform the geographic coordinates of two target points on the virtual model into the virtual reality software installed on the mobile device according to the real-time geographic coordinates, so as to obtain the world coordinates of the two target points in the virtual reality software; a scaling value determination module, configured to determine a first distance between the two target points on the virtual model according to the geographic coordinates of the two target points, determine a second distance between the two target points in the virtual reality software according to the world coordinates of the two target points, and determine a scaling value of the virtual model according to the first distance and the second distance; a superposition module, configured to scale the virtual model according to the scaling value to obtain a scaled virtual model, determine a placement direction and a placement orientation of the scaled virtual model, and place the scaled virtual model in a real scene captured by a primary viewing angle camera in the virtual reality software according to the placement direction and the placement orientation; The step of converting the geographic coordinates of the two target points on the virtual model into the virtual reality software installed on the mobile device according to the real-time geographic coordinates to obtain the world coordinates of the two target points in the virtual reality software includes: Determine the target world coordinates of the mobile device in the virtual reality software according to the real-time geographic coordinates; For each target point, calculate the longitude span and latitude span from the point where the mobile device is located on the virtual model to the target point, calculate a first product of the length of the north direction indicated by the compass in the virtual reality software and the latitude span of the target point, calculate a second product of the length of the east direction indicated by the compass in the virtual reality software and the longitude span of the target point, calculate the sum of the target world coordinates, the first product and the second product, and use the sum as the world coordinates of the target point in the virtual reality software; The step of calculating the longitude span from the point on the virtual model of the mobile device to the target point for each target point includes: For each target point, the longitude span from the point where the mobile device is located on the virtual model to the target point is calculated according to the following formula: locross(n)=(lon-lon(n))*Gn'O*π Gn'O=GnO*cos(lat(n)) Wherein, locross(n) is the longitude span from the point where the mobile device is located on the virtual model to the target point, n is the identifier of the target point, lon is the degree of the angle QOG', lon(n) is the degree of the angle QOGn', Q is the intersection of the prime meridian and the equator, O is the center point of the equator, G' is the first intersection point between the first perpendicular line from the point where the mobile device is located on the virtual model to the equatorial plane and the equatorial plane, Gn' is the second intersection point between the second perpendicular line from the target point to the equatorial plane on the virtual model and the equatorial plane, Gn'O is the line between the center point of the equator and the second intersection point, π is pi, GnO is the line between the target point on the virtual model and the center point of the equator, lat(n) is the degree of the angle GnOGn', Gn is the target point on the virtual model, the ranges of lon and lon(n) are both [-180°, 180°], and the range of lat(n) is [-90°, 90°]; The step of calculating the latitude span from the point where the mobile device is located on the virtual model to the target point includes: For each target point, the latitude span from the point where the mobile device is located on the virtual model to the target point is calculated according to the following formula: lacross(n)=(lat-lat(n))*GnO*π Wherein, lacross(n) is the latitude span from the point where the mobile device is located on the virtual model to the target point, n is the identifier of the target point, lat is the degree of the angle GOG', G is the point where the mobile device is located on the virtual model, O is the center point of the equator, G' is the first intersection point between the first perpendicular line from the point where the mobile device is located on the virtual model to the equatorial plane and the equatorial plane, lat(n) is the degree of the angle GnOGn', Gn is the target point on the virtual model, Gn' is the second intersection point between the second perpendicular line from the target point to the equatorial plane on the virtual model and the equatorial plane, GnO is the line between the target point and the center point of the equator on the virtual model, π is pi, and the ranges of lat and lat(n) are both [-90°, 90°].

6. An electronic device, It is characterized in that include: A processor and a memory storing execution instructions, when the processor executes the execution instructions stored in the memory, the processor executes the method according to any one of claims 1 to 4.