Method, device, electronic device and storage medium for positioning virtual objects

Abstract

The present application discloses a method, device, electronic device and storage medium for positioning a virtual object. The method for positioning a virtual object includes: positioning the virtual object on the area to be fused, and obtaining first position information of the virtual object on the area to be fused; obtaining the intersection of the virtual object and the virtual model in the area to be fused using the first position information; and adjusting the second position information of the virtual object using the intersection to realize the placement of the virtual object in the virtual digital space. In the above manner, the present application can simplify the operation of virtual objects in the virtual-reality fusion scene to reduce the difficulty of virtual fusion and improve its convenience of use.

Description

Technical Field

[0001] The present application relates to the technical field of augmented reality, and in particular to a method, device, electronic device and storage medium for positioning a virtual object. Background Art

[0002] Augmented Reality (AR) is a technology that cleverly integrates virtual information with the real world. It widely uses a variety of technical means such as multimedia, 3D modeling, real-time tracking and registration, intelligent interaction, and sensing. AR simulates computer-generated text, images, 3D models, music, and videos, and applies them to the real world, so that the two types of information complement each other, thereby achieving "enhancement" of the real world. AR technology has been developed for many years. I hope that one day I can use this technology in my daily life and work to provide convenience and improve work efficiency. At present, many work scenes have also appeared. With the advancement of technology, there are also global positioning methods based on multi-sensor fusion such as visual high-precision maps and signal positioning. Various science fiction concepts such as digital twins and parallel worlds are gradually becoming possible.

[0003] How to place virtual objects in a virtual-real fusion scene has become a key operation. The existing technology usually uses a self-developed editor to place virtual objects, or uses a plug-in on Unity3D and a plug-in edited by Unreal Engine 4 to align virtual and real content. However, these tools are highly professional and the operations are very complicated. Therefore, obtaining a more convenient method to realize the operation of virtual objects in a virtual-real fusion scene has become an urgent problem to be solved in augmented reality applications. Summary of the invention

[0004] The present application provides a method, device, electronic device and storage medium for positioning a virtual object, which can simplify the operation of virtual objects in a virtual-reality fusion scene to reduce the difficulty of virtual fusion and improve its convenience of use.

[0005] In order to solve the above technical problems, the first aspect of the present application provides a method for positioning a virtual object. The method for positioning a virtual object includes: positioning a virtual object on a region to be fused, and obtaining first position information of the virtual object on the region to be fused; using the first position information to obtain an intersection between the virtual object and a virtual model in the region to be fused; and using the intersection to adjust the second position information of the virtual object to realize the placement of the virtual object in a virtual digital space.

[0006] Therefore, by locating and acquiring the first position information of the virtual object, using the first position information of the virtual object on the area to be fused to acquire the intersection of the virtual object and the virtual model in the area to be fused, and using the intersection to adjust the second position information of the virtual object, the position of the virtual object in the virtual digital space can be finally determined to realize the placement of the virtual object in the virtual digital space. The present application can realize the placement of the virtual object in the virtual digital space by locating the virtual object on the area to be fused and using the first position information of the virtual object on the area to be fused. The operation is simple and easy, and the virtual object editing operation in the virtual fusion scene can be simplified to reduce the difficulty of virtual fusion and improve its convenience.

[0007] Among them, the first position information includes GPS information. Before using the first position information to obtain the intersection of the virtual object and the virtual model in the area to be fused, the positioning method of the virtual object further includes: obtaining the conversion relationship between the GPS coordinate system and the high-precision map coordinate system; using the conversion relationship and GPS information to transform the virtual object to the high-precision map space; the above-mentioned use of the first position information to obtain the intersection of the virtual object and the virtual model in the area to be fused includes: obtaining the intersection of the virtual object and the virtual model in the high-precision map space according to the GPS information.

[0008] Therefore, by obtaining the conversion relationship between the GPS coordinate system and the high-precision map coordinate system, and using the conversion relationship and GPS information to transform the virtual object into the high-precision map space, the virtual object can be placed in the high-precision map space to improve the placement accuracy of the virtual object and increase the virtual-reality fusion effect.

[0009] Among them, the GPS information includes longitude and latitude coordinates. Obtaining the intersection of the virtual object and the virtual model in the high-precision map space according to the GPS information includes: determining the gravity direction of the virtual object in the high-precision map space; obtaining the local coordinates of the virtual object in the high-precision map space, and obtaining the intersection of the straight line along the gravity direction and passing through the local coordinates with the virtual model in the high-precision map space.

[0010] Because the setting of the virtual model in the high-precision map space is the same as the setting of the real object in the real scene, both are along the direction of gravity. Therefore, the intersection of the straight line along the direction of gravity and passing through the local coordinates of the virtual object in the high-precision map space and the virtual model in the high-precision map space is the intersection of the real object in the real scene and the virtual object of the virtual model. Therefore, the placement accuracy of the virtual object can be improved and the virtual-real fusion effect can be enhanced.

[0011] Among them, using the intersection to adjust the second position information of the virtual object includes: obtaining filtering information; wherein the filtering information includes preset information of the intersection or preset information of the virtual model; using the filtering information to filter multiple intersections, and adjusting the second position information of the virtual object in the high-precision map space according to the filtered intersections.

[0012] Because there are usually multiple virtual models with the same longitude and latitude as the virtual object along the gravity direction in the high-precision map space, therefore, by screening the intersections through the preset information of the intersections or the preset information of the virtual models, the range of the virtual object placement can be narrowed, the placement accuracy can be improved, and the virtual-real fusion effect can be increased.

[0013] Among them, the second position information includes the height of the virtual object relative to the ground plane in the high-precision map space, and adjusting the second position information of the virtual object in the high-precision map space according to the filtered intersection further includes: obtaining a preset height; adjusting the distance between the virtual object and the ground plane along the direction of gravity to the preset height.

[0014] Because both the virtual object and the virtual model have a certain size, and the intersection of the virtual object and the virtual model is usually the intersection of a mark point on the virtual object and a mark point on the virtual model, by adjusting the distance between the virtual object and the ground plane along the direction of gravity to a preset height, the relative position of the virtual object and the virtual object along the direction of gravity can be fine-tuned to improve the placement accuracy of the virtual object.

[0015] Among them, obtaining the conversion relationship between the GPS coordinate system and the high-precision map coordinate system includes: dividing the area to be fused into multiple sub-areas; obtaining a high-precision map of each sub-area respectively, and obtaining the high-precision map coordinate system of each sub-area; obtaining the conversion matrix between the GPS coordinate system of the area to be fused and the high-precision map coordinate system of each sub-area.

[0016] Therefore, the high-precision map is usually a high-precision map of a local area. By obtaining the transformation matrix between the GPS coordinate system of the area to be fused and the high-precision map coordinate system of each sub-area, the high-precision map coordinate system of each sub-area can be aligned with the GPS coordinate system using the transformation relationship to unify the high-precision map coordinate system of each sub-area.

[0017] Among them, the positioning method of the virtual object further includes: obtaining a viewpoint and determining a sub-region to be fused corresponding to the viewpoint; obtaining a virtual object in the sub-region to be fused; and fusing and displaying the virtual object with a real scene in the sub-region to be fused.

[0018] Therefore, the sub-region to be fused is first determined by the viewpoint, and then the virtual object in the sub-region to be fused is obtained, and the virtual object is fused and displayed with the real scene of the sub-region to be fused, which can simplify the complexity of virtual fusion.

[0019] Wherein, determining the sub-region to be fused corresponding to the viewpoint includes: obtaining digital position coordinates corresponding to the position coordinates of the viewpoint in the virtual digital space; obtaining an area with the digital position coordinates as the center and a preset distance as the radius as the sub-region to be fused.

[0020] Therefore, the digital position coordinates corresponding to the position coordinates of the viewpoint in the virtual digital space are first obtained, and then the area with the digital position coordinates as the center and the preset distance as the radius is obtained as the sub-area to be fused, so that the sub-area to be fused can be determined.

[0021] To solve the above technical problems, the second aspect of the present application provides a positioning device for a virtual object, which includes: a positioning module, used to position the virtual object on the area to be fused; an acquisition module, coupled to the positioning module, and the acquisition module is used to obtain the first position information of the virtual object on the area to be fused from the positioning module; a calculation module, coupled to the acquisition module, and the calculation module is used to use the first position information to obtain the intersection of the virtual object and the virtual model in the area to be fused; an adjustment module, coupled to the calculation module and the positioning module, and the adjustment module is used to use the intersection to adjust the second position information of the virtual object to realize the placement of the virtual object in the virtual digital space.

[0022] To solve the above technical problems, the third aspect of the present application provides an electronic device, which includes: a memory and a processor coupled to each other, the processor being used to execute program data stored in the memory to implement any of the above virtual object positioning methods.

[0023] In order to solve the above technical problems, the fourth aspect of the present application provides a computer-readable storage medium, which stores program data, and the program data can be executed to implement any of the above-mentioned virtual object positioning methods.

[0024] The above scheme first positions the virtual object on the area to be fused, and obtains the first position information of the virtual object on the area to be fused; then uses the first position information to obtain the intersection of the virtual object and the virtual model in the area to be fused; and finally uses the intersection to adjust the second position information of the virtual object to achieve the placement of the virtual object in the virtual digital space. The present application positions the virtual object on the area to be fused, determines the first position information of the virtual object in the virtual digital space, determines the second position information of the virtual object in the virtual digital space through the intersection of the virtual object and the virtual model in the area to be fused, and finally determines the placement position of the virtual object in the virtual digital space based on the first position information and the second position information, thereby achieving the placement of the virtual object in the virtual digital space. The operation method is simple and can simplify the operation of virtual objects in the virtual-real fusion scene, thereby reducing the difficulty of virtual-real fusion and improving its convenience of use. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] Figure 1 It is a flowchart of an embodiment of a method for locating a virtual object of the present application;

[0026] Figure 2 It is a flowchart of an embodiment of a method for locating a virtual object of the present application;

[0027] Figure 3 yes Figure 2 A schematic diagram of a specific flow chart of step S202 in the method for locating a virtual object in an embodiment;

[0028] Figure 4 yes Figure 2 A schematic diagram of a specific flow chart of step S204 in the method for locating a virtual object in an embodiment;

[0029] Figure 5 yes Figure 2 A schematic diagram of a specific flow chart of step S205 in the method for locating a virtual object in the embodiment;

[0030] Figure 6 yes Figure 2 A schematic diagram of a specific flow chart of step S502 in the method for locating a virtual object in an embodiment;

[0031] Figure 7 It is a flowchart of an embodiment of a method for locating a virtual object of the present application;

[0032] Figure 8 yes Figure 7 A schematic diagram of a specific flow chart of step S706 in the method for locating a virtual object in the embodiment;

[0033] Fig. 9 It is a structural schematic diagram of an embodiment of a positioning device for a virtual object of the present application;

[0034] Fig.10 It is a structural schematic diagram of an embodiment of the electronic device of the present application;

[0035] Fig.11 It is a structural diagram of an embodiment of a computer-readable storage medium of the present application. DETAILED DESCRIPTION

[0036] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0037] See also Figure 1 , Figure 1 1 is a flow chart of an embodiment of a method for locating a virtual object of the present application. The method for locating a virtual object of the present embodiment includes the following steps:

[0038] Step S101: Positioning a virtual object on the area to be fused, and obtaining first position information of the virtual object on the area to be fused.

[0039] In this embodiment, the area to be fused that needs to be fused can be determined first, wherein the area to be fused can be selected according to actual needs, and the area to be fused is the area that needs to be studied through the virtual-real fusion technology; then, a regional map of the area to be fused is obtained; then the virtual object is positioned on the regional map of the area to be fused, and the first position information of the virtual object on the regional map is obtained. Specifically, the model of the virtual object can be imported into the virtual digital space, and the virtual object can be moved to the target position on the regional map through the control command input by the input module such as the mouse, and the position information of the target position is obtained as the first position information of the virtual object on the regional map.

[0040] The regional map of this embodiment is an electronic map in a geographic information system, and the electronic map may be in the form of a planning map or an aerial photo, and the electronic map may be a Baidu map or an Amap map.

[0041] The first position information of the virtual object on the electronic map in this embodiment is GPS information. In other embodiments, the first position information of the virtual object on the electronic map may also be relative position information of the virtual object and a reference point in the electronic map.

[0042] In this embodiment, the virtual object can be positioned by map software, navigation software or secondary development tools.

[0043] In one application scenario, first determine the area "park" to be integrated that needs to be integrated with the virtual and the real; then open the secondary development tool, and find the regional map of the area "park" to be integrated from the Gaode map; then select the virtual object "tree" that needs to be placed in the area "park" to be integrated, and position the virtual object "tree" on the regional map of the area "park" to be integrated, and obtain the GPS information of the virtual object "tree" on the regional map of the area "park" to be integrated.

[0044] Since electronic maps usually include large-scale regional maps, for example, regional maps of the entire city, the entire country, or even the world, regional maps of the areas to be fused can be obtained from electronic maps through regional searches and other methods based on information such as the identification of the areas to be fused. Therefore, when locating virtual objects in different areas to be fused, there is no need to divide and splice the areas. The operation is simple, and the requirements for the user's professional knowledge are relatively low, which can improve the convenience of user use.

[0045] Step S102: using the first position information to obtain the intersection point between the virtual object and the virtual model in the area to be fused.

[0046] In this embodiment, the regional map of the area to be fused, such as an electronic map, is usually formed by obtaining real scene pictures and location information by means of a signal collection vehicle or aerial photography, so the virtual model in the electronic map usually corresponds to the real object in the real scene. Therefore, the intersection of the virtual object in this embodiment and the virtual model in the area to be fused corresponds to the intersection of the virtual object and the real object in the real scene of the area to be fused.

[0047] The GPS information includes longitude and latitude coordinates. In this embodiment, a point on the virtual model having the same longitude and latitude coordinates as the virtual object can be obtained as the intersection point of the virtual object and the virtual model.

[0048] In one application scenario, a point on a virtual model "tree" in the area "park" to be fused that has the same longitude and latitude coordinates as the virtual object "butterfly" is obtained as the intersection of the virtual object "butterfly" and the virtual model "tree".

[0049] The GPS information of the virtual object may be the GPS information of the center point of the virtual object, or the GPS information of the maximum point of the virtual object in a certain direction, for example, the GPS information of the minimum point of the virtual object along the gravity direction.

[0050] Step S103: using the intersection point to adjust the second position information of the virtual object to achieve placement of the virtual object in the virtual digital space.

[0051] The second position information of this embodiment may be height information of the virtual object relative to a reference plane (such as a ground plane) of the area to be fused. In other embodiments, the second position information may also be relative height information of the virtual object and other virtual models.

[0052] From the above analysis, it can be seen that in this embodiment, the virtual object can be positioned through map software, navigation software or secondary development tools to determine the first position information of the virtual object in the virtual digital space, and the second position information of the virtual object in the virtual digital space can be determined through the intersection of the virtual object and the virtual model in the area to be merged, so as to finally determine the placement position of the virtual object in the virtual digital space, and realize the placement of the virtual object in the virtual digital space.

[0053] Among them, the second position information and intersection point of this embodiment can be the position information and intersection point in the high-precision map space coordinate system; the position information and intersection point can also be converted into the geodetic coordinate system.

[0054] Different from the prior art, this embodiment locates the virtual object on the area to be fused, determines the first position information of the virtual object in the virtual digital space, determines the second position information of the virtual object in the virtual digital space through the intersection of the virtual object and the virtual model in the area to be fused, and finally determines the placement position of the virtual object in the virtual digital space according to the first position information and the second position information, so as to realize the placement of the virtual object in the virtual digital space. The operation method is simple, and can simplify the operation of virtual objects in the virtual-reality fusion scene, thereby reducing the difficulty of virtual-reality fusion and improving its convenience of use.

[0055] The present application further proposes a method for positioning a virtual object in another embodiment, such as Figure 2 As shown, Figure 2 1 is a flow chart of an embodiment of a method for realizing virtual-real fusion in the present application. The method for realizing virtual-real fusion in the present embodiment includes the following steps:

[0056] Step S201: locate the virtual object on the electronic map of the area to be fused, and obtain the GPS information of the virtual object on the electronic map.

[0057] In this embodiment, the area to be fused that needs to be fused with virtual and real objects can be first determined, wherein the area to be fused can be selected according to actual needs, and the area to be fused is an area that needs to be studied through the virtual and real fusion technology; then, an electronic map of the area to be fused is obtained; then, the virtual object is located on the electronic map, and the GPS information of the virtual object on the electronic map is obtained.

[0058] In this embodiment, the virtual object can be positioned by map software, navigation software or secondary development tools.

[0059] In an application scenario, first determine the area "square" to be integrated where virtual and real integration is required; then open the secondary development tool, and find the electronic map of the area "square" to be integrated from Baidu Maps; then select the virtual object "fountain" that needs to be placed in the area "square" to be integrated, and locate the virtual object "fountain" on the electronic map of the area "square" to be integrated, and obtain the GPS information of the virtual object "fountain" in the area "square" to be integrated.

[0060] Since electronic maps usually include large-scale regional maps, such as regional maps of the entire city, the entire country, or even the entire world, regional maps of the area to be fused can be obtained from the electronic map through simple operations. Therefore, when locating virtual objects in different areas to be fused, there is no need to divide and splice the areas. The operation is simple, the requirements for user expertise are low, and the convenience of user use can be improved.

[0061] Step S202: Obtain the conversion relationship between the GPS coordinate system and the high-precision map coordinate system.

[0062] Specifically, this embodiment can be implemented by Figure 3 The method shown implements step S202. The method of this embodiment includes steps S301 to S303.

[0063] Step S301: Divide the area to be fused into multiple sub-areas.

[0064] The entire area to be integrated is divided into multiple sub-areas, wherein the multiple sub-areas may have cross-region, cross-building, cross-floor or global association relationships.

[0065] The sub-area division standard may be based on the maximum range that can be collected at one time by the virtual digital space, or other division standards, which are not limited here.

[0066] Step S302: Obtain a high-precision map of each sub-area respectively, and obtain a high-precision map coordinate system of each sub-area.

[0067] Step S303: Obtain the transformation matrix between the GPS coordinate system of the area to be fused and the high-precision map coordinate system of each sub-area.

[0068] Specifically, the area to be fused is divided into multiple sub-areas, and at least three sets of GPS longitude and latitude coordinate sets C0 are obtained for each sub-area. The Gaussian three-dimensional zone projection is performed on the central meridian where the GPS is located to obtain the ENZ coordinate system and the new coordinate set E0. Since the GPS coordinate system belongs to the global coordinate system, the projected ENZ coordinate system also belongs to the global coordinate system. In order to improve the accuracy of the local coordinate system, the origin of the ENZ coordinate system is translated, as long as the origin can include all sub-areas after translation, and finally the local ENZ coordinate system and the new coordinate set E1 are obtained.

[0069] The Sfm algorithm is used to construct a point cloud for each sub-area separately, and the point cloud is projected to obtain a 2D point cloud projection map (high-precision map), and a transformation matrix R0 from the Sfm coordinate system to the point cloud projection image coordinate system (high-precision map coordinate system) is generated.

[0070] Then, the ENZ coordinate system and the point cloud projection image coordinate system (high-precision map coordinate system) are projected and transformed to obtain the transformation matrix R1.

[0071] Finally, the transformation matrix Sfm coordinate system *R0*R1 is obtained. Through this transformation matrix, the point cloud projection image coordinate system (high-precision map coordinate system) of each sub-area can be projected onto the ENZ coordinate system, thereby realizing the unification of the point cloud projection image coordinate system (high-precision map coordinate system) of all sub-areas.

[0072] When building high-precision maps for ultra-large areas, such as areas of tens of kilometers or larger, it is necessary to divide the ultra-large area into multiple sub-areas. Each sub-area has an independent coordinate system after being mapped separately. In order to support seamless integration of regional AR navigation and AR effects, the coordinate systems of all sub-areas must be unified.

[0073] In other embodiments, the execution order of step S202 and step S201 may not be limited, and step S202 may be executed before step S201 or synchronously with step S201.

[0074] Step S203: transform the virtual object into a high-precision map space using the conversion relationship and GPS information.

[0075] From the above analysis, it can be seen that the conversion relationship is the conversion matrix between the GPS coordinate systems of multiple sub-areas of the area to be fused and the high-precision map coordinate systems of the multiple sub-areas. Therefore, this embodiment can align the GPS coordinate system of the sub-area where the virtual object is located with the high-precision map coordinate system of the sub-area according to the conversion matrix, and can align the GPS coordinates of the virtual object in the sub-area to the corresponding digital coordinates of the high-precision map coordinate system.

[0076] By obtaining the conversion relationship between the GPS coordinate system and the high-precision map coordinate system, and using the conversion relationship and GPS information to transform the virtual object into the high-precision map space, the virtual object can be placed in the high-precision map space to improve the placement accuracy of the virtual object and increase the virtual-reality fusion effect.

[0077] Step S204: Obtain the intersection point between the virtual object and the virtual model in the high-precision map space according to the GPS information.

[0078] The virtual model in the high-precision map space of the area to be fused in this embodiment usually corresponds to the real object in the real scene. Therefore, the intersection of the virtual object in this embodiment and the virtual model in the high-precision map space of the area to be fused corresponds to the intersection of the virtual object and the real object in the real scene of the area to be fused.

[0079] The GPS information includes longitude and latitude coordinates. This embodiment converts the longitude and latitude coordinates of the virtual object into local coordinates in a high-precision coordinate system, and obtains the point on the virtual model in the high-precision coordinate system that has the same local coordinates as the virtual object as the intersection of the virtual object and the virtual model.

[0080] The GPS information of the virtual object may be the GPS information of the center point of the virtual object, or the GPS information of the maximum point of the virtual object in a certain direction, for example, the GPS information of the minimum point of the virtual object along the gravity direction.

[0081] Specifically, this embodiment can be implemented by Figure 4 The method shown implements step S204. The method of this embodiment specifically includes step S401 and step S402.

[0082] Step S401: Determine the gravity direction of the virtual object in the high-precision map space.

[0083] From the above analysis, we can know that the gravity direction of the high-precision map space should be perpendicular to the ground plane in the high-precision map space. For example, in a limited area, if the ground plane in the high-precision map space is located in the xy plane, then the gravity direction is perpendicular to the xy plane, that is, parallel to the z axis.

[0084] Step S402: Obtain the local coordinates of the virtual object in the high-precision map space, and obtain the intersection of the straight line along the gravity direction and passing through the local coordinates and the virtual model in the high-precision map space.

[0085] Because the setting of the virtual model in the high-precision map space is the same as the setting of the real object in the real scene, both are along the direction of gravity. Therefore, the intersection of the straight line along the direction of gravity and passing through the local coordinates of the virtual object in the high-precision map space and the virtual model in the high-precision map space is the intersection of the real object in the real scene and the virtual object of the virtual model. Therefore, the placement accuracy of the virtual object can be improved and the virtual-real fusion effect can be enhanced.

[0086] In this embodiment, a point on the virtual model in the high-precision map space that has the same local coordinates as the virtual object can be obtained as the intersection point of the virtual object and the virtual model.

[0087] In an application scenario, in a high-precision map space, a point on the virtual model "ground" in the area "square" to be fused that has the same local coordinates as the virtual object "fountain" is obtained as the intersection of the virtual object "fountain" and the virtual model "ground".

[0088] The local coordinates of the virtual object may be the coordinates of the center point of the virtual object, or the coordinates of the maximum point of the virtual object in a certain direction, for example, the coordinates of the minimum point of the virtual object along the gravity direction.

[0089] Step S205: using the intersection point to adjust the second position information of the virtual object in the high-precision map space to achieve the placement of the virtual object in the virtual digital space.

[0090] The second position information of this embodiment may be the height of the virtual object relative to the ground plane in the high-precision map space. In other embodiments, the second position information may also be the relative height between the virtual object and other virtual models in the high-precision map space, such as the relative height between the virtual object and a roof model in the high-precision map space.

[0091] Specifically, this embodiment can be implemented by Figure 5 The method shown implements step S205. The method of this embodiment specifically includes step S501 and step S502.

[0092] Step S501: obtaining filtering information, wherein the filtering information includes preset information of intersections or preset information of virtual models.

[0093] From the above analysis, we can know that in the high-precision map space, the intersections of virtual objects and virtual models have the same local coordinates, and there are usually multiple virtual models in the high-precision map space that have the same local coordinates as the virtual objects along the gravity direction. For example, in the high-precision map space, the floor models of a multi-story building model are usually stacked along the gravity direction, so the virtual object has intersections with each floor model along the gravity direction; for another example, in the high-precision map space, the square model and the airplane model are stacked along the gravity direction, so the virtual object has intersections with both the square model and the airplane model along the gravity direction, and so on.

[0094] The preset information of the intersection in this embodiment may be the height of the intersection from the ground plane in the high-precision map space. In other embodiments, the preset information of the intersection may also be the relative height or relative height of the intersection and the marker point in the high-precision map space.

[0095] The preset information of the virtual model in this embodiment may include the height or mark information of the virtual model, etc. For example, the height of the first floor model in the multi-floor building model is A1, and the height of the second floor model is A2, or the mark information of the first floor model in the multi-floor building model is F1, and the mark information of the second floor model is F2, and so on.

[0096] Step S502: Filter multiple intersections using the filtering information, and adjust the second position information of the virtual object in the high-precision map space according to the filtered intersections.

[0097] Because there are usually multiple virtual models with the same local coordinates as the virtual object along the gravity direction in the high-precision map space, therefore, by screening the intersections through the preset information of the intersections or the preset information of the virtual models, the range of the placement of the virtual object in the high-precision map can be narrowed, the placement accuracy can be improved, and the virtual-real fusion effect can be increased.

[0098] Specifically, this embodiment can be implemented by Figure 6 The method shown implements step S502. The method of this embodiment specifically includes step S601 and step S602.

[0099] Step S601: Obtain a preset height.

[0100] Step S602: adjusting the distance between the virtual object and the ground plane along the gravity direction to a preset height.

[0101] In the high-precision map space, since both virtual objects and virtual models have a certain size, and the intersection of the virtual object and the virtual model is usually the intersection of the mark point on the virtual object and the mark point on the virtual model, the relative position of the virtual object and the virtual object along the gravity direction can be fine-tuned by adjusting the distance between the virtual object and the ground plane to a preset height, so as to improve the placement accuracy of the virtual object in the high-precision map space.

[0102] This embodiment can first use the above-mentioned preset information to screen multiple intersections between virtual objects and virtual models in the high-precision map space to obtain the intersections where the virtual objects need to be placed; then adjust the distance between the virtual objects and the ground plane of the high-precision map space along the direction of gravity to a preset height to achieve the final placement of the virtual objects in the high-precision map space, that is, the virtual digital space.

[0103] From the above analysis, it can be seen that in the high-precision map space, the above-mentioned method of screening intersections can determine at which intersection the virtual object is specifically placed, but the virtual object itself has a certain size. Therefore, it is necessary to further fine-tune the second position information of the virtual object in the high-precision map space to improve the accuracy of the placement of the virtual object in the virtual digital space. The virtual object has a position coordinate point. When locating the virtual object on the electronic map, the GPS information of the virtual object is the longitude and latitude coordinates of the position coordinate point. For example, when locating the virtual object "flower" on the electronic map, the GPS information of the virtual object "flower" is the longitude and latitude coordinates of the center point of the virtual object "flower". The local coordinates of the high-precision map space corresponding to the longitude and latitude coordinates are the local coordinates of the center point of the virtual object "flower"; after the intersection screening, it can be determined that the center point of the virtual object "flower" is the intersection of the virtual object "flower" and the virtual model "ground" in the high-precision map space. At this time, part of the virtual object "flower" is on the virtual model "ground", and the other part of the virtual object "flower" is under the virtual model "ground". At this time, the placement of the virtual object "flower" is unrealistic. Therefore, it is necessary to further adjust the height of the virtual object "flower" in the high-precision map space.

[0104] The preset height of this embodiment can be the height of the virtual object relative to the ground plane in the high-precision map space. In the high-precision map space, the ground plane is used as the reference plane for placing virtual objects, or the virtual object coordinates can be adjusted to match the ground plane, so it is easier to obtain the height of the ground plane.

[0105] In other embodiments, the preset height may be the height of the virtual object relative to other virtual models in the high-precision map space.

[0106] Furthermore, the present embodiment screens the intersections through a screening method to roughly determine the height of the virtual object in the virtual digital space, and can flexibly select the placement height of the virtual object in the virtual digital space; further, the height of the virtual object in the virtual digital space is adjusted by a preset height, which can improve the accuracy of the placement position of the virtual object in the virtual digital space.

[0107] This application further proposes a method for positioning a virtual object, such as Figure 7 As shown, Figure 7 1 is a flow chart of an embodiment of a method for locating a virtual object in this embodiment. The method for locating a virtual object in this embodiment includes the following steps:

[0108] Step S701: locate the virtual object on the electronic map of the area to be fused, and obtain the GPS information of the virtual object on the electronic map.

[0109] Step S702: Obtain the conversion relationship between the GPS coordinate system and the high-precision map coordinate system.

[0110] Step S703: transform the virtual object into a high-precision map space using the conversion relationship and GPS information.

[0111] Step S704: Obtain the intersection of the virtual object and the virtual model in the high-precision map space according to the GPS information.

[0112] Step S705: Use the intersection point to adjust the second position information of the virtual object in the high-precision map space to achieve the placement of the virtual object in the virtual digital space.

[0113] Steps S701 to S705 are similar to the above-mentioned steps S201 to S205 and are not described in detail here.

[0114] From the above analysis, it can be seen that by executing the above steps multiple times, multiple virtual objects can be placed in the virtual digital space, and these virtual objects can be placed in different sub-areas of the area to be merged in the virtual digital space. Therefore, this embodiment can realize the placement of virtual objects in a virtual digital space over a very large range.

[0115] From the above analysis, it can be seen that the above method can realize the placement of virtual objects in a virtual digital space of a very large range, and save the above information in a computer-readable storage medium for use in the subsequent virtual-reality fusion display process.

[0116] Step S706: Acquire the viewpoint and determine the area to be fused corresponding to the viewpoint position.

[0117] In the augmented reality process, the target point of visual positioning is the viewpoint, the viewpoint is positioned to obtain the position coordinates of the viewpoint, and the sub-area to be fused is obtained according to the position coordinates of the viewpoint.

[0118] The sub-region to be fused in this embodiment is a part of the above-mentioned region to be fused, and the sub-region to be fused includes the viewpoint.

[0119] Specifically, this embodiment can be implemented by Figure 8 The method shown implements step S706. The method of this embodiment specifically includes step S801 and step S802.

[0120] Step S801: Acquire a viewpoint, and acquire digital position coordinates corresponding to the position coordinates of the viewpoint in the virtual digital space.

[0121] Step S802: obtaining an area with the digital position coordinate as the center and a preset distance as the radius as the sub-area to be fused.

[0122] The preset distance may be a visually visible distance or a visible distance separately defined in the image material package.

[0123] Step S707: Acquire the virtual object in the sub-region to be fused.

[0124] The sub-region to be fused may include one, two or more virtual objects.

[0125] From the above analysis, it can be seen that the above method of this embodiment can realize the placement of virtual objects in an ultra-large-scale virtual digital space, that is, the placement position of each virtual object in the ultra-large-scale virtual digital space has been determined. Therefore, it is only necessary to determine the sub-region to be fused, and then the sub-region to be fused and the virtual objects whose coordinates are located in the sub-region to be fused, as well as the specific placement position of each virtual object in the sub-region to be fused, can be obtained from the computer-readable storage medium.

[0126] Step S708: Fusing the virtual object with the real scene of the sub-area to be fused for display.

[0127] The virtual and real areas are displayed or applied based on the regional coordinate system to facilitate users to understand the map structure or details.

[0128] In an application scenario, when displaying the area after the fusion of virtual and real, the editing operations described above can also be performed on the area after the fusion of virtual and real, so as to facilitate the user's understanding of the area after the fusion of virtual and real, and increase the user's freedom to operate the area after the fusion of virtual and real, thereby increasing the practicality of the virtual and real fusion technology.

[0129] Since high-precision maps collect map information through sensors and / or cameras, the virtual digital space they construct will include visual information such as map structure, as well as non-visual signals such as Bluetooth signals or WIFI signals. After the virtual-real fusion is performed in the area to be fused, the non-visual signals will also be fused. For example, when the area after the virtual-real fusion is displayed to the user, the user can learn the route to a destination in the area through the area after the virtual-real fusion, and also learn whether the destination is equipped with WIFI and other information through the area after the virtual-real fusion.

[0130] The application scenarios of this embodiment further include large-scale map navigation or information acquisition. For example, the technical solution provided by this embodiment can help users accurately obtain virtual digital space information in a large area (for example, the entire airport, factory, stadium, shopping mall, hospital and garage). Through these virtual digital space information and their own display devices, they can obtain various visual information and / or non-visual information such as tourist attraction introductions, flight information, factory IOT information, shopping mall marketing, hospital registration queues and car locations anytime and anywhere, helping users achieve various goals.

[0131] This application further proposes a device for fusion of virtuality and reality, such as Fig. 9 As shown, Fig. 9 It is a structural diagram of an embodiment of a positioning device for virtual objects of the present application. The positioning device 90 of the virtual object of the present embodiment includes: a positioning module 91, an acquisition module 92 and an adjustment module 93; wherein the positioning module 91 is used to position the virtual object on the area to be fused; the acquisition module 92 is coupled to the positioning module 91, and the acquisition module 92 is used to obtain the first position information of the virtual object on the area to be fused from the positioning module 91, and obtain the intersection of the virtual object and the virtual model in the area to be fused using the first position information; the adjustment module 93 is coupled to the acquisition module 92 and the positioning module 91, and the adjustment module 93 is used to adjust the second position information of the virtual object using the intersection to realize the placement of the virtual object in the virtual digital space.

[0132] The virtual object positioning device 90 of this embodiment is also used to implement the above-mentioned virtual object positioning method, which will not be described in detail here.

[0133] Different from the prior art, this embodiment can determine the first position information of the virtual object in the virtual digital space through the positioning module 91, can obtain the intersection of the virtual object and the virtual model in the area to be fused through the acquisition module 92, and can realize the placement of the virtual object in the virtual digital space through the adjustment module 93. The operation method is simple and can simplify the operation of virtual objects in the virtual-real fusion scene, thereby reducing the difficulty of virtual-real fusion and improving its convenience of use.

[0134] The present application further proposes an electronic device, such as Fig.10 As shown, Fig.10 The electronic device 80 of this embodiment includes a processor 81 , a memory 82 , an input / output device 83 and a bus 84 .

[0135] The processor 81 , the memory 82 , and the input / output device 83 are respectively connected to the bus 84 . The memory 82 stores program data, and the processor 81 is used to execute the program data to realize the positioning of the virtual object in the above embodiment.

[0136] In this embodiment, the processor 81 may also be referred to as a CPU (Central Processing Unit). The processor 81 may be an integrated circuit chip having signal processing capabilities. The processor 81 may also be a general-purpose processor, a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. A general-purpose processor may be a microprocessor or the processor 81 may also be any conventional processor, etc.

[0137] The present application further proposes a computer-readable storage medium, such as Fig.11 As shown, Fig.11 The computer-readable storage medium 101 stores program data 111 thereon, and the program data 111 is executed by a processor (not shown) to realize the positioning of the virtual object.

[0138] The computer-readable storage medium 101 of this embodiment may be, but is not limited to, a USB flash drive, an SD card, a PD optical drive, a mobile hard disk, a large-capacity floppy drive, a flash memory, a multimedia memory card, a server, and the like.

[0139] Different from the prior art, the embodiment of the present application first positions the virtual object on the area to be fused, and obtains the first position information of the virtual object on the area to be fused; then uses the first position information to obtain the intersection of the virtual object and the virtual model in the area to be fused; and finally uses the intersection to adjust the second position information of the virtual object to achieve the placement of the virtual object in the virtual digital space. The present application positions the virtual object on the area to be fused, determines the first position information of the virtual object in the virtual digital space, determines the second position information of the virtual object in the virtual digital space through the intersection of the virtual object and the virtual model in the area to be fused, and finally determines the placement position of the virtual object in the virtual digital space based on the first position information and the second position information, thereby achieving the placement of the virtual object in the virtual digital space. The operation method is simple and can simplify the operation of virtual objects in the virtual-reality fusion scene, thereby reducing the difficulty of virtual-reality fusion and improving its convenience of use.

[0140] The present application discloses the field of augmented reality, which obtains the image information of the target object in the real environment, and then uses various vision-related algorithms to detect or identify the relevant features, states and attributes of the target object, so as to obtain an AR effect that combines virtuality and reality and matches the specific application. Exemplarily, the target object may involve the face, limbs, gestures, actions, etc. related to the human body, or markers and signs related to objects, or sand tables, display areas or display items related to venues or places. Vision-related algorithms may involve visual positioning, SLAM, 3D reconstruction, image registration, background segmentation, key point extraction and tracking of objects, and position or depth detection of objects. Specific applications may not only involve interactive scenes such as guided tours, navigation, explanations, reconstruction, virtual effect superposition and display related to real scenes or objects, but also special effects processing related to people, such as makeup beautification, limb beautification, special effects display, virtual model display and other interactive scenes.

[0141] The convolutional neural network can be used to detect or identify the relevant features, states and attributes of the target object. The above convolutional neural network is a network model obtained by model training based on a deep learning framework.

[0142] In addition, if the above functions are implemented in the form of software functions and sold or used as independent products, they can be stored in a storage medium readable by a mobile terminal, that is, the present application also provides a storage device storing program data, the program data can be executed to implement the method of the above embodiment, and the storage device can be, for example, a USB flash drive, an optical disk, a server, etc. In other words, the present application can be embodied in the form of a software product, which includes a number of instructions for enabling a smart terminal to execute all or part of the steps of the method described in each embodiment.

[0143] In the description of the present application, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradicting each other.

[0144] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0145] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present application includes alternative implementations in which functions may not be performed in the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by technicians in the technical field to which the embodiments of the present application belong.

[0146] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (which can be a personal computer, server, network device or other system that can fetch instructions from the instruction execution system, device or apparatus and execute the instructions), or in combination with these instruction execution systems, devices or apparatuses. For the purpose of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus or in combination with these instruction execution systems, devices or apparatuses. More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk box (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), a fiber optic device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or processing in other suitable ways if necessary, and then stored in a computer memory.

[0147] The above description is only an implementation method of the present application, and does not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly used in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A method for positioning a virtual object, characterized in that: The virtual object positioning method comprises: Positioning the virtual object on the area to be fused, and acquiring first position information of the virtual object on the area to be fused, wherein the first position information includes GPS information; Acquire an intersection point between the virtual object and the virtual model in the area to be fused using the first position information; Using the intersection point to adjust the second position information of the virtual object to achieve placement of the virtual object in the virtual digital space; The obtaining the intersection of the virtual object and the virtual model in the area to be fused by using the first position information comprises: Acquire the intersection of the virtual object and the virtual model in the high-precision map space according to the GPS information; The GPS information includes longitude and latitude coordinates; The step of obtaining the intersection of the virtual object and the virtual model in the high-precision map space according to the GPS information includes: Determining the gravity direction of the virtual object in the high-precision map space; Acquire the local coordinates of the virtual object in the high-precision map space, and acquire the intersection of a straight line along the gravity direction and passing through the local coordinates and the virtual model in the high-precision map space; Before using the first position information to obtain the intersection of the virtual object and the virtual model in the area to be fused, the method for locating the virtual object further includes: Obtain the conversion relationship between the GPS coordinate system and the high-precision map coordinate system; The virtual object is transformed into a high-precision map space using the conversion relationship and the GPS information.

2. The method for positioning a virtual object according to claim 1, characterized in that: The adjusting the second position information of the virtual object by using the intersection point comprises: Acquire screening information; wherein the screening information includes preset information of the intersection or preset information of the virtual model; The filtering information is used to filter the plurality of intersections, and the second position information of the virtual object in the high-precision map space is adjusted according to the filtered intersections.

3. The method for positioning a virtual object according to claim 2, characterized in that: The second position information includes the height of the virtual object relative to the ground plane in the high-precision map space, and adjusting the second position information of the virtual object in the high-precision map space according to the filtered intersection further includes: Get the preset height; The distance between the virtual object and the ground plane along the gravity direction is adjusted to the preset height.

4. The method for positioning a virtual object according to claim 1, characterized in that: The acquisition of the conversion relationship between the GPS coordinate system and the high-precision map coordinate system includes: Dividing the area to be fused into multiple sub-areas; Obtaining a high-precision map of each of the sub-areas respectively, and obtaining a high-precision map coordinate system of each of the sub-areas; Obtain a transformation matrix between the GPS coordinate system of the area to be fused and the high-precision map coordinate system of each sub-area.

5. The method for positioning a virtual object according to any one of claims 1 to 4, characterized in that: The virtual object positioning method further comprises: Acquire a viewpoint, and determine a sub-region to be fused corresponding to the viewpoint; Acquire the virtual object in the sub-region to be fused; The virtual object is fused with the real scene of the sub-area to be fused and displayed.

6. The method for positioning a virtual object according to claim 5, characterized in that: The determining of the sub-region to be fused corresponding to the viewpoint comprises: Acquire digital position coordinates corresponding to the position coordinates of the viewpoint in the virtual digital space; An area with the digital position coordinate as the center and a preset distance as the radius is obtained as the sub-area to be fused.

7. A virtual object positioning device, characterized in that: The positioning device of the virtual object comprises: A positioning module, used to position the virtual object on the area to be fused; an acquisition module, coupled to the positioning module, and configured to acquire first position information of the virtual object on the to-be-fused area from the positioning module, wherein the first position information includes GPS information; A calculation module, coupled to the acquisition module, and configured to acquire an intersection point between the virtual object and the virtual model in the area to be fused by using the first position information; an adjustment module, coupled to the calculation module and the positioning module, and configured to adjust the second position information of the virtual object using the intersection point to achieve placement of the virtual object in the virtual digital space; The obtaining the intersection of the virtual object and the virtual model in the area to be fused by using the first position information comprises: Acquire the intersection of the virtual object and the virtual model in the high-precision map space according to the GPS information; The GPS information includes longitude and latitude coordinates; The obtaining of the intersection of the virtual object and the virtual model in the high-precision map space according to the GPS information includes: determining the gravity direction of the virtual object in the high-precision map space; obtaining the local coordinates of the virtual object in the high-precision map space, and obtaining the intersection of a straight line along the gravity direction and passing through the local coordinates and the virtual model in the high-precision map space; Before using the first position information to obtain the intersection of the virtual object and the virtual model in the area to be fused, the method for locating the virtual object further includes: Obtain the conversion relationship between the GPS coordinate system and the high-precision map coordinate system; The virtual object is transformed into a high-precision map space using the conversion relationship and the GPS information.

8. An electronic device, characterized in that: The electronic device comprises: a memory and a processor coupled to each other, wherein the processor is used to execute program data stored in the memory to implement the virtual object positioning method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores program data, and the program data can be executed to implement the virtual object positioning method according to any one of claims 1 to 6.

Images