Augmented reality registration method and device for outdoor large scene
By using augmented reality devices to register three-dimensional virtual and real scenes in large outdoor scenes, the problem that the existing technology is difficult to adapt to the outdoor environment is solved, and high-precision virtual reality fusion is achieved, which improves immersion and training efficiency.
Patent Information
- Application Number
- CN202411125311.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-16
- Publication Date
- 2025-01-10
AI Technical Summary
The existing augmented reality registration method based on binocular vision systems is difficult to apply to large outdoor scenarios, and it is impossible to effectively realize accurate registration of virtual environments and real world.
By wearing an augmented reality device on the user's head, using a translucent AR glasses observation environment, recording the coordinates, rotations and viewing angle offsets between the three-dimensional real perspective scene and the virtual scene, and making corresponding adjustments to achieve registration, and based on The real coordinates of outdoor sign locations and the distance between the user's eyes and the ground, register the real and virtual world coordinate systems.
It realizes accurate registration of virtual and real environments in large outdoor scenes, improves users' immersion and training efficiency, and can simulate disaster scenarios or other special environments in actual buildings, providing low-cost training support.
Smart Images
Figure CN119295705A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of computer vision technology, and in particular to an augmented reality registration method and device for a large outdoor scene. Background Art
[0002] Outdoor training, drills, or navigation are important application areas of mixed (augmented) reality technology. Such applications require users to immerse themselves in the surrounding environment and interact with objects and buildings in the environment. The prerequisite is to achieve registration between the virtual environment and the real world environment.
[0003] The existing registration methods are mainly: registration based on the binocular vision system, but the registration based on the binocular vision system is only applicable to a smaller distance range and is difficult to apply to large outdoor scenes. Summary of the invention
[0004] The present invention provides an augmented reality registration method and device for a large outdoor scene, so as to solve the defect that it is difficult to perform augmented reality registration in a large outdoor scene in the prior art, and realize augmented reality registration in a large outdoor scene.
[0005] The present invention provides an augmented reality registration method for a large outdoor scene, comprising: A three-dimensional virtual scene of the environment is displayed to the user according to the proportion of the three-dimensional real scene in the real world; the user wears an augmented reality device on his head; The three-dimensional real perspective scene is registered with the three-dimensional virtual scene; the three-dimensional real perspective scene is obtained by the user observing the environment through the semi-transparent AR glasses of the augmented reality device.
[0006] In some embodiments, registering the three-dimensional real perspective scene with the three-dimensional virtual scene includes: Recording the coordinate offset, rotation offset and viewing angle offset between the three-dimensional real perspective scene and the three-dimensional virtual scene; Based on the coordinate offset, the rotation offset and the viewing angle offset, the three-dimensional coordinates, direction angle, pitch angle, roll angle and viewing angle of the three-dimensional virtual scene are adjusted to align the three-dimensional real perspective scene with the three-dimensional virtual scene.
[0007] In some embodiments, the real world coordinate system and the virtual world coordinate system are aligned based on the real coordinates of a landmark location in a large outdoor scene and the distance from the user's eyes to the ground; the user stands still at the landmark location.
[0008] In some embodiments, the method further comprises: Obtaining the real initial posture of the user's head; Based on the real initial head posture and the user's head-sensed initial posture sensed by the sensor in the augmented reality device, the sensor in the augmented reality device is aligned.
[0009] In some embodiments, the sensors in the augmented reality device include: a positioning sensor, a magnetic compass, an acceleration sensor, and a gyroscope.
[0010] The present invention also provides an augmented reality registration device for an outdoor large scene, comprising: A display module is used to display a three-dimensional virtual scene of the user's environment to the user according to the proportion of the three-dimensional real scene in the real world; the user wears an augmented reality device on his head; The first registration module is used to register the three-dimensional real perspective scene with the three-dimensional virtual scene; the three-dimensional real perspective scene is obtained by the user observing the environment through the semi-transparent AR glasses of the augmented reality device.
[0011] In some embodiments, the apparatus further comprises: The second registration module is used to register the real world coordinate system and the virtual world coordinate system based on the real coordinates of the landmark location in the outdoor large scene and the distance from the user's eyes to the ground; the user stands still at the landmark location.
[0012] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, an augmented reality registration method for a large outdoor scene as described in any one of the above is implemented.
[0013] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the augmented reality registration method for a large outdoor scene as described in any one of the above is implemented.
[0014] The present invention also provides a computer program product, including a computer program, wherein when the computer program is executed by a processor, the augmented reality registration method for an outdoor large scene as described in any one of the above is implemented.
[0015] The augmented reality registration method and device for large outdoor scenes provided by the present invention display a three-dimensional virtual scene of the user's environment according to the proportion of the three-dimensional real scene in the real world, and utilize the user to observe the environment through translucent AR glasses to obtain a three-dimensional real perspective scene, and register the three-dimensional real perspective scene with the three-dimensional virtual scene to achieve augmented reality registration of large outdoor scenes and enhance the user's sense of immersion. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.
[0017] Figure 1 It is a flow chart of the augmented reality registration method for a large outdoor scene provided by the present invention; Figure 2 It is a structural schematic diagram of an augmented reality registration device for a large outdoor scene provided by the present invention; Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION
[0018] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.
[0019] Figure 1 is a flow chart of the augmented reality registration method for a large outdoor scene provided by the present invention, such as Figure 1 As shown, the present invention provides an augmented reality registration method for a large outdoor scene, comprising: Step 110, showing the user a three-dimensional virtual scene of the environment according to the proportion of the three-dimensional real scene in the real world; the user wears an augmented reality device on his head.
[0020] Specifically, the user wears an augmented reality device on his head, and the augmented reality device displays the three-dimensional virtual scene of the user's environment according to the proportion of the three-dimensional real scene in the real world. That is, the displayed three-dimensional virtual scene is the same size as the three-dimensional real scene of the user's environment, so as to enhance the user's sense of immersion.
[0021] Step 120, aligning the three-dimensional real perspective scene with the three-dimensional virtual scene; the three-dimensional real perspective scene is obtained by the user observing the environment through the semi-transparent AR glasses of the augmented reality device.
[0022] Specifically, since the three-dimensional real scene in the real world is different from the three-dimensional real perspective scene seen by human eyes, in order to further enhance the user's immersion, the registration object should be the three-dimensional real perspective scene and the three-dimensional virtual scene.
[0023] The AR glasses of the augmented reality device are set to be translucent so that the user can observe the environment through the translucent AR glasses to obtain a three-dimensional real perspective scene while seeing the three-dimensional virtual scene, so that the three-dimensional real perspective scene and the three-dimensional virtual scene can be aligned, even if the three-dimensional real perspective scene and the three-dimensional virtual scene overlap.
[0024] The augmented reality registration method for large outdoor scenes provided by the present invention displays a three-dimensional virtual scene of the user's environment according to the proportion of the three-dimensional real scene in the real world, enables the user to observe the environment through translucent AR glasses to obtain a three-dimensional real perspective scene, and registers the three-dimensional real perspective scene with the three-dimensional virtual scene, thereby realizing augmented reality registration of large outdoor scenes and enhancing the user's sense of immersion.
[0025] In some embodiments, registering the three-dimensional real perspective scene with the three-dimensional virtual scene includes: Recording the coordinate offset, rotation offset and viewing angle offset between the three-dimensional real perspective scene and the three-dimensional virtual scene; Based on the coordinate offset, rotation offset and viewing angle offset, the three-dimensional coordinates, direction angle, pitch angle, roll angle and viewing angle of the three-dimensional virtual scene are adjusted to align the three-dimensional real perspective scene with the three-dimensional virtual scene.
[0026] Specifically, the coordinate offset, rotation offset and viewing angle offset between the 3D real perspective scene and the 3D virtual scene are recorded. The 3D coordinates (i.e., XYZ coordinates) of the 3D virtual scene are adjusted based on the coordinate offset, and the 3D coordinates of the 3D virtual scene are adjusted through virtual buttons. The azimuth, pitch angle and roll angle of the 3D virtual scene are adjusted based on the rotation offset, and the azimuth, pitch angle and roll angle of the 3D virtual scene are adjusted through virtual buttons. The viewing angle of the 3D virtual scene is adjusted based on the viewing angle offset, and the viewing angle of the 3D virtual scene is adjusted by adjusting the focal length of the camera of the augmented reality device, thereby realizing the registration of the 3D real perspective scene with the 3D virtual scene.
[0027] The augmented reality registration method for a large outdoor scene provided by the present invention adjusts the three-dimensional coordinates, direction angle, pitch angle, roll angle and viewing angle of the three-dimensional virtual scene based on the coordinate offset, rotation offset and viewing angle offset between the three-dimensional real perspective scene and the three-dimensional virtual scene, thereby improving the registration accuracy between the three-dimensional real perspective scene and the three-dimensional virtual scene.
[0028] In some embodiments, the augmented reality registration method for a large outdoor scene provided by the present invention further includes: Based on the real coordinates of the landmark location in the outdoor large scene and the distance from the user's eyes to the ground, the real world coordinate system and the virtual world coordinate system are aligned; the user stands still at the landmark location.
[0029] Specifically, before displaying the 3D virtual scene of the user's environment, in order to correctly display the 3D virtual scene (that is, the displayed 3D virtual scene is consistent with the 3D real scene), the real world coordinate system and the virtual world coordinate system need to be aligned.
[0030] In a large outdoor scene, a landmark with good vision, easy access, unobstructed sky, and specific marks on the ground is selected, and the user wears an augmented reality device and stands still at the landmark. The real coordinates of the landmark and the distance from the user's eyes to the ground are obtained, and the real world coordinate system and the virtual world coordinate system are aligned according to the real coordinates of the landmark and the distance from the user's eyes to the ground, that is, the distance from the user's eyes to the ground is used to ensure that the user's perspective in the real world and the virtual world is consistent, and the virtual world coordinate system is aligned according to the deviation between the coordinates of the landmark in the virtual world coordinate system and the real coordinates of the landmark, so that the real world coordinate system is consistent with the virtual world coordinate system.
[0031] The augmented reality registration method for a large outdoor scene provided by the present invention aligns the real-world coordinate system with the virtual-world coordinate system based on the real coordinates of the landmark locations in the large outdoor scene and the distance from the user's eyes to the ground, which is conducive to the subsequent correct display of the three-dimensional virtual scene to the user and enhances the user's sense of immersion.
[0032] In some embodiments, the augmented reality registration method for a large outdoor scene provided by the present invention further includes: Obtain the user's actual initial head posture; Based on the real initial head posture and the user's head-sensed initial posture sensed by the sensor in the augmented reality device, the sensor in the augmented reality device is aligned.
[0033] Specifically, in order to ensure that the three-dimensional real perspective scene and the three-dimensional virtual scene are always aligned when the user is active in a large outdoor scene, it is necessary to ensure that the user's real posture is consistent with the user's sensed posture sensed by the sensor in the augmented reality device, that is, the sensor in the augmented reality device needs to be calibrated.
[0034] The real initial head posture of the user and the sensed initial head posture of the user sensed by the sensor in the augmented reality device are obtained, and based on the difference between the real initial head posture and the sensed initial head posture, the sensor in the augmented reality device is calibrated to make the real initial head posture consistent with the sensed initial head posture.
[0035] The process of acquiring the user's real initial head posture can be: the user is still or places the augmented reality device still in an open area, and the posture is obtained through a combination of measurements of sensors such as the Global Navigation Satellite System (GNSS) and accelerometers.
[0036] The augmented reality registration method for a large outdoor scene provided by the present invention aligns the sensors in the augmented reality device based on the real initial posture of the head and the initial posture of the head sensing, so that when the user is active in the large outdoor scene, the three-dimensional real perspective scene and the three-dimensional virtual scene are always aligned.
[0037] In some embodiments, the sensors in the augmented reality device include: a positioning sensor, a magnetic compass, an acceleration sensor, and a gyroscope. The positioning sensor may be a Beidou positioning sensor or a GPS positioning sensor.
[0038] In some embodiments, adding a lidar to the augmented reality device can also perform scene matching, but to use it in open areas, the lidar needs to have high power and a certain scanning range, which is costly and consumes a lot of power.
[0039] Based on the augmented reality registration method for large outdoor scenes provided by the present invention, it is possible to superimpose fire, flooding or other destructive disaster scenes on the basis of actual buildings and facilities, or simulate consumer vehicles, helicopters, professional equipment, etc. in the scenes, so as to simulate more realistic accident handling and training scenes at a relatively low cost, thereby improving training efficiency or providing support for decision-making.
[0040] The augmented reality registration device for a large outdoor scene provided by the present invention is described below. The augmented reality registration device for a large outdoor scene described below and the augmented reality registration method for a large outdoor scene described above can correspond to each other.
[0041] Figure 2 is a schematic diagram of the structure of the augmented reality registration device for outdoor large scenes provided by the present invention, such as Figure 2 As shown, the present invention provides an augmented reality registration device for an outdoor large scene, comprising: The display module 210 is used to display the three-dimensional virtual scene of the user's environment to the user according to the proportion of the three-dimensional real scene in the real world; the user wears an augmented reality device on his head; The first registration module 220 is used to register the three-dimensional real perspective scene with the three-dimensional virtual scene; the three-dimensional real perspective scene is obtained by the user observing the environment through the semi-transparent AR glasses of the augmented reality device.
[0042] In some embodiments, the first registration module 220 is specifically configured to: Recording the coordinate offset, rotation offset and viewing angle offset between the three-dimensional real perspective scene and the three-dimensional virtual scene; Based on the coordinate offset, the rotation offset and the viewing angle offset, the three-dimensional coordinates, direction angle, pitch angle, roll angle and viewing angle of the three-dimensional virtual scene are adjusted to align the three-dimensional real perspective scene with the three-dimensional virtual scene.
[0043] In some embodiments, the apparatus further comprises: The second registration module is used to register the real world coordinate system and the virtual world coordinate system based on the real coordinates of the landmark location in the outdoor large scene and the distance from the user's eyes to the ground; the user stands still at the landmark location.
[0044] In some embodiments, the apparatus further comprises: An acquisition module, used to acquire the real initial posture of the user's head; The third registration module is used to register the sensor in the augmented reality device based on the real initial head posture and the user's head-sensed initial posture sensed by the sensor in the augmented reality device.
[0045] In some embodiments, the sensors in the augmented reality device include: a positioning sensor, a magnetic compass, an acceleration sensor, and a gyroscope.
[0046] Figure 3 is a schematic diagram of the structure of the electronic device provided by the present invention, such as Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the augmented reality registration method for a large outdoor scene, the method comprising: showing a three-dimensional virtual scene of the environment to the user according to the proportion of the three-dimensional real scene in the real world; the user wears an augmented reality device on his head; registering the three-dimensional real perspective scene with the three-dimensional virtual scene; the three-dimensional real perspective scene is obtained by the user observing the environment through the semi-transparent AR glasses of the augmented reality device.
[0047] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of a software functional unit and can be stored in a computer-readable storage medium when it is sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc. Various media that can store program codes.
[0048] On the other hand, the present invention also provides a computer program product, which includes a computer program, and the computer program can be stored on a non-transitory computer-readable storage medium. When the computer program is executed by a processor, the computer can execute the augmented reality alignment method for large outdoor scenes provided by the above methods, and the method includes: displaying a three-dimensional virtual scene of the environment to a user according to the proportion of the three-dimensional real scene in the real world; the user wears an augmented reality device on his head; and aligns the three-dimensional real perspective scene with the three-dimensional virtual scene; the three-dimensional real perspective scene is obtained by the user observing the environment through the translucent AR glasses of the augmented reality device.
[0049] On the other hand, the present invention also provides a non-transitory computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by the processor, it is implemented to execute the augmented reality registration method of the outdoor large scene provided by the above methods, the method comprising: showing the user a three-dimensional virtual scene of the environment according to the proportion of the three-dimensional real scene in the real world; the user wears an augmented reality device on his head; aligning the three-dimensional real perspective scene with the three-dimensional virtual scene; the three-dimensional real perspective scene is obtained by the user observing the environment through the semi-transparent AR glasses of the augmented reality device. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without creative labor.
[0050] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.
[0051] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.
Claims
1. An augmented reality registration method for a large outdoor scene, characterized in that: include: Display the 3D virtual scene of the environment to the user according to the proportion of the 3D real scene in the real world; The user wears an augmented reality device on his head; Registering the three-dimensional real perspective scene with the three-dimensional virtual scene; The three-dimensional real perspective scene is obtained by the user observing the environment through the semi-transparent AR glasses of the augmented reality device.
2. The augmented reality registration method for outdoor large scenes according to claim 1, characterized in that: The registering the three-dimensional real perspective scene with the three-dimensional virtual scene includes: Recording the coordinate offset, rotation offset and viewing angle offset between the three-dimensional real perspective scene and the three-dimensional virtual scene; Based on the coordinate offset, the rotation offset and the viewing angle offset, the three-dimensional coordinates, direction angle, pitch angle, roll angle and viewing angle of the three-dimensional virtual scene are adjusted to align the three-dimensional real perspective scene with the three-dimensional virtual scene.
3. The augmented reality registration method for outdoor large scenes according to claim 1, characterized in that: The method further comprises: Based on the real coordinates of the marked location in the outdoor large scene and the distance from the user's eyes to the ground, the real world coordinate system and the virtual world coordinate system are aligned; the user stands still on the marked location.
4. The augmented reality registration method for outdoor large scenes according to claim 3, characterized in that: The method further comprises: Obtaining the real initial posture of the user's head; Based on the real initial head posture and the user's head-sensed initial posture sensed by the sensor in the augmented reality device, the sensor in the augmented reality device is aligned.
5. The augmented reality registration method for outdoor large scenes according to claim 4, characterized in that: The sensors in the augmented reality device include: a positioning sensor, a magnetic compass, an acceleration sensor and a gyroscope.
6. An augmented reality registration device for outdoor large scenes, characterized in that: include: A display module is used to display a three-dimensional virtual scene of the user's environment according to the proportion of the three-dimensional real scene in the real world; The user wears an augmented reality device on his head; A first registration module, used for registering the three-dimensional real perspective scene with the three-dimensional virtual scene; The three-dimensional real perspective scene is obtained by the user observing the environment through the semi-transparent AR glasses of the augmented reality device.
7. The augmented reality registration device for outdoor large scenes according to claim 6, characterized in that: The device also includes: The second registration module is used to register the real world coordinate system and the virtual world coordinate system based on the real coordinates of the landmark location in the outdoor large scene and the distance from the user's eyes to the ground; the user stands still at the landmark location.
8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the computer program, the augmented reality registration method for a large outdoor scene as described in any one of claims 1 to 5 is implemented.
9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the augmented reality registration method for a large outdoor scene as claimed in any one of claims 1 to 5 is implemented.
10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the augmented reality registration method for an outdoor large scene as claimed in any one of claims 1 to 5 is implemented.