A method and system for displaying objects in an augmented reality environment

Abstract

An embodiment of the present invention provides a method and system for displaying objects in an augmented reality environment, characterized by comprising the following steps: an augmented reality (AR) device obtains spatial information and an object identifier of an anchor device; the AR device obtains corresponding object parameters from an anchor cloud based on the object identifier, the object parameters including an object image, object audio, and a first spatial relationship of the object relative to the anchor device; the AR device obtains a second spatial relationship relative to the anchor device based on the spatial information; obtains a third spatial relationship of the object relative to the AR device based on the first and second spatial relationships; and the AR device displays the object image and object audio based on the third spatial relationship. The present invention actively provides spatial information through the anchor device, and the AR device analyzes the information, eliminating the traditional need to continuously use a camera to analyze images, which consumes a lot of computing power and energy, and is susceptible to image quality affecting the analysis results.

Description

Technical Field

[0001] The present invention relates to augmented reality technology, and more particularly to a method and system for displaying objects in an augmented reality environment. Background Art

[0002] In existing augmented reality (AR) displays of objects, users must move their cameras to scan physical anchor points, then download object information based on the physical anchor points' features. This requires continuous calculation of camera image information to confirm the spatial relationship between the anchor points and the object, consuming significant computing power and electricity. Summary of the Invention

[0003] In view of this, the present invention provides a method for displaying objects in an augmented reality environment. An anchor device is used to actively broadcast spatial information and related information about objects in the augmented reality environment. When the user being shared passes by the anchor device, the user can receive the spatial information of the object in the augmented reality and display the object in the augmented reality without scanning or any other operation.

[0004] An embodiment of the present invention provides a method for displaying objects in an augmented reality environment, the method comprising the following steps: an augmented reality (AR) device obtains spatial information and an object identifier of an anchor device; the AR device obtains object parameters corresponding to the object identifier from an anchor cloud based on the object identifier, the object parameters including an object image, object audio, and a first spatial relationship of the object relative to the anchor device; the AR device obtains a second spatial relationship relative to the anchor device based on the spatial information; obtains a third spatial relationship of the object relative to the AR device based on the first and second spatial relationships; and the AR device displays the object image and object audio in the third spatial relationship.

[0005] An embodiment of the present invention also provides a system for displaying objects in an augmented reality environment, including: an augmented reality (AR) device, obtaining spatial information and an object identifier from an anchor device, the AR device obtaining object parameters corresponding to the object identifier from an anchor cloud based on the object identifier, the object parameters including an object image, object audio, and a first spatial relationship of the object relative to the anchor device, obtaining a second spatial relationship of the AR device relative to the anchor device based on the spatial information, obtaining object parameters from the anchor cloud based on the object identifier, obtaining a third spatial relationship of the object relative to the AR device based on the first spatial relationship and the second spatial relationship, and the AR device displaying the object image and object audio in the third spatial relationship.

[0006] The present invention will be described in detail below with reference to the accompanying drawings and specific embodiments, but this does not limit the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 A flow chart showing a method for displaying objects in an augmented reality environment according to an embodiment of the present invention.

[0008] Figure 2 A diagram showing the architecture of a display system for objects in an augmented reality environment according to an embodiment of the present invention.

[0009] Figure 3 Schematic diagrams showing objects at different positions in an augmented reality environment according to an embodiment of the present invention.

[0010] Figure 4 A schematic diagram showing a plurality of objects displayed in an augmented reality environment according to an embodiment of the present invention.

[0011] Figure 5 A schematic diagram showing the display of multiple objects in an augmented reality environment according to another embodiment of the present invention.

[0012] Figure 6 A schematic diagram showing a one-way application scenario of the method for displaying objects in an augmented reality environment according to an embodiment of the present invention.

[0013] Figure 7 A schematic diagram showing a two-way application scenario of the method for displaying objects in an augmented reality environment according to an embodiment of the present invention.

[0014] Description of main component symbols

[0015]

[0016]

[0017] The following specific embodiments will further illustrate the present invention in conjunction with the above-mentioned drawings. DETAILED DESCRIPTION

[0018] In order to facilitate those skilled in the art to understand and implement the present invention, the present invention is further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the embodiments described herein are only used to illustrate and explain the present invention and are not used to limit the present invention.

[0019] Figure 1 A flow chart showing a method for displaying objects in an augmented reality environment according to an embodiment of the present invention.

[0020] In step S101 , in an augmented reality environment, an anchor device continuously sends spatial information and object identifiers. When a user wears an augmented reality (AR) device and enters the AR environment, the user can obtain the spatial information and object identifiers from the anchor device.

[0021] In step S102, the AR device obtains the object parameters corresponding to the object identifier from the anchor cloud based on the object identifier obtained from the anchor device. The object parameters include the object's image information, object audio, and a first spatial relationship of the object relative to the anchor device. The image information includes the object's shape, color, size, and orientation, and the first spatial relationship includes the object's position, distance, and horizontal angle relative to the anchor device.

[0022] In step S103, the AR device obtains a second spatial relationship of the AR device relative to the anchor device based on the spatial information. The second spatial relationship includes the position, distance, and horizontal angle of the AR device relative to the anchor device. The spatial information includes the position, horizontal angle, a first spatial information packet sent by the anchor device via a first communication technology, and a second spatial information packet sent by the anchor device via a second communication technology. The first spatial information packet and the second spatial information packet are sent simultaneously. The distance between the AR device and the anchor device is calculated based on the known transmission rates of the first communication technology and the known second communication technology, and the time difference between the AR device receiving the first spatial information packet and the second spatial information packet. This distance is the distance in the second spatial relationship, such as disclosed in Chinese Patent Application No. 202110603549.X, entitled "A Method and System for Establishing an Augmented Reality Anchor."

[0023] In step S104, the anchor device is used as the origin in the augmented reality environment. The first position of the object in the augmented reality environment is obtained based on the first spatial relationship. The second position of the AR device in the augmented reality environment is obtained based on the second spatial relationship. Therefore, the third spatial relationship of the object relative to the AR device is calculated based on the first and second positions. The third spatial relationship includes the object's position, distance, and horizontal angle relative to the AR device.

[0024] In step S105 , the AR device displays the object based on the image information and the audio of the object, and adjusts the size of the object image, the volume and directionality of the object audio according to the distance in the third spatial relationship to display the object, thereby creating a more realistic spatial sense of the object in the augmented reality environment.

[0025] Figure 2 The following is a diagram showing the architecture of the display system for objects in an augmented reality environment according to an embodiment of the present invention. Figure 2As shown, the display system includes an AR device 210, an anchor device 220, an anchor cloud 230, and an object 240. The AR device 210 obtains spatial information and an object identifier of the object 240 from the anchor device 220. Based on the object identifier, the AR device 210 obtains object parameters of the object 240 from the anchor cloud 230. The anchor cloud 230 is used to store and provide object parameters related to the object 240. The object parameters include object images, object audio, and a first spatial relationship of the object 240 relative to the anchor device 220. The first spatial relationship includes the position, distance 250, and horizontal angle 251 of the object 240 relative to the anchor device 220.

[0026] A second spatial relationship of the AR device 210 relative to the anchor device 220 is obtained based on the spatial information provided by the anchor device 220. The second spatial relationship includes the position, distance 260, and horizontal angle 261 of the AR device 210 relative to the anchor device 220. The spatial information includes the position and horizontal angle 261 of the AR device 210 relative to the anchor device 220, a first spatial information packet sent by the anchor device 220 via a first communication technology, and a second spatial information packet sent by the anchor device 220 via a second communication technology. The first and second spatial information packets are sent simultaneously. The distance 260 between the AR device 210 and the anchor device 230 is calculated based on the known propagation rates of the first and second communication technologies and the time difference between the AR device 210 receiving the first and second spatial information packets. This distance 260 is the distance 260 in the second spatial relationship.

[0027] The AR device 210 obtains object parameters of the object 240 from the anchor cloud 230 based on the object identifier obtained from the anchor device 220. With the anchor device 220 as the origin in the augmented reality environment, the first position of the object 240 in the augmented reality environment is obtained based on the first spatial relationship, and the second position of the AR device 210 in the augmented reality environment is obtained based on the second spatial relationship. Therefore, a third spatial relationship of the object 240 relative to the AR device 210 is calculated based on the first and second positions. The third spatial relationship includes the position, distance 270, and horizontal angle 271 of the object 240 relative to the AR device 210. The AR device 210 displays the object image and object audio based on the third spatial relationship.

[0028] The AR device 210 further includes an inertial measurement unit for obtaining a moving direction, a moving distance, and a moving height of the AR device 210. The AR device 210 updates the second spatial relationship and the second position in real time based on the moving direction, moving distance, and moving height obtained by the inertial measurement unit, and updates the third spatial relationship based on the first position and the updated second position, so that the AR device 210 can display object images and object audio based on the updated third spatial relationship.

[0029] Figure 3 A schematic diagram illustrating objects displayed at different locations in an augmented reality environment according to an embodiment of the present invention is shown. In one embodiment, an AR device 300 includes a first speaker 301, a second speaker 302, and a display unit 303. When a user wears the AR device 300, the first speaker 301 is positioned near the user's left ear, and the second speaker 302 is positioned near the user's right ear. When the AR device 300 displays an object, the output of the first speaker 301, the second speaker 302, and the display unit 303 are adjusted based on the third spatial relationship.

[0030] like Figure 3 As shown, the AR device 300 obtains the position, distance, and horizontal angle of the object relative to the AR device 300 based on the third spatial relationship. Taking the object at position 311 directly in front of the AR device 300 as an example, the display unit 303 displays the object image at position 311, and adjusts the object image size and object audio based on the distance and horizontal angle in the third spatial relationship. The first speaker 301 and the second speaker 302 both emit object audio, and the object audio is output at the same time and volume by the first speaker 301 and the second speaker 302.

[0031] Taking the example of the AR device 300 displaying an object at position 312 based on the third spatial relationship, the display unit 303 displays the object image at position 312, and adjusts the object image size and object audio based on the distance and horizontal angle in the third spatial relationship. The first speaker 301 outputs the object audio before the second speaker 302, and the output volume of the first speaker 301 is also greater than that of the second speaker 302.

[0032] Taking the example of the AR device 300 displaying an object at position 313 based on the third spatial relationship, since the object is located on the left side of the AR device 300, the display unit 303 does not display the object image. The object audio is adjusted based on the distance and horizontal angle in the third spatial relationship. The volume of the object audio output by the second speaker 302 is lower than that of the first speaker 301, or it is silent. The second speaker 302 also outputs the object audio later than the first speaker 301.

[0033] Similarly, according to the third spatial relationship, when the object is displayed at positions 311 to 318 , the AR device 300 adjusts the object image and object audio as shown in Table 1.

[0034] Table 1

[0035]

[0036]

[0037] Figure 4 A schematic diagram showing the presentation of multiple objects in an augmented reality environment according to an embodiment of the present invention. Figure 4 As shown, this embodiment includes an AR device 410, a first object 421, and a second object 422. The AR device 410 includes a first speaker 411 and a second speaker 412. When a user wears the AR device 410, the first speaker 411 is close to the user's left ear, and the second speaker 412 is close to the user's right ear. The first object 421 contains object audio, and the second object 422 is as shown in FIG. Figure 4 The figure shows a large object in the shape of a house. The first object 421 is located on the right side of the AR device 410 at a first distance 431. Based on the position of the first object 421 relative to the AR device 410 and the position of the second object 422 relative to the AR device 410, the left wall of the first object 421 and the second object 422 are separated by a second distance 432, and the left wall of the AR device 410 and the second object 422 are separated by a third distance 433.

[0038] The timing of the object audio output from the first object 421 by the first speaker 411 is adjusted based on "(second distance 432 + third distance 433) / speed of sound." The timing of the object audio output from the second speaker 412 is adjusted based on "first distance 431 / speed of sound." The volume of the object audio output from the first object 421 by the second speaker 412 is adjusted based on "system volume setting value - 20log first distance 431 / reference distance." The system volume setting value is the creator's initial definition of the volume of the sound emitted by the object, and the reference distance is defined by the creator. The volume of the object audio output from the first object 421 by the first speaker 411 is adjusted based on "volume output from the second speaker - 20log * (second distance 432 + third distance 433) / first distance 431."

[0039] Figure 5 A schematic diagram showing multiple objects displayed in an augmented reality environment according to an embodiment of the present invention. Figure 5 As shown, the AR device 510 obtains the second spatial relationship of the AR device 510 relative to the anchor device 520 and the object identifier through the anchor device 520, and obtains the corresponding object parameters from the anchor cloud 530 through the object identifier. In this embodiment, the object identifier includes the object identifiers of the first object 540 and the second object 550. Figure 5As shown, the size of an object image can be as small as first object 540, displayed close to the user, or as large as second object 550, encompassing the user. In one embodiment, the object parameters further include a time parameter. AR device 510 also sets the display time of first object 540 and second object 550 based on the time parameters of first object 540 and second object 550, respectively. For example, first object 550 can be displayed permanently, while second object 540 can be displayed during a specific time period. Specifically, multiple objects can be divided into fixed objects and real-time objects based on the time parameter. For example, if the object image of second object 550 is a house and the time parameter in the object parameter is set to permanent, after AR device 510 obtains second object 550 from the anchor cloud, it can be fixedly displayed in the augmented reality system. In actual implementation, the second object can also include billboards, road signs, buildings, specific scenes, etc. If the augmented reality environment is a live broadcast, speech, or football game, since these environments are activities that occur within a specific time period, the objects in these environments are objects that belong to a specific time period. By setting time parameters, objects can be displayed in real time or the display time of objects can be set to display within a specific time period.

[0040] Figure 6 A schematic diagram showing a one-way application scenario of the method for displaying objects in an augmented reality environment according to an embodiment of the present invention. Figure 6 As shown, a speaker 610 uses a camera 620 to capture an image as an object, sets object parameters, and sends them to an anchor cloud 630. This allows the speaker's object 640 to be displayed in a remote scene. A user wearing an AR device can locate the object 640 in the remote scene via an anchor device 650. In one embodiment, the object parameters further include encryption parameters, which can be either encrypted or unencrypted. Only AR devices capable of decrypting the encryption parameters of object 640 can display the encrypted object; AR devices that cannot decrypt the object cannot display the encrypted object 640. When the encryption parameters of object 640 are set to "encrypted," only AR devices capable of decrypting the object can display the object 640, limiting the viewer 660 to viewing through AR devices capable of decrypting the object. A general viewer 670, whose AR device cannot decrypt the object, cannot see the object 640. When the encryption parameters of object 640 are set to "unencrypted," anyone using an AR device can view the object 640.

[0041] Figure 7 A schematic diagram showing a two-way application scenario of the method for displaying objects in an augmented reality environment according to an embodiment of the present invention. Figure 7As shown, speaker 710 uses camera 720 to capture an image as object 740, sets object parameters, and sends it to anchor cloud 730. This allows speaker 710's object 740 to be displayed in the remote scene. Viewer 760 uses anchor device 750 to locate object 740 in the remote scene. Simultaneously, viewer 760 uses camera 770 to capture an image as object 790, sets object parameters, and sends it to anchor cloud 730. This allows viewer 760's object 790 to be displayed in the scene where speaker 710 is located. Anchor device 780 locates object 790 in the scene, thereby achieving two-way interaction between multiple objects.

[0042] In summary, this invention meets the requirements for invention patents, and a patent application is hereby filed in accordance with the law. However, the foregoing description is merely a preferred embodiment of the invention, and the scope of the invention is not limited to the foregoing embodiment. Any equivalent modifications or variations made by persons skilled in the art based on the spirit of the invention are intended to be encompassed by the following patent application.

Claims

1. A method for displaying an object in an augmented reality environment, characterized in that: The method comprises the following steps: The augmented reality (AR) device obtains spatial information and object identifiers of the anchor device; The AR device obtains object parameters corresponding to the object identifier from the anchor point cloud according to the object identifier, where the object parameters include an object image, an object audio, and a first spatial relationship of the object relative to the anchor point device; The AR device obtains a second spatial relationship relative to the anchor device based on the spatial information; obtaining a first position of the object in the augmented reality environment based on the first spatial relationship, obtaining a second position of the AR device in the augmented reality environment based on the second spatial relationship, and obtaining a third spatial relationship of the object relative to the AR device based on the first and second positions; The AR device displays the object image and the object audio according to the third spatial relationship.

2. The method for displaying an object in an augmented reality environment as claimed in claim 1, wherein: The AR device further includes an inertial measurement unit for obtaining a moving direction, a moving distance, and a moving height of the AR device when the AR device moves. The AR device updates the second spatial relationship and the third spatial relationship in real time according to the moving direction, the moving distance, and the moving height.

3. The method for displaying an object in an augmented reality environment as claimed in claim 1, wherein: The object parameters further include a time parameter, and the AR device sets the display time of the object according to the time parameter.

4. The method for displaying an object in an augmented reality environment as claimed in claim 1, wherein: The object parameters further include encryption parameters, including encryption and non-encryption. The objects displayed by the AR device include objects whose encryption parameters are unencrypted and objects whose encryption parameters are encrypted and can be decrypted by the AR device.

5. A system for displaying objects in an augmented reality environment, characterized in that: include: An augmented reality (AR) device that obtains spatial information and object identifiers from an anchor device; The AR device obtains object parameters corresponding to the object identifier from the anchor point cloud based on the object identifier, where the object parameters include an object image, an object audio, and a first spatial relationship of the object relative to the anchor point device; Obtaining a second spatial relationship between the AR device and the anchor device based on the spatial information; Obtaining the object parameters from the anchor point cloud according to the object identifier; obtaining a first position of the object in the augmented reality environment based on the first spatial relationship, obtaining a second position of the AR device in the augmented reality environment based on the second spatial relationship, and obtaining a third spatial relationship of the object relative to the AR device based on the first and second positions; The object image and the object audio are displayed on the AR device according to the third spatial relationship.

6. The object display system in an augmented reality environment as claimed in claim 5, wherein: The AR device further includes an inertial measurement unit for obtaining a moving direction, a moving distance, and a moving height of the AR device when the AR device moves. The AR device updates the second spatial relationship and the third spatial relationship in real time according to the moving direction, the moving distance, and the moving height.

7. The object display system in an augmented reality environment as claimed in claim 5, wherein: The object parameters further include a time parameter, and the AR device sets the display time of the object according to the time parameter.

8. The object display system in an augmented reality environment as claimed in claim 5, wherein: The object parameters further include encryption parameters, including encryption and non-encryption. The objects displayed by the AR device include objects whose encryption parameters are unencrypted and objects whose encryption parameters are encrypted and can be decrypted by the AR device.

Images