Information display method and information display system

Abstract

The present invention discloses an information display method and an information display system. The information display method is applicable to an information display system including a first image capture device and a transparent display, and the method includes the following steps. The totem image is acquired by photographing a totem presented on a physical object by the first image capture device, wherein the physical object is located on one side of the transparent display. The relative position information between the transparent display and the totem is acquired based on the totem image. Augmented image data is acquired based on the relative position information. The augmented image data is displayed by the transparent display, wherein the augmented image data includes a virtual object augmented based on the physical object.

Description

Technical Field

[0001] The present disclosure relates to an information display method and an information display system. Background Art

[0002] Transparent displays can be divided into penetrating transparent displays and projected transparent displays according to their technical principles. Transparent displays have the characteristics of transparency, lightness and thinness, and can combine the virtual images displayed on the display with the scenes in the real world, giving users an intuitive and direct visual experience. Transparent displays can be introduced into applications such as automotive head-up displays, vending machines, product windows, museum exhibits, sightseeing vehicle guides, and the medical industry. Viewers can not only see physical objects through transparent displays, but also see virtual information superimposed on physical objects or displayed around them on the transparent display. By projecting virtual information into the real world through transparent displays, users can increase their cognition and understanding of the surrounding environment.

[0003] Since the virtual information displayed on the transparent display is augmented based on the physical object, the degree of overlap between the virtual information and the physical object will directly affect the viewer's viewing experience. For example, if the transparent display overlaps and displays virtual information including relevant information of the first physical object around or above the second physical object, it will cause misunderstanding to the viewer. In particular, when the transparent display can be dynamically moved, the geometric position relationship between the transparent display and the physical object, such as the relative position, rotation amplitude, tilt angle, etc., cannot be accurately confirmed, resulting in the virtual information being unable to be accurately displayed at the appropriate position, making it difficult for the virtual information presented on the transparent display to be accurately overlapped with the physical object. Summary of the invention

[0004] An embodiment of the present disclosure provides an information display method, which is applicable to an information display system including a first image capture device and a transparent display, and the method includes the following steps. The first image capture device is used to capture a totem presented on a physical object to obtain a totem image, wherein the physical object is located on one side of the transparent display. The relative position information between the transparent display and the totem is obtained based on the totem image. The augmented image data is obtained based on the relative position information. The augmented image data is displayed by the transparent display, wherein the augmented image data includes a virtual object augmented based on the physical object.

[0005] An embodiment of the present disclosure provides an information display system, which includes at least one totem, a transparent display, a first image capturing device, and a processing device. The first image capturing device captures the totem presented on a physical object to obtain a totem image, wherein the physical object is located on one side of the transparent display. The processing device is coupled to the transparent display and the first image capturing device, and the processing device is configured to perform the following steps: obtaining relative position information between the transparent display and the totem according to the totem image; obtaining augmented image data according to the relative position information; and displaying the augmented image data through the transparent display, wherein the augmented image data includes a virtual object augmented based on the physical object.

[0006] In order to make the present disclosure more clearly understood, embodiments are given below with reference to the accompanying drawings for detailed description. BRIEF DESCRIPTION OF THE DRAWINGS

[0007] Figure 1 is a schematic diagram of an information display system according to an embodiment of the present disclosure.

[0008] Figure 2 is a situational schematic diagram of an information display system according to an embodiment of the present disclosure.

[0009] Figure 3 is a flow chart of an information display method according to an embodiment of the present disclosure.

[0010] Figure 4 is a flow chart of an information display method according to an embodiment of the present disclosure.

[0011] Figure 5 FIG. 4 is a schematic diagram of calculating a depth value according to an embodiment of the present disclosure.

[0012] Figure 6 is a flowchart of a camera calibration procedure of an information display method according to an embodiment of the present disclosure.

[0013] Figure 7 is a schematic diagram of an information display system according to an embodiment of the present disclosure.

[0014] Figure 8 is a flow chart of an information display method according to an embodiment of the present disclosure.

[0015] Figures 9A to 9C is a situational schematic diagram of an information display system according to an embodiment of the present disclosure.

[0016] Fig.10 is a schematic diagram of an information display system according to an embodiment of the present disclosure.

[0017] Fig.11 is a flow chart of an information display method according to an embodiment of the present disclosure.

[0018] Fig.12 is a flow chart of an information display method according to an embodiment of the present disclosure.

[0019] Fig.13A and 13B It is a schematic diagram of a scenario in which display content is determined based on a sight line vector according to an embodiment of the present disclosure.

[0020]

Explanation of symbols

[0021] 10, 20, 30: Information display system

[0022] 110: Transparent display

[0023] 120: First image capturing device

[0024] 130: Processing device

[0025] 131: Processor

[0026] 132: Storage device

[0027] P1: Totem

[0028] C1: Corner point

[0029] Obj_v1: Virtual object

[0030] Obj_r1: physical object

[0031] U1, U2: Viewers

[0032] P1_1: First Totem

[0033] P1_2: Second Totem

[0034] Obj_v11: The first virtual object

[0035] Obj_v12: Second virtual object

[0036] 140: Second image capturing device

[0037] P2_1, P2_2: Viewer Totem

[0038] D1, D2: normal direction

[0039] E1, E2: sight vector

[0040] S310~S340, S410~S440, S610~S640, S810~S860, S1110~S1170, S1210~S1260: Steps DETAILED DESCRIPTION

[0041] Some embodiments of the present disclosure will be described in detail below in conjunction with the accompanying drawings. The referenced element symbols in the following description will be regarded as the same or similar elements when the same element symbols appear in different drawings. These embodiments are only part of the present disclosure and do not disclose all possible implementation methods of the present disclosure. More specifically, these embodiments are only examples of the information display method and information display system in the present disclosure.

[0042] Figure 1 is a schematic diagram of an information display system according to an embodiment of the present disclosure. Figure 1 The information display system 10 includes at least one totem P1 , a transparent display 110 , a first image capturing device 120 , and a processing device 130 . The processing device 130 is coupled to the transparent display 110 and the first image capturing device 120 .

[0043] The totem P1 may be printed on an adhesive carrier or displayed on an electronic paper, and the adhesive carrier or electronic paper showing the totem P1 may be located at a preset position of the physical object Obj_r1. The totem P1 has a plurality of characteristic points (e.g., corner point C1) to facilitate the subsequent identification of the relative position information between the totem P1 and the transparent display 110. In one embodiment of the present disclosure, the totem P1 may include more than one one-dimensional barcode, two-dimensional barcode or other recognizable pattern, but the present disclosure is not limited thereto.

[0044] The transparent display 110 is a display with a certain degree of light transmittance, which can present the scene on the other side of the display relative to the viewer when viewed by a human user, for example, a thin film transistor liquid crystal display (TFT-LCD), a field sequential color display, an active matrix organic light emitting display (AMOLED), an electrowetting display, or other transmissive transparent displays, or a projection transparent display.

[0045] The first image capturing device 120 is suitable for performing image sensing functions. In one embodiment of the present disclosure, the first image capturing device 120 may include a lens and a photosensitive element for capturing a totem image toward the totem P1 presented on the physical object Obj_r1. The photosensitive element is used to sense the intensity of light entering the lens, thereby generating an image. The photosensitive element may be, for example, a charge coupled device (CCD), a complementary metal-oxide semiconductor (CMOS) element or other element, and the present disclosure is not limited thereto. In one embodiment of the present disclosure, the first image capturing device 120 is disposed on the transparent display 110, so that the position and shooting direction of the first image capturing device 120 in three-dimensional space will change with the movement of the transparent display 110.

[0046] The processing device 130 includes one or more processors (hereinafter, the processor 131 is used as a representative for description) and one or more storage devices (hereinafter, the storage device 132 is used as a representative for description). The processing device 130 can be implemented by a computer host, a server, or a computer device with data computing capabilities. In addition, the processing device 130 can be implemented by one or more electronic devices, which is not limited by the present disclosure.

[0047] The processor 131 is, for example, a central processing unit (CPU), or other programmable general-purpose or special-purpose microprocessor, digital signal processor (DSP), programmable controller, application specific integrated circuits (ASIC), programmable logic device (PLD) or other similar devices or a combination of these devices. The processor 131 is used to execute instructions or programs recorded in the storage device 132.

[0048] The storage device 132 may be any type of fixed or removable random access memory (RAM), read-only memory (ROM), flash memory or similar device or a combination of the above devices. The storage device 132 is used to store image data and programs or instructions that can be accessed and executed by the processor 131.

[0049] In the following embodiments, the implementation details of the information display method system and the information display method will be described in conjunction with the various components of the display system 10 .

[0050] Figure 2 is a schematic diagram of a situation of an information display system according to an embodiment of the present disclosure. Figure 2 In one embodiment of the present disclosure, the first image capturing device 120 may be, for example, fixedly disposed in the middle of the upper side of the transparent display 110, and used to perform an image sensing function toward the physical object Obj_r1 behind the transparent display 110, but the present disclosure is not limited thereto. In other embodiments, the first image capturing device 120 may be disposed on the left side, right side, or lower side of the transparent display 110. In addition, the transparent display 110 is disposed between the viewer U1 and the physical object Obj_r1. Based on this, the viewer U1 can view the physical object Obj_r1 through the transparent display 110, and can simultaneously view the augmented image data including the virtual object Obj_v1 displayed by the transparent display 110.

[0051] like Figure 2 As shown, at least one totem P1 is presented on the physical object Obj_r1, and the first image capturing device 120 captures a totem image toward the totem P1. After the first image capturing device 120 captures the totem image, the processing device 130 can analyze the totem information in the totem image to calculate the relative position information between the totem P1 and the transparent display 110, and obtain the corresponding augmented image data according to the relative position information.

[0052] Specifically, as the position or placement angle of the transparent display 110 changes, the relative position information between the totem P1 and the transparent display 110 will change accordingly. Correspondingly, in response to the change in the relative position information between the totem P1 and the transparent display 110, the transparent display 110 will also adjust the display position of the virtual object Obj_v1 in the augmented image data. In other words, the display position of the virtual object Obj_v1 in the display screen of the transparent display 110 is determined based on the relative position information between the totem P1 and the transparent display 110. Thereby, when the viewer U1 views the transparent display 110, the virtual object Obj_v1 displayed by the transparent display 110 can be superimposed on the physical object Obj_r1, thereby enhancing the viewer's viewing experience.

[0053] Figure 3 is a flow chart of an information display method according to an embodiment of the present disclosure. Figure 3 The described method flow can be combined with Figure 1 Please also refer to the information display system 10 shown in FIG. Figure 1 , Figure 2 and Figure 3 .

[0054] In step S310, the first image capturing device 120 captures the totem P1 displayed on the physical object Obj_r1 to obtain a totem image. Figure 2 As shown, the physical object Obj_r1 is located at the rear side of the transparent display 110, and the viewer U1 is located at the front side of the transparent display 110. The capturing direction of the first image capturing device 120 is toward the rear side of the transparent display 110. In step S320, the relative position information between the transparent display 110 and the totem P1 is acquired according to the totem image by the processing device 130. Here, the relative position information between the transparent display 110 and the totem P1 may include distance information, angle information, coordinate information, etc.

[0055] It should be noted that the actual size of the totem P1 can be designed according to the resolution of the first image capture device 120. The totem P1 can be attached to the physical object Obj_r1, and the carrier surface of the totem P1 may be uneven, resulting in the calculation of relative position information with errors. If the actual size of the totem P1 can be reduced, the deformation degree of the carrier surface of the totem P1 can be reduced, thereby improving the calculation accuracy of the relative position information. However, if the actual size of the totem P1 is too small, it will also lead to insufficient totem features in the totem image and the relative position information cannot be accurately calculated. In one embodiment of the present disclosure, the actual size of the totem P1 can be designed according to the resolution of the first image capture device 120, so as to effectively reduce the deformation degree of the carrier surface of the totem P1 and avoid the situation of insufficient totem features.

[0056] Next, in step S330, the processing device 130 obtains the augmented image data according to the relative position information. Here, the processing device 130 can calculate the augmented image data according to the relative position information, or retrieve the augmented image data established in advance from the image database according to the relative position information. Further, by performing a geometric transformation on a basic image according to the relative position information, the processing device 130 can obtain augmented image data corresponding to different relative position information. The above-mentioned geometric transformation may include scaling processing, rotation processing, translation processing, affine transformation or other geometric mapping transformations, etc. In step S340, the augmented image data is displayed by the transparent display 110. The augmented image data includes a virtual object Obj_v1 augmented based on the physical object Obj_r1. It can be seen that the augmented image data displayed by the transparent display 110 is determined based on the relative position information between the transparent display 110 and the totem P1.

[0057] In a medical application scenario, the physical object Obj_r1 may be a human body, and the viewer U1 may be a medical staff. The totem P1 may be attached to a preset part of the human body, for example. The transparent display 110 may display medical related information, such as organ information or surgical information, etc. When the transparent display 110 is needed to provide medical related information of the human body, the medical staff may place the transparent display 110 between themselves and the physical object Obj_r1, and simultaneously view the human body and the virtual object Obj_v1 augmented based on the human body through the transparent display 110. When the transparent display 110 is not needed to provide medical related information of the human body, the medical staff may move the transparent display 110 away. It can be seen that the medical staff may move the transparent display 110 back and forth during clinical teaching or surgery. In one embodiment of the present disclosure, the processing device 130 may determine the augmented image data based on the relative position information between the transparent display 110 and the totem P1. Even if the transparent display 110 moves dynamically, the virtual object Obj_v1 augmented based on the human body can still be superimposed on the human body. For example, assuming that the virtual object Obj_v1 is a heart image, even if the transparent display 110 is moved back and forth, the medical staff can still see that the heart image displayed by the transparent display 110 is superimposed on the heart of the human body.

[0058] Figure 4 is a flow chart of an information display method according to an embodiment of the present disclosure. Figure 4 The described method flow can be combined with Figure 1 The information display system 10 shown in FIG. Figure 4 In the embodiment of FIG. 1 , the relative position information may include the image coordinates of the totem P1 in the image coordinate system, the depth value of the totem P1 from the first image capturing device 120 , and the rotation angle of the totem P1 .

[0059] Please also refer to Figure 1 and Figure 4 , in step S410, the totem P1 presented on the physical object Obj_r1 is photographed by the first image capture device 120 to obtain a totem image. It should be noted that in one embodiment of the present disclosure, the processing device 130 may perform image correction on the totem image to correct the totem image that may be lens-deformed, so that the processing device 130 can calculate more accurate relative position information based on the corrected totem image. Here, the step S420 of obtaining the relative position information may include sub-steps S421 to S422. In step S421, the processing device 130 calculates the image coordinates of the totem P1 in the image coordinate system based on the totem image and the depth value of the totem P1 from the first image capture device 120.

[0060] Specifically, the actual size information of the totem P1 is known. By comparing the imaging size information of the totem P1 on the totem image with the actual size information of the totem P1, the processing device 130 can estimate the distance (i.e., the depth value) between the totem P1 and the first image capture device 120. The actual size information and the imaging size information of the totem P1 can be the length information between two image feature points of the totem P1, but the present disclosure is not limited to this. In one embodiment of the present disclosure, the processing device 130 can calculate the depth value based on the intrinsic parameters of the first image capture device 120, the actual size information of the totem P1, and the imaging size information of the totem P1 on the totem image. The internal parameters of the first image capture device 120 may include the focal length information of the first image capture device 120, and the internal parameters of the first image capture device 120 can be obtained via a camera calibration program.

[0061] Please refer to Figure 5 , Figure 5 Schematic diagram of calculating the depth value according to an embodiment of the present disclosure. Based on the principle of similar triangles, the ratio between the actual size information L1 of the totem P1 and the depth value Z1 is equivalent to the ratio between the imaging size information L2 of the totem P1 and the focal length information f1. Based on this, when the processing device 130 knows the actual size information L1 of the totem P1, the focal length information f1 and the imaging size information L2 of the totem P1, the processing device 130 can obtain the depth value Z1 of the totem P1 from the first image capturing device 120 according to formula (1).

[0062] L2=f1*(L1 / Z1) formula (1)

[0063] In one embodiment of the present disclosure, the processing device 130 can directly obtain the image coordinates of the totem P1 in the image coordinate system according to the imaging position of the totem P1 in the totem image (e.g., the pixel coordinates of a specific totem feature), that is, the two-dimensional image coordinates on the imaging plane. Alternatively, in one embodiment of the present disclosure, the processing device 130 can calculate the image coordinates of the totem P1 in the image coordinate system according to the world coordinates of the totem P1 and the internal parameters of the first image capture device 120. In detail, the world coordinates of the totem P1 in the world coordinate system have been calibrated as (X w , Y w , Z w ), the processing device 130 may be based on the internal parameter matrix of the first image capturing device 120 Calculate the two-dimensional image coordinates of the totem P1 projected on the imaging plane. In one embodiment, the processing device 130 can obtain the image coordinates (x, y) of the totem P1 projected on the imaging plane according to formula (2) and formula (3).

[0064] x=(f x *X w / Z w )+u0 formula (2)

[0065] y=(f y *Y w / Z w )+v0 formula (3)

[0066] Next, in step S422, the processing device 130 calculates the rotation angle of the totem P1 based on the totem image. The rotation angle of the totem P1 can be regarded as the normal direction of the carrier surface presenting the totem P1, and the above-mentioned normal direction can be defined by a three-axis rotation angle quantity. In one embodiment of the present disclosure, the processing device 130 can calculate the rotation angle of the totem P1 based on the geometric relationship between multiple image feature points of the totem P1 on the totem image. Taking the totem P1 as a rectangle as an example, the processing device 130 can calculate the normal direction of the surface of the totem P1 (that is, the rotation angle of the totem P1) based on the two actual side lengths of the totem P1 and the corresponding imaging side lengths.

[0067] In one embodiment of the present disclosure, the processing device 130 can calculate the rotation angle of the totem P1 according to the camera parameters of the first image capturing device 120. Specifically, according to formula (4), the processing device 130 can calculate the rotation angle of the totem P1 according to the world coordinates (X w , Y w , Z w ), the image coordinates (x, y) of the totem P1, and the internal parameters of the first image capturing device 120 to calculate the external parameter matrix of the first image capturing device 120. Here, the world coordinates (X w , Y w , Z w ) and the internal parameters of the first image capturing device 120 can be obtained through the previously executed camera calibration procedure. Specifically, the world coordinates of the totem P1 in the world coordinate system have been positioned as (X w , Y w , Z w ), the processing device 130 may calculate the external parameter matrix of the first image capturing device 120 based on the internal parameter matrix of the first image capturing device 120 and the image coordinates (x, y) of the totem P. Here, the internal parameter matrix and the external parameter matrix are used to describe the transformation relationship between the image coordinate system and the world coordinate system.

[0068]

[0069] in, is the internal parameter matrix of the first image capturing device 120, and is the external parameter matrix of the first image capturing device 120. The external parameter matrix includes a rotation matrix R and a translation vector t. The external parameter matrix of the first image capturing device 120 can be used to represent the position and shooting direction of the first image capturing device 120 in the world coordinate system.

[0070] Next, the processing device 130 may calculate the rotation angle according to the external parameter matrix of the first image capturing device 120. The processing device 130 may calculate the three-axis rotation angle (φ, θ, ψ), i.e., the rotation angle of the totem P1, according to the rotation matrix R in the external parameter matrix based on formula (5). Wherein, φ is the rotation angle corresponding to the Z axis; θ is the rotation angle corresponding to the Y axis; ψ is the rotation angle corresponding to the X axis.

[0071]

[0072] After obtaining the relative position information according to the totem image, in step S430, the processing device 130 obtains the augmented image data according to the image coordinates, the depth value and the rotation angle. It can be seen that the displayed augmented image data is generated by geometrically transforming the basic image data according to the image coordinates, the depth value and the rotation angle. Here, the basic image data corresponds to the reference image coordinates, the reference depth value and the reference rotation angle. The augmented image data is generated by geometrically transforming the basic image data according to the coordinate difference between the reference image coordinates and the image coordinates, the depth difference between the reference depth value and the depth value, and the angle difference between the reference rotation angle and the rotation angle. In detail, the processing device 130 can perform translation processing on the basic image data according to the coordinate difference; the processing device 130 can perform scaling processing on the basic image data according to the depth difference; the processing device 130 can perform rotation processing or affine transformation on the basic image data according to the rotation angle difference. It should be noted that the augmented image data can be generated in real time by performing real-time calculations based on the aforementioned geometric transformations. Alternatively, the augmented image data can be recorded in the image database by performing pre-calculations based on the aforementioned geometric transformations. Therefore, the processing device 130 can select the corresponding augmented image data from the image database according to the detected image coordinates, depth value, and rotation angle. In step S440, the augmented image data is displayed between the viewer U1 and the physical object Obj_r1 by the transparent display 110.

[0073] In one embodiment of the present disclosure, the operation of geometrically transforming the basic image data can be performed in advance to establish an image database. That is, the image database stores augmented image data corresponding to different preset image coordinates, different preset depth values, and different preset rotation angles. Based on this, the processing device 130 can query the image database based on the image coordinates, depth values, and rotation angles generated based on the totem image and select the corresponding augmented image data to achieve a real-time display effect. The following will list embodiments to illustrate how to define the reference image coordinates, reference depth values, and reference rotation angles corresponding to the basic image and how to establish an image database that can be queried in real time by the processing device 130.

[0074] Figure 6 is a flowchart of a camera calibration procedure of an information display method according to an embodiment of the present disclosure. Figure 6 The described method flow can be combined with Figure 1 Please also refer to the information display system 10 shown in FIG. Figure 1 and Figure 6 .

[0075] First, in step S610, the first image capturing device 120 captures the totem P1 presented on the physical object Obj_r1 to obtain a calibrated totem image. Next, in step S620, the processing device 130 defines the reference image coordinates, reference depth value, and reference rotation angle associated with the basic image data based on the calibrated totem image. Specifically, the processing device 130 may Figure 5 The principle shown in FIG. 1 is used to calculate the reference depth between the totem P1 and the first image capturing device 120. In addition, the processing device 130 may define the reference image coordinates and the reference rotation angle as corresponding preset values. For example, the processing device 130 may define the reference image coordinates (x b ,y b ) is (0, 0) and defines the base rotation angle (φ b ,θ b , ψ b ) is (0, 0, 0). b ,y b ) and the reference rotation angle (φ b ,θ b , ψ b ) is a preset value, the processing device 130 can calibrate the world coordinates of the totem P1 based on formula (4).

[0076] Next, in step S630, the processing device 130 processes the image data according to the reference depth value and the reference image coordinate (x b ,y b ) and the reference rotation angle (φ b ,θb , ψ b ) to establish an image database. Here, the processing device 130 can be based on the defined reference depth value, reference image coordinates (x b ,y b ) and the reference rotation angle (φ b ,θ b , ψ b ) and a plurality of preset depth values, a plurality of preset image coordinates, and a plurality of preset rotation angles to perform geometric transformation on the basic image to generate corresponding augmented image data, thereby establishing an image database for future query. As mentioned above, the processing device 130 can establish an image database recording a plurality of pre-stored augmented images based on the depth difference between the plurality of preset depth values ​​and the reference depth value, the coordinate difference between the plurality of preset image coordinates and the reference image coordinates, and the angle difference between the plurality of preset rotation angles and the reference rotation angle. In other words, each pre-stored augmented image in the image database can correspond to one of the preset depth values, one of the preset image coordinates, and one of the plurality of preset rotation angles. Based on this, the processing device 130 can select the augmented image data that can be superimposed with the physical object Obj_r1 from the pre-stored augmented images based on the detected depth value, image coordinates, and rotation angle.

[0077] In step S640, the processing device 130 determines whether the degree of overlap between the basic image and the physical object Obj_r1 meets specific conditions. In one embodiment of the present disclosure, the first image capture device 120 can shoot a physical object image toward the physical object, and the processing device 130 performs image recognition on the physical object image to find the overlapping area where the virtual object needs to be superimposed. Next, the processing device 130 calculates the percentage of overlap between the overlapping areas of the virtual object and the physical object image in the basic image, and determines whether this quantified percentage of overlap is greater than the overlap threshold. If step S640 is judged as no, the viewer U1 can adjust the placement position and placement angle of the transparent display 110, and return to step S610 to re-establish the image database. If step S640 is judged as yes, the processor 130 ends the camera calibration program. The camera calibration program can be executed as follows Figure 3 and Figure 4 By executing the camera calibration procedure, the processing device 130 can obtain the reference image coordinates, reference depth values, and reference rotation angles corresponding to the base image, and accordingly establish an image database storing pre-stored augmented images corresponding to different preset image coordinates, different preset depth values, and different preset rotation angles.

[0078] In one embodiment of the present disclosure, the totem P1 may further include virtual object information. Based on this, the processing device 130 may identify the totem P1 according to the totem image to obtain the virtual object information corresponding to the totem, and select the augmented image data from the image database according to the virtual object information and the relative position information. For example, assuming that the totem P1 is a two-dimensional barcode including a plurality of black squares and white squares, the totem P1 is generated by encoding based on the virtual object information. Correspondingly, the processing device 130 may analyze and decode the totem P1 to obtain the corresponding virtual object information, so that the transparent display 110 may display the corresponding augmented image data. The virtual object information may be an object name or an object ID, etc. For example, the processing device 130 may identify the totem P1 and obtain that the virtual object information corresponding to the totem P1 is "heart", and the processing device 130 may obtain the augmented image data showing the heart information from the image database according to the relative position information and the virtual object information. It can be seen that the virtual object information carried by the totem P1 can be used to determine the information content displayed by the transparent display 110. It can be seen that if the totem P1 on the physical object Obj_r1 is replaced with another totem, the transparent display 110 will display the virtual object corresponding to the other totem.

[0079] It is worth mentioning that, in addition to the totem features of the totem P1 being used to determine the display content, in one embodiment of the present disclosure, the processing device 130 may further determine the display content based on the attachment position of the totem P1 on the physical object Obj_r1. That is, the processing device 130 may identify the attachment position of the totem P1 based on the totem image to obtain the virtual object information corresponding to the totem, and select the augmented image data from the image database based on the virtual object information and the relative position information. For example, when the processing device 130 determines that P1 is attached to the first attachment position on the physical object Obj_r1 (e.g., a position adjacent to the first pair of ribs), the processing device 130 may select the augmented image data based on the virtual object information corresponding to the first attachment position (e.g., "first pair of ribs"), and the transparent display 110 will also display the augmented image data including the first pair of ribs. When the processing device 130 determines that P1 is attached to the second attachment position on the physical object Obj_r1 (for example, a position adjacent to the fifth pair of ribs), the processing device 130 can select the augmented image data based on the virtual object information corresponding to the second attachment position (for example, "fifth pair of ribs"), and the transparent display 110 will also display the augmented image data including the fifth pair of ribs.

[0080] The above embodiment is described by taking one totem as an example, but the present disclosure is not limited thereto. In one embodiment of the present disclosure, a plurality of totems may also be presented on the physical object Obj_r1. The following examples are given for illustration. Figure 7 is a schematic diagram of an information display system according to an embodiment of the present disclosure. Figure 7The information display system 20 includes a first totem P1_1 and a second totem P1_2, a transparent display 110, a first image capturing device 120, and a processing device 130. The processing device 130 is coupled to the transparent display 110 and the first image capturing device 120. The operations and functions of the transparent display 110, the first image capturing device 120, and the processing device 130 are similar to Figure 1 It should be noted that the information display system 20 includes two totems (ie, a first totem P1_1 and a second totem P1_2).

[0081] Figure 8 is a flow chart of an information display method according to an embodiment of the present disclosure. Figure 8 The described method flow can be combined with Figure 7 Please also refer to the information display system 20 shown in FIG. Figure 7 and Figure 8 .

[0082] In step S810, the first image capturing device 120 captures the first totem P1_1 and the second totem P1_2 presented on the physical object Obj_r1 to obtain a totem image. In step S820, the processing device 130 obtains first relative position information between the transparent display 110 and the first totem P1_1 and second relative position information between the transparent display 110 and the second totem P1_2 according to the totem image. The calculation method of the first relative position information and the second relative position information can be the same as the above description, which is not repeated here. In step S830, the processing device 130 recognizes the first totem P1_1 and the second totem P1_2 according to the totem image to obtain the first virtual object information corresponding to the first totem P1_1 and the second virtual object information corresponding to the second totem P1_2. For example, the first virtual object information can be "heart" and the second virtual object information can be "stomach".

[0083] In step S840, the processing device 130 selects the first sub-augmented image data from the image database according to the first virtual object information and the first relative position information associated with the first totem P1_1. In step S850, the processing device 130 selects the second sub-augmented image data from the image database according to the second virtual object information and the second relative position information associated with the second totem P1_2. Specifically, in one embodiment of the present disclosure, the image database has sub-classification databases corresponding to different virtual objects. Therefore, when the processing device 130 obtains the first virtual object information corresponding to the first totem P1_1, the processing device 130 can select the first sub-augmented image data from the first sub-classification database associated with the first virtual object according to the relative position information based on the first totem P1_1. When the processing device 130 obtains the second virtual object information corresponding to the second totem P1_2, the processing device 130 can select the second sub-augmented image data from the second sub-classification database associated with the second virtual object according to the relative position information based on the second totem P1_1. Next, in step S860, the transparent display 110 displays the first sub-amplified image data and / or the second sub-amplified image data. ...Displaying the first sub-amplified image data and / or the second sub-amplified image data

[0084] Figures 9A to 9C This is a schematic diagram of a situation of an information display system according to an embodiment of the present disclosure. Fig.9A , assuming that only the first totem P1_1 is presented on the physical object Obj_r1, the processing device 130 will calculate the relative positional relationship between the first totem P1_1 and the transparent display 110, and identify the first virtual object information corresponding to the first totem P1_1 as "heart". Therefore, the transparent display 110 will display the first sub-augmented image data with heart information. Specifically, the transparent display 110 will display the first virtual object Obj_v11 (i.e., the heart image) augmented based on the physical object Obj_r1, and the viewer can see the first virtual object Obj_v11 superimposed on the physical object Obj_r1 through the transparent display 110.

[0085] Please refer to Fig. 9B, assuming that only the second totem P1_2 is presented on the physical object Obj_r1, the processing device 130 will calculate the relative positional relationship between the second totem P1_2 and the transparent display 110, and identify the second virtual object information corresponding to the second totem P1_2 as "stomach". Therefore, the transparent display 110 will display the second sub-augmented image data with stomach information. Specifically, the transparent display 110 will display the second virtual object Obj_v12 (i.e., the stomach image) augmented based on the physical object Obj_r1, and the viewer can see the second virtual object Obj_v12 superimposed on the physical object Obj_r1 through the transparent display 110.

[0086] Please refer to Fig. 9C , assuming that the physical object Obj_r1 presents the first totem P1_1 and the second totem P1_2 at the same time, the processing device 130 will respectively calculate the second relative position relationship between the second totem P1_2 and the transparent display 110 and the first relative position relationship between the first totem P1_1 and the transparent display 110. The processing device 130 further identifies the first virtual object information corresponding to the first totem P1_1 as "heart", and identifies the second virtual object information corresponding to the second totem P1_2 as "stomach". Therefore, the processing device 130 superimposes the first sub-augmented image data and the second sub-augmented image data to generate superimposed augmented image data. The transparent display 110 will display the superimposed augmented image data having the heart information and the stomach information. Specifically, the transparent display 110 will simultaneously display the first virtual object Obj_v11 (ie, a heart image) and the second virtual object Obj_v12 (ie, a stomach image) augmented based on the physical object Obj_r1, and the viewer can see through the transparent display 110 that the first virtual object Obj_v11 and the second virtual object Obj_v12 are superimposed on the physical object Obj_r1.

[0087] It is worth mentioning that in one embodiment of the present disclosure, the augmented image content displayed by the transparent display or whether to display the augmented image data can be determined based on the viewer's line of sight and the identification information on the viewer. The following examples are listed for illustration.

[0088] Fig.10 is a schematic diagram of an information display system according to an embodiment of the present disclosure. Fig.10The information display system 30 includes a first totem P1_1 and a second totem P1_2, a viewer totem P2_1, a viewer totem P2_2, a transparent display 110, a first image capturing device 120, a second image capturing device 140, and a processing device 130. The processing device 130 is coupled to the transparent display 110, the first image capturing device 120, and the second image capturing device 140. The operations and functions of the first totem P1_1, the second totem P1_2, the transparent display 110, the first image capturing device 120, and the processing device 130 are similar to Figure 1 and Figure 7 It should be noted that the information display system 30 includes two image capturing devices (ie, the first image capturing device 120 and the second image capturing device 140).

[0089] The viewer totem P2_1 and the viewer totem P2_2 may be located on the viewer U1 and the viewer U2, respectively. The viewer totem P2_1 and the viewer totem P2_2 may include a one-dimensional barcode, a two-dimensional barcode, or other recognizable patterns. The viewer totem P2_1 and the viewer totem P2_2 may serve as identification information of the viewer U1 and the viewer U2, respectively. In other words, the viewer totem P2_1 includes the viewer information of the viewer U1, and the viewer totem P2_2 includes the viewer information of the viewer U2.

[0090] The second image capturing device 140 is suitable for performing an image sensing function. The second image capturing device 140 may include a lens and a photosensitive element to capture a viewer image toward the viewer U1 of the transparent display 110. In one embodiment of the present disclosure, the first image capturing device 120 captures the scene behind the transparent display 110 (may include the physical object Obj_r1, the first totem P1_1, and the second totem P1_2), and the second image capturing device 140 captures the scene in front of the transparent display 110 (may include the viewer U1 and / or the viewer U2).

[0091] Fig.11 is a flow chart of an information display method according to an embodiment of the present disclosure. Fig.11 The described method flow can be combined with Fig.10 Please also refer to the information display system 30 shown in FIG. Fig.10 and Fig.11 .

[0092] In step S1110, the first image capturing device 120 captures the first totem P1_1 and the second totem P1_2 presented on the physical object Obj_r1 to obtain a totem image. In step S1120, the processing device 130 obtains the first relative position information between the transparent display 110 and the first totem P1_1 and the second relative position information between the transparent display 110 and the second totem P1_2 according to the totem image. The calculation method of the first relative position information and the second relative position information can be the same as the above description, which is not repeated here. In step S1130, the processing device 130 identifies the first totem P1_1 and the second totem P1_2 according to the totem image to obtain the first virtual object information corresponding to the first totem P1_1 and the second virtual object information corresponding to the second totem P1_2. In step S1140, the viewer totem P2_1 presented on the viewer U1 is captured by the second image capturing device 140 to obtain the viewer image. In step S1150 , the processing device 130 recognizes the viewer totem P2_1 according to the viewer image to obtain the viewer information corresponding to the viewer totem P2_1 . The viewer information is, for example, a viewer ID.

[0093] In step S1160, the processing device 130 determines whether the viewer totem P2_1 matches the first totem P1_1 or the second totem P1_2 according to the viewer information, the first virtual object information, and the second virtual object information to generate a matching result. In step S1170, the processing device 130 determines whether to display the first sub-augmented image data and the second sub-augmented image data according to the matching result. Specifically, according to the viewer information, the first virtual object information, and the second virtual object information, the processing device 130 can determine whether the viewer U1 has the authority to view the first virtual object with the first virtual object information and the second virtual object with the second virtual object information, thereby determining whether the viewer totem P2_1 matches the first totem P1_1 and / or the second totem P1_2.

[0094] In other words, if the viewer totem P2_1 matches the first totem P1_1, it means that the viewer U1 has the authority to view the first virtual object having the first virtual object information, and thus the processing device 130 determines to display the first sub-augmented image data associated with the first totem P1_1 and including the first virtual object according to the matching result. In addition, if the viewer totem P2_1 matches the second totem P1_2, it means that the viewer U1 has the authority to view the second virtual object having the second virtual object information, and thus the processing device 130 determines to display the second sub-augmented image data associated with the second totem P1_2 and including the second virtual object according to the matching result.

[0095] The above embodiment is described by taking the transparent display 110 between the viewer U1 and the physical object Obj_r1 and the matching of the viewer totem P2_1 as an example. However, when the transparent display 110 is between the viewer U2 and the physical object Obj_r1, steps S1140 to S1170 can also be performed for the viewer U2 and its viewer totem P2_2. It can be seen that in a scenario where multiple viewers use the transparent display 110, the display content of the transparent display 110 can be determined based on the current viewer who is using the transparent display 110. When the transparent display 110 moves from in front of the viewer U1 to in front of the viewer U2, the transparent display 110 can not only make the currently displayed virtual object overlap on the physical object based on the relative position information, but also can determine which virtual objects to display based on the current viewer in front of the transparent display 110.

[0096] Fig.12 is a flow chart of an information display method according to an embodiment of the present disclosure. Fig.12 The described method flow can be combined with Fig.10 Please also refer to the information display system 30 shown in FIG. Fig.10 and Fig.12 .

[0097] In step S1210, the first image capturing device 120 captures the first totem P1_1 and the second totem P1_2 presented on the physical object Obj_r1 to obtain a totem image. In step S1220, the processing device 130 obtains the first relative position information between the transparent display 110 and the first totem P1_1 and the second relative position information between the transparent display 110 and the second totem P1_2 according to the totem image. The calculation method of the first relative position information and the second relative position information can be the same as the above description, which is not repeated here. In step S1230, the processing device 130 identifies the first totem P1_1 and the second totem P1_2 according to the totem image to obtain the first virtual object information corresponding to the first totem P1_1 and the second virtual object information corresponding to the second totem P1_2. In step S1240, the viewer totem P2_1 presented on the viewer U1 is captured by the second image capturing device 140 to obtain the viewer image.

[0098] In step S1250, the processing device 130 estimates the sight line vector of the viewer U1 based on the viewer image. Specifically, the processing device 130 may estimate the sight line vector of the viewer U1 based on the eye information, facial information, and head posture information in the viewer image. In step S1260, the processing device 130 determines whether to display the augmented image data based on the sight line vector. In one embodiment of the present disclosure, the processing device 130 may determine whether the extension of the sight line vector can pass through the display plane of the transparent display 110 to determine whether to display the augmented image data.

[0099] In one embodiment of the present disclosure, the processing device 130 may determine whether the sight line vector can match the normal direction of the first totem P1_1 or the normal direction of the second totem P1_2. Specifically, when the viewer U1 looks at the first totem P1_1 or its vicinity, the normal direction of the first totem P1_1 will be substantially opposite to the direction of the sight line vector of the viewer U1. Based on this, the processing device 130 may determine whether the sight line vector can match the normal direction of the first totem P1_1 based on the three-axis rotation angle of the normal direction of the first totem P1_1 and the three-axis rotation angle of the sight line vector. If the sight line vector can match the normal direction of the first totem P1_1, the processing device 130 will decide to display the first sub-amplified data associated with the first totem P1_1. If the sight line vector can match the normal direction of the second totem P1_2, the processing device 130 will decide to display the second sub-amplified data associated with the second totem P1_2. The normal direction of the first totem P1_1 may be the rotation angle in the relative position information between the first totem P1_1 and the transparent display 110 in the aforementioned embodiment.

[0100] Fig.13A and 13B This is a schematic diagram of a scenario in which display content is determined based on a sight line vector according to an embodiment of the present disclosure. Fig.13A When the processing device 130 determines that the normal direction D1 of the first totem P1_1 matches the sight vector E1, the processing device 130 controls the transparent display 110 to display the first virtual object Obj_v11 associated with the first totem P1_1, and the augmented image data including the first virtual object Obj_v11 is obtained based on the relative position information between the transparent display 110 and the first totem P1_1. Please refer to Fig. 13B , assuming that the observer U1 changes the viewing direction, the sight vector E1 changes to the sight vector E2. When the processing device 130 determines that the normal direction D2 of the second totem P1_2 matches the sight vector E2, the processing device 130 controls the transparent display 110 to display the second virtual object Obj_v12 associated with the second totem P1_2, and the augmented image data including the first virtual object Obj_v12 is obtained according to the relative position information between the transparent display 110 and the second totem P1_2.

[0101] It should be noted that Fig.10 , Fig.11 and Fig.12 The first image capturing device 120 captures totem images from the first totem P1_1 and the second totem P1_2, and the second image capturing device 140 captures viewer images from the viewers U1 and U2 for illustration. That is, the aforementioned embodiment is described by using different image capturing devices to respectively capture totem images and viewer images from both sides of the transparent display 110, however, the present disclosure is not limited thereto. In one embodiment of the present disclosure, a totem image and a viewer image can be respectively captured from both sides of the transparent display 110 by a flippable image capturing device. In detail, in one embodiment of the present disclosure, the first image capturing device 120 has a flippable function, and the lens of the first image capturing device 120 can capture totem images and viewer images from both sides of the transparent display 110 by flipping 180 degrees.

[0102] In the disclosed embodiment, the augmented image data displayed by the transparent display can be determined according to the depth value, image coordinates, and rotation angle of the totem located on the physical object. Thus, when the viewer views the transparent display, even if the transparent display moves dynamically, the virtual object on the transparent display and the physical object behind the transparent display can be well overlapped to achieve a comfortable viewing experience.

[0103] Although the present disclosure has been disclosed as above by way of embodiments, it is not intended to limit the present disclosure. Those skilled in the art may make slight changes and modifications without departing from the spirit and scope of the present disclosure. Therefore, the protection scope of the present disclosure shall be determined by the appended claims and their equivalents.

Claims

1. An information display method, applicable to an information display system including a first image capture device and a transparent display, the method comprising: The first image capturing device captures at least one totem displayed on a physical object to obtain a totem image, wherein the physical object is located on one side of the transparent display, and the totem includes one or more one-dimensional barcodes, two-dimensional barcodes or other recognizable patterns; Acquiring relative position information between the transparent display and the at least one totem according to the totem image; Recognizing the at least one totem according to the totem image to obtain virtual object information corresponding to the at least one totem; Selecting augmented image data from an image database according to the virtual object information and the relative position information, wherein the augmented image data is generated by geometrically transforming the basic image data; as well as The augmented image data is displayed by the transparent display, wherein the augmented image data includes a virtual object augmented based on the physical object.

2. The information display method as claimed in claim 1, wherein the step of obtaining the relative position information between the transparent display and the at least one totem according to the totem image comprises: Calculating the image coordinates of the at least one totem in the image coordinate system and the depth value of the at least one totem from the first image capturing device according to the totem image; as well as The rotation angle of the at least one totem is calculated according to the totem image, wherein the relative position information includes the image coordinates, the depth value and the rotation angle.

3. The information display method as claimed in claim 2, wherein the step of calculating the image coordinates of the at least one totem in the image coordinate system and the depth value of the at least one totem from the first image capturing device according to the totem image comprises: Calculating the depth value according to internal parameters of the first image capturing device, actual size information of the at least one totem, and imaging size information of the at least one totem on the totem image; as well as Calculating the image coordinates of the at least one totem in the image coordinate system according to the world coordinates of the at least one totem and the internal parameters of the first image capturing device, The step of calculating the rotation angle of the at least one totem according to the totem image comprises: Calculating an external parameter matrix of the first image capturing device according to the world coordinates of the at least one totem, the image coordinates, and internal parameters of the first image capturing device; as well as The rotation angle is calculated according to the external parameter matrix.

4. The information display method as claimed in claim 2, wherein the step of obtaining the augmented image data according to the relative position information comprises: Acquire the augmented image data according to the image coordinates, the depth value and the rotation angle, wherein the augmented image data is generated by geometrically transforming the basic image data according to the image coordinates, the depth value and the rotation angle; as well as The augmented image data is displayed between a viewer and the physical object by the transparent display, wherein the viewer is located at the other side of the transparent display.

5. The information display method as claimed in claim 4, wherein before the step of photographing the at least one totem presented on the physical object by the first image capturing device to obtain the totem image, the method further comprises: The at least one totem presented on the physical object is photographed by the first image capturing device to obtain a calibration totem image; as well as Based on the correction totem image, a reference image coordinate, a reference depth value and a reference rotation angle associated with the basic image data are defined, The augmented image data is generated by geometrically transforming the basic image data according to the coordinate difference between the reference image coordinate and the image coordinate, the depth difference between the reference depth value and the depth value, and the angle difference between the reference rotation angle and the rotation angle.

6. The information display method as claimed in claim 1, wherein the at least one totem includes a first totem and a second totem, and the step of identifying the at least one totem according to the totem image to obtain the virtual object information corresponding to the totem comprises: The first totem and the second totem are identified according to the totem image to obtain first virtual object information corresponding to the first totem and second virtual object information corresponding to the second totem. The step of obtaining the augmented image data according to the relative position information further includes: Selecting first sub-augmented image data from the image database according to the first virtual object information and first relative position information associated with the first totem; and Second sub-augmented image data is selected from the image database according to the second virtual object information and second relative position information associated with the second totem.

7. The information display method as claimed in claim 6, wherein the step of displaying the augmented image data by the transparent display comprises: The first sub-amplified image data and the second sub-amplified image data are superimposed to generate superimposed amplified image data, and the superimposed amplified image data is displayed by the transparent display.

8. The information display method according to claim 1, wherein the information display system further comprises a second image capturing device, and the method further comprises: The second image capturing device captures the viewer totem displayed on the viewer to obtain the viewer image; Recognize the viewer totem according to the viewer image to obtain viewer information corresponding to the viewer totem; Determining whether the viewer totem matches the first totem or the second totem according to the viewer information, the first virtual object information and the second virtual object information to generate a matching result, The step of displaying the amplified image data by the transparent display comprises: Whether to display the first sub-amplified image data and the second sub-amplified image data is determined according to the matching result.

9. The information display method according to claim 1, wherein the information display system further comprises a second image capturing device, and the method further comprises: The second image capturing device captures the viewer to obtain an image of the viewer; as well as Estimate the viewer's sight vector according to the viewer image, The step of displaying the amplified image data by the transparent display comprises: Whether to display the augmented image data is determined according to the sight line vector.

10. An information display system, comprising: At least one totem, present on a physical object; Transparent displays; A first image capturing device, for capturing the at least one totem displayed on the physical object to obtain a totem image, wherein the physical object is located on one side of the transparent display, and the totem includes one or more one-dimensional barcodes, two-dimensional barcodes or other recognizable patterns; and A processing device is coupled to the transparent display and the first image capturing device, and the processing device is configured to perform the following steps: Acquiring relative position information between the transparent display and the at least one totem according to the totem image; Recognizing the at least one totem according to the totem image to obtain virtual object information corresponding to the at least one totem; Selecting augmented image data from an image database according to the virtual object information and the relative position information, wherein the augmented image data is generated by geometrically transforming the basic image data; as well as The augmented image data is displayed by the transparent display, wherein the augmented image data includes a virtual object augmented based on the physical object.

11. The information display system of claim 10, wherein the processing device is configured to perform the following steps: Calculating the image coordinates of the at least one totem in the image coordinate system and the depth value of the at least one totem from the first image capturing device according to the totem image; and The rotation angle of the at least one totem is calculated according to the totem image, wherein the relative position information includes the image coordinates, the depth value and the rotation angle.

12. The information display system of claim 11, wherein the processing device is configured to perform the following steps: Calculating the depth value according to internal parameters of the first image capturing device, actual size information of the at least one totem, and imaging size information of the at least one totem on the totem image; Calculating the image coordinates of the at least one totem in the image coordinate system according to the world coordinates of the at least one totem and the internal parameters of the first image capturing device; Calculating an external parameter matrix of the first image capturing device according to the world coordinates of the at least one totem, the image coordinates, and internal parameters of the first image capturing device; and The rotation angle is calculated according to the external parameter matrix.

13. The information display system of claim 11, wherein the processing device is configured to perform the following steps: The augmented image data is acquired according to the image coordinates, the depth value and the rotation angle, wherein the augmented image data is generated by geometrically transforming the basic image data according to the image coordinates, the depth value and the rotation angle, The transparent display displays the augmented image data between a viewer and the physical object, wherein the viewer is located at the other side of the transparent display.

14. The information display system of claim 13, wherein the processing device is configured to perform the following steps: The at least one totem presented on the physical object is photographed by the first image capturing device to obtain a calibration totem image; and Based on the correction totem image, a reference image coordinate, a reference depth value and a reference rotation angle associated with the basic image data are defined, The augmented image data is generated by geometrically transforming the basic image data according to the coordinate difference between the reference image coordinate and the image coordinate, the depth difference between the reference depth value and the depth value, and the angle difference between the reference rotation angle and the rotation angle.

15. The information display system of claim 10, wherein the at least one totem comprises a first totem and a second totem, and the processing device is configured to perform the following steps: Identifying the first totem and the second totem according to the totem image to obtain first virtual object information corresponding to the first totem and second virtual object information corresponding to the second totem; Selecting first sub-augmented image data from the image database according to the first virtual object information and first relative position information associated with the first totem; and Second sub-augmented image data is selected from the image database according to the second virtual object information and second relative position information associated with the second totem.

16. The information display system of claim 15, wherein the processing device is configured to perform the following steps: The first sub-amplified image data and the second sub-amplified image data are superimposed to generate superimposed amplified image data, and the superimposed amplified image data is displayed by the transparent display.

17. The information display system according to claim 10, further comprising a second image capturing device for capturing a viewer totem displayed on the viewer to obtain a viewer image, The processing device is configured to perform the following steps: Recognize the viewer totem according to the viewer image to obtain viewer information corresponding to the viewer totem; Determining whether the viewer totem matches the first totem or the second totem according to the viewer information, the first virtual object information, and the second virtual object information to generate a matching result; and Whether to display the first sub-amplified image data and the second sub-amplified image data is determined according to the matching result.

18. The information display system according to claim 10, further comprising a second image capturing device for capturing a viewer totem displayed on the viewer to obtain a viewer image, The processing device is configured to perform the following steps: estimating a sight line vector of the viewer according to the viewer image; and Whether to display the augmented image data is determined according to the sight line vector.

Images