Computer, methods and programs for rendering 3D objects in XR space.

Abstract

[Problem] To provide a computer, method, and program that allows easy use of a device corresponding to a second tracking system within an XR space associated with a first tracking system. [Solution] The processor of the computer according to the present invention renders a virtual device (40) within an XR space defined by a first coordinate system (step S2), calculates the first coordinate of the rendered virtual device (40) in the first coordinate system (step S5), and calculates the second coordinate in the second coordinate system for a tracker set at a device being tracked by a tracking system associated with a second coordinate system different from the first coordinate system (step S4), and, in response to a specific operation caused by the user, derives a rotation matrix and a parallel translation vector based on the first and second coordinates to convert the coordinates of the second coordinate system to the coordinates of the first coordinate system (step S6).

Description

[Technical Field]

[0001] This invention relates to a computer, method, and program for rendering 3D objects in a space (hereinafter referred to as "XR space") constructed by XR technologies such as VR (Virtual Reality), AR (Augmented Reality), MR (Mixed Reality), and SR (Substitutional Reality). [Previous Technology]

[0002] In XR technology, a tracking system is used to detect the position and orientation of various devices such as head-mounted displays and controllers. One example of such a tracking system is configured to include a plurality of cameras and to determine the position and orientation of each device in the XR space based on the images of each device captured by each camera.

[0003] Furthermore, before using the tracking system, a calibration is performed to ensure that the position and orientation of the device displayed in the XR space are consistent with the position and orientation of the actual device. Examples of such calibration are disclosed in Patent Documents 1 and 2. [Prior Art Documents] [Patent Documents]

[0004] [Patent Document 1] US Patent No. 10,127,732 [Patent Document 2] International Publication No. 2019-175582 [Summary of the Invention]

[0005] [Problem to be Solved by the Invention] In recent years, the types of devices corresponding to XR technology have become increasingly diverse. This has led to a demand for simultaneously using multiple devices manufactured by different manufacturers within a single XR space. However, the devices usable within an XR space constructed using a particular manufacturer's tracking system are limited to those corresponding to that tracking system, and it is not easy to use devices corresponding to tracking systems from other manufacturers within an XR space constructed using a particular manufacturer's tracking system.

[0006] Therefore, one object of the present invention is to provide a computer, method, and program that allows easy use of a device corresponding to a second tracking system within the XR space associated with the first tracking system. [Means for solving the problem]

[0007] The computer resulting from the present invention is a computer equipped with a processor, characterized in that: the processor renders a virtual device in an XR space defined by a first coordinate system, calculates the first coordinate of the virtual device rendered in the XR space in the first coordinate system, and calculates the second coordinate in the second coordinate system for a tracker set at a device being tracked by a tracking system connected to a second coordinate system different from the first coordinate system, and, in response to a specific operation caused by the user, calculates a coordinate transformation formula based on the calculated first and second coordinates to convert the coordinates of the second coordinate system to the coordinates of the first coordinate system.

[0008] The method resulting from the present invention includes the steps of: rendering a virtual device in an XR space defined by a first coordinate system; and calculating the first coordinate of the virtual device rendered in the XR space in the first coordinate system, and calculating the second coordinate in the second coordinate system for a tracker set at a device being tracked by a tracking system associated with a second coordinate system different from the first coordinate system, and, in response to a specific operation caused by a user, calculating a coordinate transformation formula based on the calculated first and second coordinates to convert the coordinates of the second coordinate system to the coordinates of the first coordinate system.

[0009] The program resulting from this invention is a program for causing a computer to perform the following steps: rendering a virtual device within an XR space defined by a first coordinate system; calculating the first coordinate of the virtual device rendered in the aforementioned XR space in the aforementioned first coordinate system, and calculating the second coordinate in the aforementioned second coordinate system for a tracker located at a device tracked by a tracking system configured in a second coordinate system different from the aforementioned first coordinate system, and, in response to a specific operation caused by the user, calculating a coordinate transformation formula based on the calculated first and second coordinates to convert the coordinates of the aforementioned second coordinate system to the coordinates of the aforementioned first coordinate system. [Effects of the Invention]

[0010] According to the present invention, it becomes possible to provide a computer, method and program that can be easily used in the XR space associated with the first tracking system and the device corresponding to the second tracking system.

Implementation Method

[0012] Hereinafter, with reference to the attached drawings, the embodiments of the present invention will be described in detail.

[0013] Figure 1 is a schematic block diagram showing the functional blocks of the 3D object rendering system 1 as described in the first embodiment of the present invention. Figure 2 is a diagram showing the overall structure of the 3D object rendering system 1.

[0014] First, referring to Figure 1, the 3D object rendering system 1 is composed of a computer 10, cameras 11a and 11b, a head-mounted display 12, cameras 13a to 13c, a combination device 14, and controllers C1 and C2. Furthermore, the computer 10 is functionally composed of an application unit 20, an XR system unit 21, a tracking processing unit 22, a device information acquisition unit 23, and a coordinate transformation processing unit 24.

[0015] Cameras 11a and 11b, head-mounted display 12, connecting device 14, and controller C1 are configured to communicate directly with computer 10 via wired or wireless means. Furthermore, cameras 13a-13c and controller C2 are configured to communicate directly with connecting device 14 via wired or wireless means, and are also configured to communicate with computer 10 via connecting device 14. Cameras 11a and 11b and XR system unit 21 constitute a tracking system T1 for tracking (detecting their position and tilt) head-mounted display 12 and controller C1, and cameras 13a-13c, connecting device 14, and tracking processing unit 22 constitute a tracking system T2 for tracking (detecting their position and tilt) controller C2.

[0016] Next, referring to Figure 2, the 3D object rendering system 1 is further configured to include a position detection device 15. The position detection device 15 has a touch surface and is a device capable of detecting the position of an indicator on the touch surface. In one example, the position detection device 15 is a digitizer connected to the computer 10 via wired or wireless connection, and is configured to sequentially supply the detected position of the indicator to the computer 10. Furthermore, although the position detection device 15 and the computer 10 are depicted as separate devices in Figure 2, the position detection device 15 can also be a device embedded within the computer 10. In this case, the display of the computer 10 can also function as a touch surface.

[0017] Cameras 11a, 11b, and 13a-13c are configured to photograph the space above the top of the desk where the user is seated. More specifically, cameras 11a and 11b are positioned above each end of one side of the desk, which is the deepest side from the user's perspective. Camera 13a is positioned above the center of one side of the desk, which is the deepest side from the user's perspective. Cameras 13b and 13c are positioned above the center of each side of the desk, which is closer to the user's position. The connecting device 14 is concealed within camera 13a.

[0018] Controllers C1 and C2 are respectively configured to be handheld devices for use by a user. Also, the head-mounted display 12 is a display device that is mounted on the user's head and is configured to display 3D images by reflecting different images to the left and right eyes.

[0019] The position and tilt of the controller C1 and the head-mounted display 12 are detected by the tracking system T1 shown in FIG. 1. Specifically, the XR system unit 21 constituting the tracking system T1 detects the individual positions and tilts of the controller C1 and the head-mounted display 12 based on the images captured by the cameras 11a and 11b. The position detected by the XR system unit 21, as an example, is represented by coordinates in a coordinate system with a specific position of the head-mounted display 12 as the origin (the coordinate system shown in FIG. 2 with coordinate axes X1, Y1, Z1, hereinafter referred to as "coordinate system 1"). The tilt is represented by quaternions representing rotation in coordinate system 1.

[0020] On the other hand, the position and tilt of the controller C2 are detected by the tracking system T2 shown in FIG1. ​​Although the details will be described later, a plurality of trackers (for example, a plurality of points) are provided on the surface of the controller C2. The tracking processing unit 22 constituting the tracking system T2 identifies at least three positions of these trackers based on the images captured by the cameras 13a to 13c, and thereby detects the position and tilt of the controller C2. The position detected by the tracking processing unit 22 is, for example, represented by coordinates in a coordinate system with the specific position of the connecting device 14 as the origin (the coordinate system shown in FIG2 with coordinate axes X2, Y2, Z2, hereinafter referred to as "coordinate system 2"). The tilt is represented by a rotation matrix representing the rotation in coordinate system 2. However, just as the tilt detected by the XR system unit 21 can be represented by a quaternion representing rotation in coordinate system 2, the tilt of controller C2 can also be represented by the tilt of controller C2.

[0021] In the example of Figure 2, computer 10 is constructed by a laptop computer placed in the center of a desk. However, it is not necessary to place computer 10 in the center of the desk; it can be placed in a position that allows communication with cameras 11a and 11b, head-mounted display 12, connecting device 14, and controller C1. Furthermore, computer 10 can be constructed by various types of computers, such as desktop computers, tablet computers, smartphones, and server computers, in addition to laptop computers.

[0022] Figure 3 is a diagram showing the basic hardware configuration of the computer 10. As shown in the figure, the computer 10 is configured to have a structure in which the processor 101, memory device 102, communication device 103, input device 104, and output device 105 are interconnected via bus 106.

[0023] The processor 101 is a central processing unit that reads and executes programs stored in the memory device 102. The application unit 20, XR system unit 21, tracking processing unit 22, device information acquisition unit 23, and coordinate conversion processing unit 24 shown in FIG1 are implemented by the processor 101 reading and executing programs stored in the memory device 102. The processor 101 is configured to communicate with various parts of the server via the bus 106, and to control each part and process data stored in the memory device 102 in accordance with the description of the executed program.

[0024] The memory device 102 is a device for temporarily or continuously storing various programs and data. The memory device 102 is usually constructed by combining multiple memory devices, such as a main memory device composed of DRAM (Dynamic Random Access Memory) or SRAM (Static Random Access Memory) and an auxiliary memory device composed of hard disk or SSD (Solid State Drive).

[0025] Communication device 103 is a device that, under the control of processor 101, enables communication between itself and external communication devices (including cameras 11a and 11b, head-mounted display 12, connecting device 14, and controller C1 shown in FIG. 1). The communication method performed by communication device 103 is not particularly limited, but as an example, wired or wireless WAN (Wide Area Network) or LAN (Local Area Network), or short-range wireless communication such as Bluetooth (registered trademark) can be listed.

[0026] Input device 104 is a device for receiving input from the user and includes various input methods such as a mouse, keyboard, and touch panel. The user input received by input device 104 is transmitted to processor 101 via bus 106. Output device 105 is a device for outputting to the user in accordance with the control of processor 101 and includes various output methods such as a display and speaker.

[0027] Referring again to Figure 1, the XR system unit 21 is software used to work in conjunction with the cameras 11a, 11b, the head-mounted display 12 and the controller C1 to realize the XR space. In addition to the functions of detecting the position and tilt of the head-mounted display 12 and the controller C1 as described above, it also has the functions of obtaining the operating status of the operation switches set on the surface of the controller C1 and rendering the XR space on the display surface of the head-mounted display 12 based on the instructions of the application unit 20.

[0028] The application unit 20, for example, is an application corresponding to XR, and has the function of providing users with various experiences in the XR space through the XR system unit 21. As one example, the application unit 20 is sold as a single unit together with the XR system unit 21, cameras 11a and 11b, head-mounted display 12, and controller C1. The application unit 20 constructs the XR space based on a program pre-stored in the memory device 102, and renders a 3D object representing the controller C1 in the XR space based on the position and tilt of the controller C1 detected by the XR system unit 21. Furthermore, it controls the XR space in response to the operating state of the operation switch obtained by the XR system unit 12.

[0029] The application unit 20 supplies the controlled XR space information to the XR system unit 21. The XR system unit 21 sets the user's viewpoint in the XR space based on the detected position and tilt of the head-mounted display 12, and supplies the image seen from this viewpoint to the head-mounted display 12. By displaying the supplied image on the head-mounted display 12, the user can have various experiences in the XR space.

[0030] The tracking processing unit 22 is configured to have the functions of detecting the position and tilt of the controller C2 as described above, and to obtain the operating status of the operating switches provided on the surface of the controller C2. The data detected or obtained by the tracking processing unit 22 is supplied to the application unit 20 via the device information acquisition unit 23 and the coordinate conversion processing unit 24.

[0031] The device information acquisition unit 23 is software created using software development tools provided by the manufacturer of the application unit 20, and it functions to supply the data detected or acquired by the tracking processing unit 22 to the application unit 20. The coordinate transformation processing unit 24 is plug-in software for the application unit 20, and it functions to convert the position and tilt (position and tilt in coordinate system 2) supplied from the tracking processing unit 22 into position and tilt (position and tilt in coordinate system 1) in XR space. The specific method of this transformation (coordinate transformation type) is determined by the correction processing performed by the application unit 20 and the device information acquisition unit 23. Details of this correction processing will be explained in detail later.

[0032] Figure 4(a) shows the appearance of the controller C2, and Figure 4(b) shows the internal structure of the controller C2. First, referring to Figure 4(a), the controller C2 is a device with a handle C2b attached to the central part of the pen C2a. As shown in Figure 2, the user uses the controller C2 while holding the handle C2b.

[0033] The controller C2 is configured as shown in FIG4(b) to include a control unit 30, a core 31, a pen tip electrode 32, pressure sensors 33 and 34, a battery 35, a wireless communication unit 36, an operation switch 37, and a plurality of light-emitting units 38.

[0034] The control unit 30 is a microprocessor that controls the entire controller C2. The core 31 is a rod-shaped object arranged along the pen axis of the pen C2a, and its front end forms the pen tip of the pen C2a. The end of the core 31 abuts against the pressure sensor 33. The pressure sensor 33 uses this abutment to obtain the pressure applied to the pen tip. The control unit 30 is configured to obtain the pen pressure as the pressure obtained by the pressure sensor 33. The pressure sensor 34 is provided on the surface of the controller C2 and is a sensor that obtains the force of the user gripping the controller C2. The control unit 30 is configured to obtain the pen pressure as the force obtained by the pressure sensor 34. In this way, the control unit 30 can obtain the pen pressure even when the controller C2 is used without the pen tip in contact with a hard surface such as the touch surface.

[0035] The pen tip electrode 32 is a conductor disposed near the front end of the core 31 and electrically connected to the control unit 30. The control unit 30 is configured to send and receive signals bidirectionally with the position detection device 15 via the pen tip electrode 32, or to send signals to the position detection device 15. The position detection device 15 is configured to obtain the position of the controller C2 on the touch surface using signals received from the controller C2, and to send information representing the obtained position to the computer 10 sequentially. The signals sent by the controller C2 to the position detection device 15 may include a pen pressure value representing the pen pressure obtained by the control unit 30 from the pressure sensor 33.

[0036] Furthermore, the control unit 30 is configured to, in addition to transmitting signals to the position detection device 15, also transmit a signal representing the pen pressure value obtained from the pressure sensor 34 to the connection device 14 via the wireless communication unit 36. The connection device 14 transmits a signal representing the pen pressure value obtained from the pressure sensor 34 to the computer 10. Therefore, even when the controller C2 is used in the air, pen pressure can be generated.

[0037] Battery 35 provides the operating power (including the power required to make the light-emitting part 38 light up) to the controller C2. Wireless communication unit 36 ​​is a communication device for communicating with the connecting device 14 shown in FIG1 via short-range wireless communication such as Bluetooth (registered trademark). Control unit 30 communicates with the connecting device 14 via the wireless communication unit 36.

[0038] The operation switch 37 is configured to be operated ON / OFF by a user, and is disposed on the surface of the grip C2b, for example, as illustrated in FIG. 4(a). However, the placement of the operation switch 37 is not particularly limited as long as it is a place where the user can operate it. The tracking processing unit 22 is configured to obtain the operating state of the operation switch 37.

[0039] The light-emitting part 38 is, for example, an LED (Light Emitting Diode) that emits light in the infrared region, and is disposed at various locations on the surface of the controller C2 as illustrated in FIG. 4(a). The light-emitting part 38 constitutes the tracker described above. The tracking processing unit 22 identifies the positions of at least three light-emitting parts 38 based on the images captured by the cameras 13a to 13c, and thereby detects the position and tilt of the controller C2. It is also possible to provide at least eight light-emitting parts 38 on the controller C2 in such a way that even when part of the controller C2 is covered by the user's hand, the tracking processing unit 22 can identify the positions of at least three light-emitting parts 38, and the tracking processing unit 22 can identify the positions of at least three light-emitting parts 38 without being affected by the angle of the controller C2.

[0040] Hereinafter, the correction processing performed on the application unit 20 and the device information acquisition unit 23 will be described in detail.

[0041] Figure 5 is a flowchart illustrating the processing flow of the correction process performed by the application unit 20 and the device information acquisition unit 23. In this process, firstly, the application unit 20 receives the correction execution instruction (step S1). In one example, this instruction is performed by the user using the controller C1 in the XR space. Next, the application unit 20 renders a virtual device 40 representing the controller C2 in the XR space defined by coordinate system 1 (step S2). The position of the rendered virtual device 40 in coordinate system 1 can be predetermined.

[0042] Figure 6 is an example of the display of the virtual device 40 rendered in step S2. In this example, the virtual device 40 is displayed on the head-mounted display 12. As shown in the figure, the virtual device 40 has the same shape as the controller C2. When the XR space is a VR space, the controller C2 shown in the figure is not visible to the user wearing the head-mounted display 12. The user moves the hand holding the controller C2 and uses tactile sensation to align the controller C2 with the position of the virtual device 40 rendered in the XR space. Then, in this state, the operation switch 37 is pressed.

[0043] Hereinafter, the orientation of the virtual device 40 rendered in step S2 is ideally set in such a way that the part corresponding to pen C2a is horizontal or vertical. By setting it in this way, compared to the case where the part corresponding to pen C2a is tilted, the alignment performed by the user becomes easier, and therefore, the error of the rotation matrix A described later can be reduced.

[0044] Returning to Figure 5. During the period when the virtual device 40 is being displayed, the device information acquisition unit 23 waits for a specific operation performed by the user (step S3). One example of such a specific operation is pressing the operation switch 37. After detecting the specific operation, the device information acquisition unit 23 performs a series of processes, including "cooperating with the application unit 20 to calculate the coordinate transformation formula (steps S4 to S6), and setting the calculated coordinate transformation formula in the coordinate transformation processing unit 24 (step S7)".

[0045] To be more specific, the device information acquisition unit 23 first identifies the coordinates VHP in coordinate system 2 for at least three of the plurality of trackers (light-emitting units 38) set at the controller C2 (step S4). Furthermore, the device information acquisition unit 23 causes the application unit 20 to identify the coordinates VUP in coordinate system 1 for each of the at least three trackers located on the virtual device 40 (step S5), and obtains the identified coordinates VUP.

[0046] Next, the device information acquisition unit 23 substitutes the acquired coordinates VUP and VHP (3 in total) into the following formula (1) to derive the rotation matrix A and the parallel translation vector B (step S6). Then, as a coordinate transformation formula including the derived rotation matrix A and the parallel translation vector B, it is set at the coordinate transformation processing unit 24 (step S7), and the processing ends. Then, the coordinate transformation processing unit 24 uses the set rotation matrix A and the parallel translation vector B to perform the conversion of the position and tilt provided by the tracking processing unit 22 into the position and tilt in XR space. VUP = A × VHP + B ・・・(1)

[0047] As explained above, the computer 10 according to this embodiment is capable of calculating and setting a coordinate transformation formula (specifically, rotation matrix A and parallel translation vector B) to convert the coordinates from coordinate system 2 to coordinate system 1 in response to user operations such as "the user wearing the head-mounted display 12 moves the hand holding the controller C2 and aligns the position of the controller C2 with the virtual device 40 displayed in the XR space, and then presses the operation switch 37". Therefore, it is possible to use the controller C2, which corresponds only to the tracking system T2, in the XR space constructed using the tracking system T1.

[0048] Next, the 3D object rendering system 1 resulting from the second embodiment of the present invention will be described. In this embodiment, the controller C2 is equipped with an inertial measurement unit (IMU), and the tracking system T2 detects the tilt of the controller C2 based on the measurement results of this inertial measurement unit. In this respect, it differs from the first embodiment. Hereinafter, the differences will be described in detail.

[0049] Figure 7 is a diagram showing the internal structure of the controller C2 according to this embodiment. As can be understood by comparing this figure with Figure 4(b), the controller C2 according to this embodiment is equipped with an inertial measurement device 39, which is different from the controller C2 according to the first embodiment. The inertial measurement device 39 is a device that integrates a 3-axis gyroscope and a 3-direction accelerometer, and performs the function of detecting the angle and acceleration of the controller C2 along 3 axes. The tracking processing unit 22 according to this embodiment is configured to detect the tilt of the controller C2 in the coordinate system 2 based on the angle and acceleration measured by the inertial measurement device 39. Specifically, this tilt is detected in the form of a 3×3 attitude matrix.

[0050] Figure 8 is a flowchart showing the processing flow of the correction processing performed by the application unit 20 and the device information acquisition unit 23 due to this embodiment. First, the application unit 20 receives the correction execution instruction (step S10), which is the same as in the first embodiment. The application unit 20 that receives the correction execution instruction renders the virtual device 40 representing the controller C2 in the XR space, just as in the first embodiment, and then renders tilt display information 41 representing the tilt of the virtual device 40 in coordinate system 1 (step S11).

[0051] Figure 9 is a diagram showing the virtual device 40 and tilt display information 41 rendered in step S11. The tilt display information 41 is rendered in XR space by means of a cube shape arranged to surround the virtual device 40. Ideally, the size of the tilt display information 41 is set to be sufficiently large relative to the virtual device 40, so that it can fall sufficiently within the user's field of vision.

[0052] Returning to Figure 8. Next, the device information acquisition unit 23 acquires the attitude matrix of the controller C2 based on the measurement results of the inertial measurement device 39 (step S12). Next, the application unit 20 renders tilt display information 42 in XR space based on the attitude matrix acquired by the device information acquisition unit 23 (step S13).

[0053] Referring again to Figure 9, tilt display information 42 is also shown in that figure. For ease of explanation, the tilt display information 42 is depicted with a dashed line in that figure; however, the actual tilt display information 42 is displayed in a way that allows the user to visually recognize it within the XR space. The application unit 20 renders the tilt display information 42 based on the position of the virtual device 40 within the XR space. Therefore, by aligning the tilt display information 42 with the tilt display information 41, the user can align the tilt of the controller C2 with the tilt of the virtual device 40. Thus, compared to relying solely on the virtual device 40 for alignment, the alignment of the controller C2 and the virtual device 40 can be performed with high precision.

[0054] Returning to Figure 8. During the period when the virtual device 40 is being displayed, the device information acquisition unit 23 waits for a specific operation performed by the user (step S14). The specific operation is the same as in the first embodiment, which is the pressing operation of the operation switch 37. After detecting the specific operation, the device information acquisition unit 23 performs a series of processes, including "cooperating with the application unit 20 to calculate the coordinate transformation formula (steps S15 to S20), and setting the calculated coordinate transformation formula in the coordinate transformation processing unit 24 (step S21)".

[0055] To be more specific, the device information acquisition unit 23 first obtains the attitude matrix VHR of the controller C2 based on the measurement results of the inertial measurement device 39 (step S15), and then obtains the attitude matrix VUR of the virtual device 40 from the application unit 20 (step S16). Afterwards, the rotation matrix A is derived by substituting the two obtained attitude matrices into the following equation (2) (step S17). VUR=A×VHR ​​・・・(2)

[0056] Next, the device information acquisition unit 23 identifies the coordinates VHP in coordinate system 2 for at least one of the plurality of points (light-emitting parts 38) set at the controller C2 (step S18). Furthermore, the device information acquisition unit 23 causes the application unit 20 to identify the coordinates VUP in coordinate system 1 for the same point of the virtual device 40 (step S19), and obtains the identified coordinates VUP. Then, by substituting the obtained coordinates VHP, VUP, and the rotation matrix A derived in step S17 into the above equation (1), the parallel translation vector B is derived (step S20).

[0057] Subsequently, the device information acquisition unit 23 sets the derived rotation matrix A and parallel translation vector B as coordinate transformation formulas at the coordinate transformation processing unit 24 (step S21) and ends the processing. Afterwards, the coordinate transformation processing unit 24 uses the set rotation matrix A and parallel translation vector B to perform the conversion of the position and tilt provided by the tracking processing unit 22 into position and tilt in XR space.

[0058] As explained above, in the case of the computer 10 according to this embodiment, since the tilt display information 41 and 42 are rendered in XR space, the user can align the positions of the controller C2 and the virtual device 40 with higher accuracy compared to the first embodiment. Therefore, it is possible to calculate the coordinate transformation formula (specifically, the rotation matrix A and the parallel translation vector B) for converting the coordinates caused by coordinate system 2 to the coordinates caused by coordinate system 1 with higher accuracy, and set it at the coordinate transformation processing unit 24.

[0059] The above description is based on the ideal embodiment of the present invention. However, the present invention is not limited to these embodiments. Of course, the present invention can be implemented in various forms without departing from its spirit.

[0060] For example, in the above embodiment, although the description is directed to a 3D object rendering system 1 that uses a 3D object shape tracking system based on camera images to detect the shape of 3D objects, the present invention can also be applied to a 3D object rendering system that uses a laser shape tracking system. [Simplified Explanation of the Diagram]

[0011] [Figure 1] is a schematic block diagram showing the functional blocks of the 3D object rendering system 1 as described in the first embodiment of the present invention. [Figure 2] is a diagram showing the overall structure of the 3D object rendering system 1. [Figure 3] is a diagram showing the basic hardware structure of the computer 10. [Figure 4] (a) is a diagram showing the appearance of the controller C2, and (b) is a diagram showing the internal structure of the controller C2. [Figure 5] is a flowchart showing the processing flow of the correction processing performed on the application unit 20 and the device information acquisition unit 23. [Figure 6] is a diagram showing the virtual device 40 displayed at step S2 in Figure 5. [Figure 7] is a diagram showing the internal structure of the controller C2 as described in the second embodiment of the present invention. [Figure 8] is a flowchart showing the processing flow of the correction processing performed on the application unit 20 and the device information acquisition unit 23. [Figure 9] is a diagram showing the virtual device 40 and tilted display information 41 displayed at step S11 in Figure 8.

Claims

1. A computer having a processor, characterized in that: the processor calculates first information representing the tilt of a 3D object being rendered in the XR space in a first coordinate system defined for the coordinates of a first device tracked by a first tracking system in XR space; and calculates second information representing the tilt of a second device tracked by a second tracking system different from the first tracking system in XR space in a second coordinate system different from the first coordinate system, based on the measurement results of an inertial measurement device included in the second device. The processor controls a display by displaying the first information and the second information in the XR space.

2. The computer described in request item 1, wherein, The aforementioned processor controls the display by displaying the first information and the second information together with the aforementioned 3D object within the aforementioned XR space.

3. The computer described in claim 2, wherein, The aforementioned first piece of information is displayed around the aforementioned 3D object as information representing tilt.

4. The computer described in Request 1, wherein, The aforementioned second information is displayed in a manner that overlaps with the aforementioned first information within the aforementioned XR space.

5. The computer described in request item 1, wherein, The aforementioned 3D object is a 3D object representing a virtual device.

6. The computer described in Request 1, wherein, The aforementioned processor, in response to a specific operation caused by the user, calculates a rotation matrix based on the aforementioned first information and the aforementioned second information to convert the tilt of the aforementioned second coordinate system into the tilt of the aforementioned first coordinate system.

7. The computer described in claim 6, wherein, The aforementioned specific operation refers to pressing the operation switch located at the aforementioned second device.

8. The computer described in claim 6, wherein, The aforementioned processor derives parallel translation vectors based on the coordinates of at least three of the trackers located at the aforementioned second device in the aforementioned second coordinate system, and the coordinates of each position on the aforementioned 3D object corresponding to the position of each of the aforementioned at least three trackers in the aforementioned first coordinate system, and calculates coordinate transformation formulas based on the aforementioned rotation matrix and the aforementioned parallel translation vectors.

9. A method for rendering a 3D object in XR space, characterized by comprising: a step of calculating first information representing the tilt of the 3D object being rendered in the XR space in a first coordinate system defined for the coordinates of a first device tracked by a first tracking system in the XR space; and a step of calculating second information representing the tilt of the second device in the XR space in a second coordinate system defined for the coordinates of a second device tracked by a second tracking system different from the first tracking system in the XR space, based on the measurement results of an inertial measurement device included in the second device; and a step of controlling a display in a manner that displays the first information and the second information in the XR space.

10. A program characterized in that it is used to cause a computer to perform the following steps: calculating first information representing the tilt of a 3D object rendered in the XR space in a first coordinate system defined for the coordinates of a first device tracked by a first tracking system in XR space; and calculating second information representing the tilt of a second device tracked by a second tracking system different from the first tracking system in XR space in a second coordinate system different from the first coordinate system, based on the measurement results of an inertial measuring device included in the second device.

Images