Data processing system, method for determining coordinates, and related products

Abstract

Embodiments of the present disclosure provide a data processing system, a method for determining coordinates, and related products. The method comprises: receiving a plurality of positioning data, wherein the plurality of positioning data corresponds to device positions of a plurality of positioning devices in a real world, and the positioning devices comprise a first positioning device and a second positioning device; in response to determining that the first positioning device is selected as a reference point of a coordinate system of a virtual world, determining coordinates of the second positioning device in the coordinate system of the virtual world based on a relative position between a device position of the first positioning device and a device position of the second positioning device.

Description

Technical Field

[0001] The present invention relates to a data processing system, a method for determining coordinates, and a computer-readable storage medium. Background Technology

[0002] When content creators use cameras to capture content for virtual production (e.g., the creation of virtual reality content), accurately determining the coordinates of the camera and / or other tracked objects in the virtual world is crucial. Therefore, it is important for those skilled in the art to design suitable solutions for accurately determining the coordinates of cameras and / or other tracked objects in the virtual world. Summary of the Invention

[0003] Therefore, the present invention relates to a data processing system that can be used to solve the above-mentioned technical problems, a method for determining coordinates, and a computer-readable storage medium.

[0004] Embodiments of the present invention provide a data processing system including a data processing device. The data processing device performs the following actions: receiving multiple positioning data, wherein the multiple positioning data correspond to the device locations of multiple positioning devices in the real world, and the positioning devices include a first positioning device and a second positioning device; in response to determining that the first positioning device is selected as a reference point in the coordinate system of the virtual world, determining the coordinates of the second positioning device in the coordinate system of the virtual world based on the relative position between the device locations of the first positioning device and the second positioning device.

[0005] Embodiments of the present invention provide a method for determining coordinates, applicable to a data processing apparatus. The method includes: receiving multiple positioning data, wherein the multiple positioning data correspond to the device positions of multiple positioning devices in the real world, and the positioning devices include a first positioning device and a second positioning device; in response to determining that the first positioning device is selected as a reference point in a coordinate system of a virtual world, determining the coordinates of the second positioning device in the coordinate system of the virtual world based on the relative position between the device positions of the first positioning device and the second positioning device.

[0006] Embodiments of the present invention provide a computer-readable storage medium that records an executable computer program, which is loaded by a data processing device to perform the following steps: receiving a plurality of positioning data, wherein the plurality of positioning data corresponds to the device positions of a plurality of positioning devices in the real world, and the positioning devices include a first positioning device and a second positioning device; in response to determining that the first positioning device is selected as a reference point in a coordinate system of a virtual world, determining the coordinates of the second positioning device in the coordinate system of the virtual world based on the relative position between the device positions of the first positioning device and the device positions of the second positioning device. Attached Figure Description

[0007] The accompanying drawings are included to provide a further understanding of the invention, and are incorporated in and constitute a part of this specification. The drawings illustrate embodiments of the invention and, together with the embodiments, serve to explain the principles of the invention.

[0008] Figure 1 A schematic diagram of a data processing system according to an embodiment of the present invention is shown.

[0009] Figure 2 A flowchart illustrating a data processing method according to an embodiment of the present invention is shown.

[0010] Figure 3 A flowchart illustrating a data processing method according to an embodiment of the present invention is shown.

[0011] Figure 4 Showing the invention Figure 1 The data processing system.

[0012] Figure 5 A flowchart illustrating a method for determining coordinates according to an embodiment of the present invention is shown.

[0013] Figure 6 A user interface provided by a data processing apparatus according to an embodiment of the present invention is shown.

[0014] Explanation of icon numbers

[0015] 10: Data Processing System

[0016] 11, 11a, 11b: Signal processing devices

[0017] 12, 12a, 12b: Positioning devices

[0018] 13: Lens encoder

[0019] 14: Camera device

[0020] 15: Data processing device

[0021] 16: Terminal device

[0022] 17: Clock Source

[0023] 40: Data Processing System

[0024] 111: First data transmission port

[0025] 112: Second data transmission port

[0026] 113: First Processor

[0027] 114: Network Interface

[0028] 151: First network interface

[0029] 152: Second network interface

[0030] 153: Second Processor

[0031] 610: User Interface

[0032] 612: Recentering Function

[0033] 614a, 614b, 614c: Icons

[0034] CS: Reference clock signal

[0035] L1: Lens parameters

[0036] LC: Lens Control Signal

[0037] P1: Location data

[0038] P11: First Positioning Data

[0039] P12: Second Positioning Data

[0040] PA1: First data packet

[0041] PA2: Second data packet

[0042] PO: Camera pose

[0043] S1: Data signal

[0044] S210, S220, S230, S240, S310, S320, S330, S340, S350, S510, S520: Steps

[0045] TS: timestamp Detailed Implementation

[0046] See Figure 1 This illustrates a schematic diagram of a data processing system according to an embodiment of the present invention. Figure 1 In the process, the data processing system 10 may include a signal processing device 11, a positioning device 12, a lens encoder 13, a camera device 14, a data processing device 15, a terminal device 16, and a clock source 17.

[0047] In embodiments of the present invention, camera device 14 can be used to capture virtually created content. For example, camera device 14 can be arranged in a photography studio, which may be equipped with, for example, a green screen and / or other equipment for filming (e.g., light sources). In one embodiment, a male / female actor can stand in front of a green screen, and camera device 14 can be used to capture an image of the male / female actor in front of the green screen. Subsequently, the image area corresponding to the male / female actor can be extracted and combined with a virtual background as virtually created visual content, but the present invention is not limited thereto.

[0048] In one embodiment, the positioning device 12 may be (fixedly) mounted on the camera device 14 via, for example, screws or other adapter structures. In one embodiment, the positioning device 12 may be used to detect positioning data P1 of the positioning device 12. Since the positioning device 12 is (fixedly) mounted on the camera device 14, the positioning device 12 can move in response to movement of the camera device 14. In this case, the positioning data P1 of the positioning device 12 can be regarded as the positioning data P1 of the camera device 14.

[0049] In some embodiments, the positioning data P1 of the camera device 14 may include raw measurement data of the translational and / or rotational components (e.g., 6 degrees of freedom) of the camera device 14, but the invention is not limited thereto. In one embodiment, the positioning device 12 may be a tracker attached to the camera device 14, and the positioning data P1 of the positioning device 12 and / or the camera device 14 may be detected via, for example, an outside-in tracking mechanism or an inside-out mechanism. In the case of outside-in tracking, the environment may be provided with several base stations emitting beacons for the tracker to detect the positioning data P1 of the positioning device 12 and / or the camera device 14. In other embodiments, the positioning device 12 may use any existing positioning mechanism to determine the positioning data P1 of the positioning device 12 and / or the camera device 14.

[0050] In one embodiment, the lens encoder 13 may be disposed on the camera device 14 and connected to the lens of the camera device 14 for measuring / reading / detecting the lens parameter L1 of the lens of the camera device 14.

[0051] In one embodiment, the lens parameter L1 obtained by the lens encoder 13 may include at least one count value, which corresponds to at least one of the aperture, focus, and focal length of the lens of the camera device 14. In one embodiment, the count value may be mapped by, for example, the terminal device 16 to the actual set values ​​of the aperture, focus, and / or focal length of the lens of the camera device 14, but the invention is not limited thereto.

[0052] exist Figure 1In this configuration, the signal processing device 11 may include a first data transmission port 111, a second data transmission port 112, a first processor 113, and a network interface 114. The first data transmission port 111 is coupled to the positioning device 12 and receives positioning data P1 from the positioning device 12. The second data transmission port 112 is coupled to the lens encoder 13 and receives lens parameters L1 from the camera device 14 from the lens encoder 13.

[0053] In other embodiments, the signal processing device 11 may include additional data transmission ports for connection to other positioning devices and / or lens encoders. For example, in one embodiment, the signal processing device 11 may include two additional data transmission ports for connecting two additional lens encoders and receiving corresponding lens parameters measured by said two additional lens encoders, but the invention is not limited thereto.

[0054] In some embodiments, the first data transmission port 111, the second data transmission port 112 (and other data transmission ports) may be plug-and-play ports / interfaces, such as Universal Serial Bus (USB) ports / interfaces, but the invention is not limited thereto.

[0055] The first processor 113 is coupled to the first data transmission port 111 and the second data transmission port 112. In various embodiments, the first processor 113 may be, for example, a general-purpose processor, a special-purpose processor, a conventional processor, a digital signal processor (DSP), multiple microprocessors, one or more microprocessors associated with a DSP core, a controller, a microcontroller, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA) circuit, any other type of integrated circuit (IC), a state machine, and similar processor.

[0056] In one embodiment, the first processor 113 encapsulates positioning data P1 and lens parameters L1 in at least one data packet. In one embodiment, the at least one data packet may include a first data packet PA1 containing lens parameters L1 and a second data packet PA2 containing positioning data P1, but the invention is not limited thereto.

[0057] exist Figure 1 In this embodiment, network interface 114 is coupled to first processor 113 and sends the at least one data packet to data processing device 15 for further processing. In embodiments of the invention, network interface 114 (and other network interfaces considered in this invention) may be, for example, an Ethernet interface, but the invention is not limited thereto.

[0058] In one embodiment, the first data packet PA1 may be a User Datagram Protocol (UDP) packet, and the first processor 113 may control the network interface 114 to send the first data packet PA1 via the UDP protocol. Furthermore, the second data packet PA2 may be a Transmission Control Protocol / Internet Protocol (TCP / IP) packet, and the first processor 113 may control the network interface 114 to send the second data packet PA2 via the Universal Serial Bus / IP (USB / IP) protocol, but the invention is not limited thereto.

[0059] In other embodiments, the first data packet PA1 and the second data packet PA2 may be implemented as packets corresponding to the designer's desired protocol.

[0060] exist Figure 1 In the process, the data processing device 15 includes a first network interface 151, a second network interface 152, and a second processor 153.

[0061] In one embodiment, the second processor 153 is coupled to the first network interface 151 and the second network interface 152, and possible implementations of the second processor 153 may refer to the description of the first processor 113, but the invention is not limited thereto.

[0062] In one embodiment, a first network interface 151 is coupled to a network interface 114 of a signal processing device 11 and receives at least one data packet (e.g., a first data packet PA1 and a second data packet PA2) from the signal processing device 11. In one embodiment, a second processor 153 may obtain positioning data P1 and lens parameters L1 from the at least one data packet. For example, the second processor 153 may obtain lens parameters L1 from the first data packet PA1 and positioning data P1 from the second data packet PA2, but the invention is not limited thereto.

[0063] In one embodiment, the second processor 153 determines the camera pose PO of the camera device 14 based on positioning data P1. In embodiments where the positioning data P1 includes raw measurement data of the translation and / or rotation components of the camera device 14, the second processor 153 may analyze / process the positioning data P1 and determine the actual translation and / or rotation components of the camera device 14 as the camera pose PO. Alternatively, the signal processing device 11 forwards raw measurement data of the camera pose PO (e.g., positioning data P1 detected by the positioning device 12) to the data processing device 15, and the data processing device 15 determines the camera pose PO by analyzing the raw measurement data; however, the invention is not limited thereto.

[0064] In one embodiment, the second processor 153 receives a reference clock signal CS from a clock source 17 coupled to the camera device 14, the data processing device 15, and the terminal device 16. In one embodiment, the reference clock signal CS may be a reference signal used for synchronization of the camera device 14, the data processing device 15, and the terminal device 16. In another embodiment, the reference clock signal CS may be a pulse train comprising multiple clock pulses, wherein the clock pulses are provided at a specific frequency. In this case, the duration between adjacent clock pulses may be the reciprocal of the specific frequency.

[0065] In one embodiment, the second processor 153 may determine a timestamp TS corresponding to a camera pose PO. For example, the second processor 153 may determine the camera pose PO and the timestamp TS in response to one of the clock pulses, wherein the timestamp TS corresponds to the time point of said one of the clock pulses. In one embodiment, the second processor 153 may use the time point of said one of the clock pulses as the timestamp TS corresponding to the camera pose PO. That is, the second processor 153 determines the camera pose PO and the corresponding timestamp TS by following the timing of a reference clock signal CS.

[0066] In one embodiment, the second processor 153 encapsulates the camera pose PO, timestamp TS, and lens parameter L1 in a data signal S1 and transmits the data signal S1 via the second network interface 152.

[0067] exist Figure 1 In this embodiment, the terminal device 16 coupled to the data processing device 15 can be any smart device and / or computer device used to present virtually created visual content. In one embodiment, the terminal device 16 can be a computer used by a content creator to edit virtually created visual content, but the invention is not limited thereto.

[0068] In one embodiment, the terminal device 16 can receive a data signal S1 from the data processing device 15. Therefore, the terminal device 16 can know the state of the lens parameter L1 and the camera pose PO at the timestamp TS from the data signal S1. Furthermore, each image captured by the camera device 14 can be marked with a corresponding timestamp by the camera device 14 based on a reference clock signal CS, and the captured images with the corresponding timestamps will be provided to the terminal device 16. In this case, the terminal device 16 can know the camera pose of the camera device 14 when capturing the image based on the timestamp of each image and each camera pose.

[0069] For example, when the terminal device 16 determines that a specific image in the images captured by the camera device 14 has the same timestamp as the timestamp TS, the terminal device 16 can determine that the camera pose PO is the camera pose of the camera device 14 when capturing the specific image.

[0070] In one embodiment, terminal device 16 can run / present a virtual scene (e.g., a virtual reality (VR) scene). In this case, when camera device 14 is used to capture some real objects (e.g., male actors / female actors) as the specific image, terminal device 16 can crop / segment an image area corresponding to the real object from the specific image and combine the cropped image area with the virtual scene to generate a virtual production video. In this embodiment, the virtual scene includes a virtual camera corresponding to camera device 14, and when capturing a specific image, the pose of the virtual camera in the virtual scene needs to be referenced to the camera pose PO of camera device 14 (e.g., rotation and / or translation). Since the solution provided by the embodiments of the present invention can obtain an accurate camera pose PO of camera device 14, the cropped image area can be accurately combined with the virtual scene, thereby improving the quality of the virtual production.

[0071] In one embodiment, the signal processing device 11 may also receive a lens control signal LC and control the lens encoder 13 accordingly to adjust the lens settings of the camera device 14. In one embodiment, the lens settings of the camera device 14 may include, for example, the aperture, focus, and / or focal length of the lens of the camera device 14, and the user may determine the lens settings via, for example, a control interface (e.g., software / hardware) of the operating terminal device 16. In this case, the terminal device 16 may control the data processing device 15 to send the lens control signal LC to the signal processing device 11 according to the user's settings. Therefore, the signal processing device 11 may control the lens encoder 13 to adjust the lens settings of the camera device 14 to correspond to the user's requirements, but the invention is not limited thereto.

[0072] See Figure 2 This document illustrates a flowchart of a data processing method according to an embodiment of the present invention. The method of this embodiment can be derived from... Figure 1 The signal processing device 11 in the middle is used to perform the operation, and the following will utilize Figure 1 The components shown are used to describe Figure 2 Details of each step.

[0073] In step S210, the signal processing device 11 receives positioning data P1 from the positioning device 12. In step S220, the signal processing device 11 receives lens parameters L1 from the camera device 14 from the lens encoder 13. In step S230, the signal processing device 11 encapsulates the positioning data P1 and the lens parameters L1 in at least one data packet (e.g., a first data packet PA1 and a second data packet PA2). In step S240, the signal processing device 11 transmits the at least one data packet. Figure 2 For details of the steps, please refer to the explanation above, which will not be repeated here.

[0074] See Figure 3 This document illustrates a flowchart of a data processing method according to an embodiment of the present invention. The method of this embodiment can be derived from... Figure 1 The data processing device 15 in the middle is used to perform the operation, and the following will utilize Figure 1 The components shown are used to describe Figure 3 Details of each step.

[0075] In step S310, the data processing device 15 receives the at least one data packet from the signal processing device 11 and obtains positioning data P1 and lens parameters L1 from the at least one data packet. In step S320, the data processing device 15 determines the camera pose PO of the camera device 14 based on the positioning data P1.

[0076] In step S330, the data processing device 15 receives the reference clock signal CS and determines the timestamp TS corresponding to the camera pose PO. In step S340, the data processing device 15 encapsulates the camera pose PO, the timestamp TS, and the lens parameter L1 in a data signal S1. In step S350, the data processing device 15 transmits the data signal S1. Figure 3 For details of the steps, please refer to the explanation above, which will not be repeated here.

[0077] As can be understood from the foregoing, embodiments of the present invention provide a simple and effective solution for accurately tracking the camera posture of a camera device while capturing content. In embodiments of the present invention, the user (e.g., a content creator) only needs to install the positioning device 12 and the lens encoder 13 onto the camera device 14, and the signal processing device 11 and the data processing device 15 can cooperate to enable the terminal device 16 to synchronize the captured image with the camera posture of the camera device 14. Therefore, the efficiency of creating virtual content can be improved.

[0078] Furthermore, embodiments of the present invention also provide a method for determining coordinates, which can be performed by the data processing system of the present invention, and its details will be described below.

[0079] See Figure 4It shows the invention Figure 1 The data processing system. Figure 4 In this embodiment, the data processing system 40 may include signal processing devices 11a and 11b, positioning devices 12a and 12b, and a data processing device 15. Each of the signal processing devices 11a and 11b may be configured to... Figure 1 The signal processing device 11 is of the same type as the positioning device 12a and 12b. Figure 1 The positioning device 12 is the same type of device as the positioning device in the middle, and Figure 4 The data processing device 15 in the middle can be used with Figure 1 The data processing device 15 is the same device as the data processing device in the system. Furthermore, the data processing system 40 may also include... Figure 1 Other devices in the invention include, for example, the lens encoder 13, the terminal device 16, and the clock source 17, but the invention is not limited thereto.

[0080] In embodiments of the present invention, similar to Figure 1 In the case where the signal processing device 11 and the positioning device 12 are mounted on the camera device 14, the signal processing device 11a and the positioning device 12a can be mounted on the corresponding camera device (not shown) or other objects / locations to be tracked. Similarly, the signal processing device 11b and the positioning device 12b can be mounted on the corresponding camera device (not shown) or other objects / locations to be tracked.

[0081] Furthermore, despite Figure 4 Only two combinations of signal processing devices and positioning devices are shown (each combination includes at least one signal processing device and one positioning device), but the data processing system 40 may include more combinations of signal processing devices and positioning devices, but the invention is not limited thereto. Furthermore, to connect to signal processing devices 11a, 11b (and other additional signal processing devices), the data processing device 15 may be provided with a corresponding number of first network interfaces 151. For example, if the data processing device 15 needs to be able to connect to up to N (N is a positive integer) signal processing devices, the data processing device 15 may be designed with N first network interfaces 151, but the invention is not limited thereto.

[0082] In some embodiments, the data processing device 15 may determine the camera pose of the camera device corresponding to the positioning devices 12a and 12b based on positioning data from the positioning devices 12a and 12b. The data processing device 15 may send the determined camera pose of the camera device to the terminal device 16, so that the terminal device 16 can create a corresponding virtual scene (e.g., combine a cropped image area corresponding to a real object with the presented virtual scene), but the present invention is not limited thereto.

[0083] See Figure 5 This document illustrates a flowchart of a method for determining coordinates according to an embodiment of the present invention. The method of this embodiment can be derived from... Figure 4 The data processing device 15 in the middle is used to perform the operation, and the following will utilize Figure 4 The components shown are used to describe Figure 5 Details of each step.

[0084] In step S510, the data processing device 15 receives multiple positioning data, wherein the multiple positioning data corresponds to the device locations of multiple positioning devices in the real world, and the positioning devices include a first positioning device and a second positioning device.

[0085] To better illustrate the concept of the present invention, positioning devices 12a and 12b will be assumed to be the first positioning device and the second positioning device, respectively, but the present invention is not limited thereto.

[0086] exist Figure 4 In this embodiment, the plurality of positioning data received by the data processing device 15 may include first positioning data P11 corresponding to positioning device 12a and second positioning data P12 corresponding to positioning device 12b. Specifically, the data processing device 15 may receive the first positioning data P11 from a signal processing device 11a connected to the positioning device 12a, and receive the second positioning data P12 from a signal processing device 11b connected to the positioning device 12b. More specifically, the signal processing device 11a may receive positioning data P11 from the positioning device 12a and send the positioning data P11 to the data processing device 15, and the signal processing device 11b may receive positioning data P12 from the positioning device 12b and send the positioning data P12 to the data processing device 15.

[0087] In other embodiments, the data processing device 15 may receive positioning data P11 and P12 directly from each of the positioning devices 12a and 12b, but the invention is not limited thereto.

[0088] As in Figure 1 As described in the embodiments, positioning data P11 may correspond to the first camera posture of the first camera device on which the signal processing device 11a and the positioning device 12a are mounted, and positioning data P12 may correspond to the second camera posture of the second camera device on which the signal processing device 11b and the positioning device 12b are mounted, but the present invention is not limited thereto.

[0089] In one embodiment, positioning devices 12a and 12b may obtain positioning data P11 and P12 through an outside-in tracking mechanism or an inside-out tracking mechanism (e.g., simultaneous localization and mapping, SLAM). In the case of outside-in tracking, positioning data P11 and P12 can be understood as characterizing the positions of positioning devices 12a and 12b relative to, for example, a base station, but the invention is not limited thereto.

[0090] In one embodiment, the data processing device 15 can determine the device positions of positioning devices 12a and 12b in the real world. In an embodiment where positioning data P11 includes raw measurement data of the translational and / or rotational components of positioning device 12a, the data processing device 15 can analyze / process the positioning data P11 to determine the actual translational and / or rotational components of positioning device 12a, which can be used to determine the device position of positioning device 12a in the real world. Similarly, in an embodiment where positioning data P12 includes raw measurement data of the translational and / or rotational components of positioning device 12b, the data processing device 15 can analyze / process the positioning data P12 to determine the actual translational and / or rotational components of positioning device 12b, which can be used to determine the device position of positioning device 12b in the real world.

[0091] In step S520, in response to determining that the first positioning device (e.g., positioning device 12a) is selected as the reference point of the coordinate system of the virtual world, the data processing device 15 determines the coordinates of the second positioning device (e.g., positioning device 12b) in the coordinate system of the virtual world based on the relative position between the device position of the first positioning device and the device position of the second positioning device.

[0092] To better illustrate the concept of this invention, the reference point of the coordinate system in the virtual world will be assumed to be the origin of the coordinate system. In other embodiments, the reference point may be any other desired point in the coordinate system.

[0093] For example, suppose the default reference point is a location in one of the base stations (i.e., the location of the base station considered as the reference point could be (0, 0, 0)), and the device locations of positioning devices 12a and 12b in the real world are (x1, y1, z1) and (x2, y2, z2), respectively. In this case, when positioning device 12a is selected as the origin of the coordinate system in the virtual world, the coordinates of positioning device 12a in the coordinate system in the virtual world could be, for example, (0, 0, 0). Based on the relative positions between the device locations of positioning devices 12a and 12b, the data processing device 15 can determine the coordinates of positioning device 12b in the coordinate system in the virtual world as, for example, (x2-x1, y2-y1, z2-z1), but the present invention is not limited thereto.

[0094] Therefore, the data processing device 15 can easily use the relative position of each of the positioning devices 12a and 12b with a base station in the real world to determine the coordinates of the positioning devices 12a and 12b in the virtual world, which increases the convenience of creating virtual productions.

[0095] In one embodiment, the data processing device 15 may also provide the coordinates of the positioning devices 12a and 12b to... Figure 1 The terminal device 16 in the invention can use this information to create visual content of better quality, but the invention is not limited thereto.

[0096] Furthermore, by using the solution described in this invention, the coordinates of positioning devices 12a and 12b in the virtual world can be easily determined even if the base station is moved. Specifically, a user may change the position of the base station in a studio based on specific requirements. In this case, the user can simply use positioning device 12a as a reference point, and the coordinates of positioning devices 12a and 12b in the virtual world can be appropriately determined even if the base station has been moved to a new location.

[0097] For example, after the base station used for outward tracking is moved, the device positions of positioning devices 12a and 12b relative to the base station may become (x1', y1', z1') and (x2', y2', z2'), respectively. However, when positioning device 12a is selected as the reference point again, the coordinates of positioning devices 12a and 12b in the virtual world will be determined as (0, 0, 0) and (x2'-x1', y2'-y1', z2'-z1'), respectively, where (x2'-x1', y2'-y1', z2'-z1') will be the same as (x2-x1, y2-y1, z2-z1). Therefore, the efficiency of creating virtual productions can be improved.

[0098] In one embodiment, in response to determining that the second positioning device (e.g., positioning device 12b) is selected as the reference point of the coordinate system in the virtual world, the data processing device 15 may also determine the coordinates of the first positioning device (e.g., positioning device 12a) in the coordinate system in the virtual world based on the relative position between the device position of the first positioning device and the device position of the second positioning device.

[0099] In this scenario, when the positioning device 12b is selected as the origin of the coordinate system in the virtual world, the coordinates of the positioning device 12b in the coordinate system of the virtual world can be, for example, (0, 0, 0). Based on the relative positions between the device positions of the positioning devices 12a and 12b, the data processing device 15 can determine the coordinates of the positioning device 12b in the coordinate system of the virtual world as, for example, (x1-x2, y1-y2, z1-z2), but the present invention is not limited thereto.

[0100] In other words, after the positioning device 12b is used as the origin, the data processing device 15 can switch to using the positioning device 12b as the origin of the coordinate system of the virtual world, and the coordinates of the positioning devices 12a and 12b will be updated accordingly.

[0101] In embodiments where the data processing system 40 includes other positioning devices, the data processing device 15 may determine the coordinates of each of the other positioning devices in a virtual world coordinate system based on the relative position between the device position of the selected positioning device as a reference point and the device positions of each of the other positioning devices, but the invention is not limited thereto.

[0102] See Figure 6 This illustrates a user interface provided by the data processing device 15 according to an embodiment of the present invention. Figure 6 In this embodiment, the data processing device 15 may provide a user interface 610, which includes a re-centering function 612. In this embodiment, the user can use the re-centering function 612 to select a desired reference point in the positioning device as a coordinate system for the virtual world.

[0103] In one embodiment, in response to the determination that the recentering function 612 is triggered, the data processing device 15 may provide multiple detected positioning devices in the user interface 610. In one embodiment, the data processing device 15 may detect a positioning device by examining positioning device information in encapsulated data / attitude data received from the positioning device, but the invention is not limited thereto. In one embodiment, the user interface 610 may only show the positioning devices currently detected by the data processing device 15.

[0104] exist Figure 6 In this context, icons 614a to 614c correspond to positioning devices 12a, 12b, and another positioning device detected by the data processing device 15, respectively. In this scenario, the user can select the desired positioning device as the reference point for the coordinate system of the virtual world.

[0105] In one embodiment, assuming the user selects the icon 614a corresponding to the positioning device 12a, the data processing device 15 may perform step S520 based on the description above, but the present invention is not limited thereto.

[0106] The present invention also provides a computer-readable storage medium for performing the data processing method and / or the method for determining coordinates. The computer-readable storage medium comprises a plurality of program instructions (e.g., setting program instructions and deploying program instructions) implemented therein. These program instructions can be loaded into and executed by a data processing apparatus and a signal processing apparatus, thereby performing the aforementioned data processing method and / or the method for determining coordinates, as well as the functions of the data processing apparatus and the signal processing apparatus.

[0107] In summary, embodiments of the present invention provide a simple and effective solution for accurately tracking the camera posture of a camera device while capturing content. In these embodiments, the user (e.g., a content creator) only needs to install a positioning device and / or lens encoder onto the camera device, and the signal processing and data processing devices can cooperate to synchronize the captured image with the camera posture of the camera device. Therefore, the terminal device can more accurately combine the cropped image area (e.g., the image area corresponding to a real object such as a male or female actor) with the presented virtual scene based on the camera posture of the camera device, thus improving the quality and efficiency of virtual production.

[0108] Furthermore, embodiments of the present invention also provide a solution for determining the coordinates of a positioning device in a virtual world coordinate system when one of the positioning devices is selected as a reference point in the virtual world. Therefore, embodiments of the present invention can easily determine the coordinates of the positioning device in the virtual world using the relative position of each of the positioning devices to a base station in the real world, which increases the convenience of creating virtual content.

[0109] It will be apparent to those skilled in the art that various modifications and variations can be made to the structure of the invention without departing from the scope or spirit of the invention. In summary, this invention is intended to cover modifications and variations of the invention that fall within the scope of the following claims and their equivalents.

Claims

1. A data processing system, characterized in that, include: Data processing device, configured to perform: Receive multiple positioning data, wherein the multiple positioning data corresponds to the device locations of multiple positioning devices in the real world, and the positioning devices include a first positioning device and a second positioning device, wherein the multiple positioning data includes first positioning data corresponding to the first positioning device and second positioning data corresponding to the second positioning device; In response to determining that the first positioning device is selected as a reference point in the coordinate system of the virtual world via a user interface, the coordinates of the second positioning device in the coordinate system of the virtual world are determined based on the relative position between the device position of the first positioning device and the device position of the second positioning device.

2. The data processing system according to claim 1, wherein the reference point of the coordinate system in the virtual world is the origin of the coordinate system.

3. The data processing system according to claim 1, wherein the data processing device performs: Receive the first positioning data from a first signal processing device connected to the first positioning device; The second positioning data is received from a second signal processing device connected to the second positioning device.

4. The data processing system according to claim 3 further includes: The first signal processing device receives the first positioning data from the first positioning device and sends the first positioning data to the data processing device; The second signal processing device receives the second positioning data from the second positioning device and sends the second positioning data to the data processing device.

5. The data processing system according to claim 4, wherein the first positioning data corresponds to a first camera pose of the first camera device, and the second positioning data corresponds to a second camera pose of the second camera device.

6. The data processing system according to claim 5, wherein the first signal processing device, together with the first positioning device, is mounted on the first camera, and the second signal processing device, together with the second positioning device, is mounted on the second camera.

7. The data processing system according to claim 1, wherein the data processing device further performs: In response to determining that the second positioning device is selected as the reference point of the coordinate system of the virtual world, the coordinates of the first positioning device in the coordinate system of the virtual world are determined based on the relative position between the device position of the first positioning device and the device position of the second positioning device.

8. The data processing system according to claim 7, wherein the data processing device further performs: The user interface displays the recentering function; In response to determining that the recentering function is triggered, a plurality of detected positioning devices are provided in the user interface, wherein the detected positioning devices include the first positioning device and the second positioning device.

9. The data processing system of claim 1, wherein each location data is detected by using an outside-in tracking mechanism.

10. The data processing system of claim 1, wherein in response to determining that the first positioning device is selected as the reference point of the coordinate system of the virtual world, the data processing device further performs: The coordinates of each of the other positioning devices in the virtual world are determined based on the relative position between the device position of the first positioning device and the device position of each of the other positioning devices.

11. A method for determining coordinates, applicable to a data processing device, characterized in that, include: Receive multiple positioning data, wherein the multiple positioning data corresponds to the device locations of multiple positioning devices in the real world, and the positioning devices include a first positioning device and a second positioning device, wherein the multiple positioning data includes first positioning data corresponding to the first positioning device and second positioning data corresponding to the second positioning device; In response to determining that the first positioning device is selected as a reference point in the coordinate system of the virtual world via a user interface, the coordinates of the second positioning device in the coordinate system of the virtual world are determined based on the relative position between the device position of the first positioning device and the device position of the second positioning device.

12. The method for determining coordinates according to claim 11, wherein the reference point of the coordinate system in the virtual world is the origin of the coordinate system.

13. The method for determining coordinates according to claim 11, wherein the step of receiving the plurality of positioning data includes: Receive the first positioning data from a first signal processing device connected to the first positioning device; The second positioning data is received from a second signal processing device connected to the second positioning device.

14. The method for determining coordinates according to claim 13, wherein the first positioning data corresponds to a first camera pose of a first camera device, and the second positioning data corresponds to a second camera pose of a second camera device.

15. The method for determining coordinates according to claim 11, wherein after the step of determining the coordinates of the second positioning device in the coordinate system of the virtual world, the method further comprises: In response to determining that the second positioning device is selected as the reference point of the coordinate system of the virtual world, the coordinates of the first positioning device in the coordinate system of the virtual world are determined based on the relative position between the device position of the first positioning device and the device position of the second positioning device.

16. The method for determining coordinates according to claim 15, wherein after the step of determining the coordinates of the second positioning device in the coordinate system of the virtual world, the method further comprises: The user interface displays the recentering function; In response to determining that the recentering function is triggered, a plurality of detected positioning devices are provided in the user interface, wherein the detected positioning devices include the first positioning device and the second positioning device.

17. The method for determining coordinates according to claim 11, wherein each positioning data is detected by using an outside-to-in tracking mechanism.

18. The method for determining coordinates according to claim 11, wherein in response to determining that the first positioning device is selected as the reference point of the coordinate system of the virtual world, the method further comprises: The coordinates of each of the other positioning devices in the virtual world are determined based on the relative position between the device position of the first positioning device and the device position of each of the other positioning devices.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium records an executable computer program, which is loaded by a data processing device to perform the following steps: Receive multiple positioning data, wherein the multiple positioning data corresponds to the device locations of multiple positioning devices in the real world, and the positioning devices include a first positioning device and a second positioning device, wherein the multiple positioning data includes first positioning data corresponding to the first positioning device and second positioning data corresponding to the second positioning device; In response to determining that the first positioning device is selected as a reference point in the coordinate system of the virtual world via a user interface, the coordinates of the second positioning device in the coordinate system of the virtual world are determined based on the relative position between the device position of the first positioning device and the device position of the second positioning device.

Images