Video camera synchronization

Abstract

A system and method are disclosed for synchronously capturing video using respective cameras of multiple devices, where the devices capture video according to a schedule based on a common camera period and a common synchronized clock, and adjust video capture parameters based on deviations from the schedule.

Description

[Technical Field]

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims the benefit of U.S. Patent Application No. 63 / 420,858, filed October 31, 2022, entitled "SYNCHRONIZATION OF VIDEO CAMERAS OVER IP."

[0002] The contents of the above applications are incorporated herein by reference in their entirety for all purposes.

[0003] The present invention relates to a method and system for synchronizing video capture by multiple devices, particularly portable electronic devices such as smartphones or tablets. In an example disclosed herein, the method and system are used to capture video of a sporting activity, such as a sporting event or practice session, and multiple devices are used to capture video of the activity from multiple viewpoints. [Background technology]

[0004] Sports are a significant source of entertainment and leisure for millions of people around the world. The use of video in sports has revolutionized the way teams and athletes analyze, strategize, and improve their performance. Video analysis allows for a detailed review of game plays, techniques, and tactics, providing valuable insights that can lead to improved performance on the field. Coaches can study their team's strengths and weaknesses, identify patterns, and make data-driven decisions to optimize their strategies. Players can analyze their own performance, identify areas for improvement, and hone their skills with precision.

[0005] Video technology has also transformed the fan experience, allowing spectators to relive and analyze key moments, gain a deeper understanding of the match, and participate in discussions and debates with other fans. Summary of the Invention

[0006] According to a first aspect of the present disclosure, there is provided a system for capturing video, the system comprising: a plurality of electronic devices, each of the electronic devices comprising a respective camera; and at least one server configured to send synchronized clock values ​​to each of the plurality of devices and receive video data from each of the plurality of devices, the system being configured to determine a camera synchronization time, the determining including by adding an integer multiple of a camera period to the synchronized clock value, and each of the plurality of devices being configured to capture video using the device's camera, determine that the camera synchronization time has elapsed, and in response, determine a camera offset by calculating a difference between a timestamp time of a current frame captured by the device and the camera synchronization time, and adjust one or more video capture parameters of the device based on the camera offset.

[0007] In some examples, each of the plurality of devices is configured to compare the camera offset to a threshold and adjust the one or more video capture parameters of the device based on the amount by which the camera offset exceeds the threshold.

[0008] Additionally or alternatively, the system may be further configured to determine one or more additional camera synchronization times, including, for each of the one or more additional camera synchronization times, by adding a second integer multiple of the camera period to the immediately preceding camera synchronization time, wherein each of the plurality of devices is further configured to capture video using a camera of the device, and for each of the one or more additional camera synchronization times, determine that the additional camera synchronization time has elapsed and, in response, determine a camera offset by calculating a difference between a timestamp time of a current frame captured by the device and the additional camera synchronization time, and then adjust one or more video capture parameters of the device based on the camera offset. In an example, the second integer is equal to the integer.

[0009] In an example, each of the plurality of devices is configured such that adjusting one or more video capture parameters by the device comprises capturing frames at an exposure time determined based on a camera offset of the device. Additionally or alternatively, each of the plurality of devices is configured such that adjusting one or more video capture parameters by the device comprises temporarily increasing or decreasing a capture duration of the device based on a camera offset of the device. Additionally or alternatively, each of the plurality of devices is configured such that adjusting one or more video capture parameters by the device comprises stopping video capture by the device for a period of time determined based on the camera offset of the device.

[0010] In some examples, each of the plurality of devices is configured to transmit video data to the at least one server using Internet Protocol. Additionally or alternatively, the at least one server is configured to transmit synchronized clock values ​​to each of the plurality of devices using Internet Protocol.

[0011] In some examples, at least one server comprises a clock server configured to transmit the synchronized clock value to each of the plurality of devices, the clock server being configured to transmit the synchronized clock value to each of the plurality of devices using, for example, Internet Protocol.

[0012] In some examples, at least one server (and in particular examples, a clock server configured by the at least one server) is configured to store a current server time and to send a request for a current device time to each of the plurality of devices, each of the plurality of devices being configured to send the device's current device time to the at least one server (e.g., the clock server) in response to receiving the request, and the at least one server (e.g., the clock server) is further configured to determine a first clock offset for each of the plurality of devices using the current server time, the device's current device time, and a transmission time of a communication between the at least one server (e.g., the clock server) and the device. In such examples, the at least one server (e.g., the clock server) may be configured to determine a second clock offset for each of the plurality of devices and to determine a clock drift for each of the plurality of devices by comparing the device's first clock offset and the second clock offset.

[0013] In some examples, the at least one server comprises a grabber server, and each of the plurality of devices is configured to transmit the video data to the grabber server. In such examples, the plurality of devices may be configured to transmit the video data to the grabber server using Internet Protocol.

[0014] In some examples, the at least one server is configured to determine camera periods for cameras of the plurality of devices, which includes triggering at least one group of the plurality of devices to capture video and measuring a camera period for each device in the group while the group of the plurality of devices is capturing video, and the at least one server determines the camera periods for the cameras of the plurality of devices based on the measured camera periods for the group of devices.

[0015] According to a further aspect of the present disclosure, there is provided a method for synchronizing video capture by respective cameras of a plurality of devices, the method comprising: determining a camera period of the cameras of the plurality of devices; determining a camera synchronization time, the determining including by adding an integer multiple of the camera period (T) to a synchronized clock value; capturing video at each of the plurality of devices using the respective cameras of the plurality of devices; at each of the plurality of devices, when the device determines that the camera synchronization time has elapsed, determining a camera offset by calculating a difference between a timestamp time of a current frame captured by the device and the camera synchronization time; and adjusting one or more video capture parameters of one or more devices of the plurality of devices based on the respective camera offsets of the one or more devices.

[0016] In an example, the method comprises, at each of a plurality of devices, comparing a camera offset of the device to a threshold, and adjusting one or more video capture parameters of the device based on the amount by which the camera offset of the device exceeds the threshold.

[0017] Additionally or alternatively, the method further comprises determining one or more additional camera synchronization times, including, for each of the one or more additional camera synchronization times, by adding a second value that is an integer multiple of the camera period to the immediately preceding camera synchronization time; capturing video using a camera of each of the multiple devices; and, for each of the multiple devices and for each of the one or more additional camera synchronization times, when the device determines that the additional camera synchronization time has elapsed, determining a camera offset by calculating a difference between a timestamp time of a current frame captured by the device and the additional camera synchronization time, and then adjusting one or more video capture parameters of the device based on the camera offset. In an example, the second integer is equal to the integer.

[0018] In some examples, the adjusting comprises, for each of the one or more devices, triggering the device to capture frames at an exposure time determined based on a camera offset of the device. Additionally or alternatively, the adjusting comprises, for each of the one or more devices, temporarily increasing or decreasing a capture duration of the device based on a camera offset of the device. Additionally or alternatively, the adjusting comprises, for each of the one or more devices, pausing video capture by the device for a period of time determined based on a camera offset of the device.

[0019] In some examples, the method further comprises transmitting the synchronized clock values ​​from a clock server to each of a plurality of devices, hi such examples, the transmitting of the synchronized clock values ​​may be performed using Internet Protocol.

[0020] In some examples, the method further comprises storing a current server time of at least one server (and, in particular examples, a clock server configured by the at least one server), transmitting, by the at least one server (e.g., the clock server), a request for the current device time to each of the plurality of devices, transmitting, at each of the plurality of devices, the current device time of the device in response to receiving the request, and determining a first clock offset for each of the plurality of devices using the current server time, the current device time of the device, and a transmission time of a communication between the at least one server (e.g., the clock server) and the device. In such examples, the method may further comprise determining a second clock offset for each of the plurality of devices, and determining clock drift for each of the plurality of devices by comparing the first clock offset and the second clock offset of the device.

[0021] In some examples, the method further comprises transmitting the video data from each of the plurality of devices to a grabber server. In such examples, the transmitting of the video data may be performed using Internet Protocol.

[0022] In some examples, determining the camera period of the cameras of the plurality of devices comprises triggering at least one group of the plurality of devices to capture video and measuring the camera period of each device in the group while the group of the plurality of devices is capturing video, wherein determining the camera period of the cameras of the plurality of devices is based on the measured camera period for the group of devices.

[0023] According to a further aspect of the present disclosure, there is provided a device for capturing video, the device comprising: at least one processor; a camera; and a computer-readable storage medium comprising instructions that, when executed by the at least one processor, cause the device to: receive a synchronized clock value from at least one server; determine a camera synchronization time, the determining including by adding an integer multiple of a camera period to the synchronized clock value; capture video using the at least one camera; determine that the camera synchronization time has elapsed and, in response, determine a camera offset by calculating a difference between a timestamp time of a current frame captured by the device and the camera synchronization time; adjust one or more video capture parameters based on the camera offset; and transmit the video to the at least one server.

[0024] In some examples, the instructions, when executed by at least one processor, cause the device to compare the camera offset to a threshold and adjust the one or more video capture parameters based on the amount by which the camera offset exceeds the threshold.

[0025] Additionally or alternatively, the instructions, when executed by the at least one processor, cause the device to: determine one or more additional camera synchronization times, including, for each of the one or more additional camera synchronization times, by adding the integer multiple of the camera period to the immediately preceding camera synchronization time; capture video using the camera; and, for each of the one or more additional camera synchronization times, when the device determines that the additional camera synchronization time has elapsed, determine a camera offset by calculating the difference between a timestamp time of a current frame captured by the device and the additional camera synchronization time, and then adjust one or more video capture parameters of the device based on the camera offset.

[0026] In some examples, adjusting one or more video capture parameters by the device comprises capturing frames at an exposure time determined based on a camera offset of the device. Additionally or alternatively, adjusting one or more video capture parameters by the device comprises temporarily increasing or decreasing a capture duration of the device based on a camera offset of the device. Additionally or alternatively, adjusting one or more video capture parameters by the device comprises stopping video capture by the device for a period of time determined based on the camera offset of the device.

[0027] In some examples, the instructions, when executed by the at least one processor, cause the device to transmit the video to at least one server using Internet Protocol.

[0028] According to a further aspect of the present disclosure, a computer program product is provided comprising instructions that, when executed by at least one processor of a device for capturing video, cause the device to: receive a synchronized clock value from at least one server, determine a camera synchronization time, the camera synchronization time including by adding an integer multiple of a camera period to the synchronized clock value, capture video using at least one camera, determine that the camera synchronization time has elapsed and, in response, determine a camera offset by calculating a difference between a timestamp time of a current frame captured by the device and the camera synchronization time, adjust one or more video capture parameters based on the camera offset, and transmit the video to at least one server. In some examples, the instructions, when executed by the at least one processor, cause the device to compare the camera offset to a threshold and adjust the one or more video capture parameters based on the amount by which the camera offset exceeds the threshold.

[0029] Additionally or alternatively, the instructions, when executed by the at least one processor, cause the device to: determine one or more additional camera synchronization times, including, for each of the one or more additional camera synchronization times, by adding the integer multiple of the camera period to the immediately preceding camera synchronization time; capture video using the camera; and, for each of the one or more additional camera synchronization times, when the device determines that the additional camera synchronization time has elapsed, determine a camera offset by calculating the difference between a timestamp time of a current frame captured by the device and the additional camera synchronization time, and then adjust one or more video capture parameters of the device based on the camera offset.

[0030] In some examples, adjusting one or more video capture parameters by the device comprises capturing frames at an exposure time determined based on a camera offset of the device. Additionally or alternatively, adjusting one or more video capture parameters by the device comprises temporarily increasing or decreasing a capture duration of the device based on a camera offset of the device. Additionally or alternatively, adjusting one or more video capture parameters by the device comprises stopping video capture by the device for a period of time determined based on the camera offset of the device.

[0031] In any example of the above embodiment, the integer is at least two.

[0032] In any of the above examples, at least some of the devices may be portable electronic devices such as mobile devices. Additionally or alternatively, at least some of the devices may be smartphones or tablet computing devices.

[0033] Further features and advantages will become apparent from the following description, given by way of example only, with reference to the accompanying drawings, in which: [Brief explanation of the drawings]

[0034] [Figure 1] FIG. 1 is a schematic diagram illustrating a video capture system, in accordance with an exemplary embodiment. [Figure 2A] FIG. 2 is a diagram illustrating a detailed example of a synchronization procedure for devices in the system of FIG. 1. [Figure 2B] FIG. 2 is a diagram illustrating a detailed example of a synchronization procedure for devices in the system of FIG. 1. [Figure 3] FIG. 2 is a perspective view of the system of FIG. 1 when deployed in a basketball game. [Figure 4]FIG. 10 is a schematic diagram illustrating a method for synchronizing video capture by respective cameras of multiple devices, according to a further exemplary embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0035] Embodiments of this application relate to synchronously capturing video using cameras on each of multiple devices. In the embodiments described below, the devices capture video according to a schedule based on a common camera period and a common synchronized clock, and adjust video capture parameters based on deviations from the schedule. Particular embodiments of this application relate to capturing video of sporting activities, particularly when video of the sporting activity is captured from multiple perspectives using cameras on multiple devices.

[0036] Video frames are considered synchronized if they are captured at approximately the same time. In embodiments, video frames are considered synchronized if they are captured within a threshold time window of each other. In embodiments, the threshold time window may be 1 microsecond, 10 microseconds, 1 millisecond, 10 milliseconds, or 100 milliseconds or more. The threshold time window may vary for different applications, devices, or environments. For example, in applications where images do not change frequently, it may be acceptable to use a larger threshold time window (e.g., 100 milliseconds) than in applications where the video contains fast moving objects or people. As another example, in a system with a large number of devices and cameras (e.g., 100 or more), it may be appropriate to use a larger threshold time window (e.g., 100 milliseconds), for example, due to the difficulty of synchronizing capture across such a large number of devices. In contrast, in a system with a small number of cameras (e.g., 100 or less), a smaller threshold time window may be appropriate, thereby reducing the likelihood of difficulty synchronizing capture. In embodiments, the time window threshold may be changed during system operation and / or adjusted based on the application, the type of video captured, the number of devices capturing video, the distance between devices, network characteristics (latency, reliability, jitter), etc.

[0037] 1, a schematic diagram illustrating a video capture system according to an exemplary embodiment is shown. As shown, system 100 includes multiple devices 110(a)-(f), each including at least one camera 112. As shown, each of devices 110(a)-(f) may be positioned to capture video from, for example, a different viewpoint 114. In the illustrated example, system 100 includes six devices 110(a)-(f), although of course this is merely illustrative and not required.

[0038] 1, each device 110 is a portable electronic device, and more specifically, a smartphone. However, this is by no means required, and in other embodiments, some or all of devices 110(a)-(f) may be different types of portable electronic devices, such as, for example, tablet computing devices, or larger (non-portable) devices, or indeed any device having sufficient processing power and a camera (or cameras) to operate as described herein. In embodiments, devices 110(a)-(f) may lack built-in mechanisms for precise frame synchronization between devices and / or may not include dedicated video cameras (which often include such built-in mechanisms for precise frame synchronization between devices).

[0039] Modern portable electronic devices 110 with cameras (e.g., smartphones and tablet computing devices, among others) are readily available, allow the system 100 to be easily expanded by adding further portable electronic devices 110(a)-(f) and to be easily upgraded to take advantage of future advances in portable electronic device capabilities, are easy to configure for use in the system 100 by installing appropriate software on the portable electronic device 110 and by using the portable electronic device's 110 wireless connectivity, and typically can use the camera of such devices to capture video at high resolution and / or high frame rates and provide significant processing power.

[0040] Returning to Figure 1, it should be noted that system 100 further includes server 120. As indicated by the lines extending from electronic devices 110(a)-(f) and server 120 in Figure 1, server 120 is in data communication with electronic devices 110(a)-(f) via network 130, which may be (or include) the Internet, for example. Thus (or otherwise), devices 110(a)-(f) and server 120 may communicate (at least in part) using the Internet Protocol and / or Transmission Control Protocol. However, it should be understood that network 130 need not comprise the Internet, and thus (or otherwise), network 130 may be (or include) a LAN, VPN, and / or intranet. Furthermore, it is by no means necessary for devices 110(a)-(f) and server 120 to communicate using Internet Protocol or Transmission Control Protocol; other embodiments of system 100 may utilize any communication protocol suitable for the particular type of network 130 connecting devices 110(a)-(f) and server 120.

[0041] It should also be understood that in embodiments, the devices 110(a)-(f) and the server 120 may communicate wirelessly, for example, using 4G, 5G, or Wi-Fi protocols. Wireless connections allow for greater flexibility in the placement / arrangement of multiple devices 110(a)-(f). However, in other embodiments, some or all of the data connections between the portable electronic devices 110(a)-(f) and the server 120 may be physical connections.

[0042] The interaction between the server 120 and the devices 110(a)-(f) of the system of Figure 1 will now be described. In the embodiment of Figure 1, the server 120 provides a synchronized clock value (t REF ) periodically (i.e., each of the multiple devices transmits the same synchronized clock value (t REF) to assist in synchronizing video capture by multiple devices 110(a)-(f). REF ) may be based on a reference clock onboard server 120 or a reference clock that transmits a time signal to server 120. Server 120 is further configured to receive video data from each of multiple devices 110(a)-(f). Server 120 may perform various processing of the received video data, for example, to analyze the video and / or prepare the video for streaming.

[0043] As mentioned above, devices 110(a)-(f) synchronize the video capture by their respective cameras using the synchronized clock values ​​(t REF ) to each device 110(a)-(f) (e.g., by appropriate programming of an on-board processor) to synchronize clock values ​​(t REF ) and an integer number (n) of camera periods (T), the future camera synchronization time (t sync[1] For example, the camera synchronization time (t sync[1] ) is determined by the synchronized clock value (t REF ) by adding an integer number (n) of camera periods (T). The same value of n is used by all of devices 110(a)-(f) to aid in synchronizing video capture.

[0044] In the embodiment, the camera synchronization time (t sync The same camera period (T) is used to determine the camera period (T). The value of the camera period (T) may optionally be determined by measuring the respective values ​​of the camera period for the group (or possibly all) of the plurality of devices 110(a)-(f) during operation of the devices. For example, the respective values ​​of the camera period for the group (or all) of the plurality of devices 110(a)-(f) may be measured during a test run of the devices.

[0045] Such measurement of camera period for some or all of devices 110(a)-(f) may be triggered by server 120. For example, server 120 may send or broadcast a message to each of multiple devices 110(a)-(f) to cause the device to capture video, measure the camera period value while capturing video, and transmit the measured camera period value to server 120.

[0046] However, it should be understood that it is not necessary that the value of camera period (T) be determined by such measurements of camera periods for some or all of devices 110(a)-(f). Accordingly (or otherwise), in other embodiments, camera period (T) may be based on existing data / information, such as camera period data from the manufacturer of device 110.

[0047] In the above, the devices 110(a) to (f) synchronize with the camera synchronization time (t sync[1] ), in other examples, the server 120 may instead determine the camera synchronization time (t sync[1] ) and convert it to the synchronized clock value (t REF ) separately or synchronized with the clock value (t REF ) to each device 110(a)-(f).

[0048] A more detailed example of a synchronization procedure for the multiple devices 110(a)-(f) of the system 100 of Figure 1 will be described with reference to Figures 2A and 2B. Figures 2A and 2B illustrate the timing of the capture of a series of frames 200(1)-200(12) by one of the multiple devices 110(a)-(f) of the system 100 of Figure 1. In each of Figures 2A and 2B, the horizontal axis indicates time, and the time range indicated on the horizontal axis of Figure 2B immediately follows the time range indicated on the horizontal axis of Figure 2A, and therefore Figure 2B is a continuation of Figure 2A.

[0049] In the example of FIGS. 2A and 2B, the first frame 200(1) is synchronized with the clock value (t REF ), and the second, third, and fourth frames 200(2)-200(4) are captured at successive times thereafter. As can be seen from FIG. 2A, in the particular example shown, the camera synchronization time (t sync[1] ) is the synchronized clock value (t REF ) plus three camera periods (T). In other words, in the example of FIG. 2, n=3. However, of course, this is merely illustrative, and in other examples, n may be 1, 2, 4, or any other suitable integer value.

[0050] The fourth frame 200(4) is the camera synchronization time (t sync[1] Note that the image is captured after the camera synchronization time (t sync[1] ) has elapsed, and the timestamp time of the current frame captured by the device (the fourth frame 200(4) in the example of FIG. 2) is sync[1] ) by calculating the difference between the camera offset (t offset ) is configured to determine

[0051] The camera offset (t) thus determined is used to determine adjustments to one or more video capture parameters of device 110 in accordance with the synchronization procedure of FIGS. 2A and 2B. offset ) to use.

[0052] The one or more video capture parameters adjusted by device 110 may comprise, for example:

[0053] You can adjust the device's camera offset (t offset ) the exposure time (e.g., the exposure time of the immediately following frame), such as capturing a frame at an exposure time determined based on the exposure time.

[0054] You can adjust the device's camera offset (t offset Temporarily increasing or decreasing the capture period (e.g., every frame until the next camera sync time) of the device based on the capture period (e.g., every frame until the next camera sync time), or conversely, the capture frequency; and / or

[0055] A delay time before capturing the next frame, where adjusting comprises stopping video capture by the device for a period of time determined based on a camera offset of the device.

[0056] In some examples, the determination of the adjustment of one or more video capture parameters of device 110 may be performed by the device itself. However, in other examples, server 120 may determine the adjustment of each video capture parameter for a group of, or all of, multiple devices 110(a)-(f) in system 100. In such examples, each device 110(a)-(f) may determine the adjustment of each respective camera offset (t offset ) value to the server 120, which then sends the camera offset (t offset ) to the devices 110(a)-(f) indicating adjustments to the video capture parameters of the devices 110(a)-(f) based on the camera offset (t). Indeed, even in examples where the determination of the adjustments to one or more video capture parameters of the devices 110 is performed by the devices themselves, each device 110(a)-(f) may transmit adjustment data to the devices 110(a)-(f) indicating adjustments to the video capture parameters of the devices 110(a)-(f) based on the camera offset (t offset ) values ​​to the server 120, which may be useful, for example, to enable the server 120 to perform analysis on the operation of the system 100 and / or to enable the server 120 to control the overall operation of the system 100.

[0057] In various embodiments, the same video capture parameters may be adjusted for each device 110(a)-(f). For example, only the exposure time may be adjusted for each device 110(a)-(f). However, in more complex embodiments, different video capture parameters may be adjusted for different devices. For example, the exposure time may be adjusted for some of the devices 110(a)-(f) (e.g., devices where only minor adjustments are appropriate), while the capture duration may be adjusted for other devices 110(a)-(f) (e.g., devices where more significant adjustments are appropriate).

[0058] In general, adjustments to the video capture parameters of devices 110(a)-(f) may be calculated to reduce the expected camera offset at subsequent camera synchronization times. An example of this is shown in FIG. 2A, where after frame 200(4), the device's capture period is shortened, resulting in a second camera synchronization time (t sync[2] ) camera offset (t offset[2] ) is significantly smaller. In embodiments, if the camera offset is determined to be less than a threshold amount, system 100 may not perform any adjustments to the device's camera parameters, as the device's camera captures may be considered sufficiently synchronized.

[0059] As can be seen from FIG. 2A, the second camera synchronization time (t sync[2] ) is the previous camera synchronization time (t sync[1] ) by adding n camera periods (T) (n=3 as before). Subsequent camera synchronization times are then determined in a similar manner.

[0060] Referring now to FIG. 2B, the device 110 receives a second synchronized clock value (t REF[2] ) is shown. REF[2] ) are displayed on the time axis. However, the second synchronized clock value (tREF[2] ) is actually received by device 110 at the time (t REF[2] ), by an amount corresponding to the transmission time between the server 120 and the device 110.

[0061] In either case, as can be seen in FIG. 2B, the second synchronized clock value (t REF[2] ) is received, the next camera synchronization time (t sync[1’] ) is the newly received second synchronized clock value (t REF[2] ) plus n camera periods (T). Specifically, as shown, t sync[1’] is the second synchronous clock value t REF[2] The camera synchronization time t is then calculated by adding three camera periods (3T) to the camera synchronization time t sync[1’] is the camera offset (T offset[3] ) Subsequent camera synchronization times are then determined as before by adding n camera periods (T) to the previous camera synchronization time.

[0062] 2A and 2B, as further (third, fourth, fifth, etc.) synchronized clock values ​​are received from server 120, the next camera synchronization time is similarly determined by adding n camera periods (T) to the newly received synchronized clock value. Subsequent camera synchronization times are then determined, as before, by adding n camera periods (T) to the previous camera synchronization time.

[0063] In some embodiments, the synchronized clock value (t REF[1] , t REF[2]) may correspond to evenly spaced time points, e.g., spaced at intervals of an integer multiple of nT. In other embodiments, the synchronized clock values ​​may correspond to semantically significant time points during the activity being captured in the video, e.g., the start of an event (e.g., the start of a sports match) and / or a break in the event (e.g., halftime during a sports match). Accordingly, or otherwise, the server 120 determines when to transmit synchronized clock values ​​to the plurality of devices 110(a)-(f) by analyzing the video received from the plurality of devices 110(a)-(f).

[0064] Returning to the system 100 of FIG. 1, to assist in synchronizing video capture by the multiple devices 110(a)-(f), each of the multiple devices 110(a)-(f) may include a clock synchronized with a reference clock that generates synchronized clock values. Each such device clock may be, for example, an internal monotonically increasing system clock. As will be appreciated, various techniques are available for synchronizing the clocks of the multiple devices 110(a)-(f) with a reference clock that generates synchronized clock values. One example of such a technique is as follows:

[0065] 1. The server 120 sends a request including the current reference clock time (RCT) to each device 110, and the server 120 stores this current reference clock time value (first reference clock time value).

[0066] 2. When the request is received by the device 110, the device determines the device clock offset as the difference between the received reference clock time (first reference clock time value) and the current device clock time and sends a response to the server.

[0067] 3. The server 120 records the reference clock time when the response was received as a second reference clock time value.

[0068] 4. The server 120 may then calculate the round-trip time between the server 120 and each device 110, for example, by calculating the difference between the reference clock time when it sent a request to each device 110 (i.e., the first reference clock time value) and the reference clock time when it received a response from each device 110 (i.e., the second reference clock time value of each device 110).

[0069] 5. If the round trip time is not abnormally long (e.g., exceeds a predetermined threshold), the server 120 sends a message to each device 110 that includes the transmission time of the device 110. The transmission time of each device 110 may be calculated, for example, as half the round trip time of the corresponding device.

[0070] 6. When this message is received by device 110, device 110 adds the transmission time to the device clock offset value calculated in step 2 and stores this as the reference clock offset value. Device 110 can then determine a time that is (at least approximately) synchronized with the reference clock (synchronized device time) by adding the stored reference clock offset value to its internal clock time. Optionally, device 110 can then send a response to server 120 that includes the current synchronized device time.

[0071] However, it will be appreciated that this is merely an illustrative example and that various other techniques are available for synchronizing the clocks of multiple devices 110(a)-(f) with a reference clock that generates synchronized clock values.

[0072] In an embodiment, resynchronization of the device clock may be performed whenever server 120 detects a clock drift exceeding a certain threshold, periodically at predetermined time intervals set based on the desired synchronization accuracy, and / or at times based on other considerations, such as during timeouts or the duration of a sports match.

[0073] Clock drift (i.e., deviation from synchronization with a reference clock) of a device clock can be determined by calculating the offset between a given device clock and a reference clock (taking into account transmission times between the server 120 and the given device) at two points in time, e.g., minutes apart. For example, a first clock offset value is determined at the time of clock synchronization and a second clock offset value is determined 10 minutes later. If the difference in the clock offset values ​​exceeds a threshold value (e.g., 500 microseconds, 1 millisecond, 10 milliseconds, or more), device clock synchronization may be repeated.

[0074] Alternatively, or additionally, clock drift may be identified by comparing features within captured images. For example, images captured by different devices but with the same timestamp may be compared and analyzed to determine whether they captured an event at the same time. In embodiments, features of frames such as game time clocks, advertising boards, etc. may be used to determine the time offset between such frames captured by different devices, and thereby the clock drift, using one or more of the techniques described in commonly assigned U.S. patent application Ser. No. 18 / 346,355.

[0075] 1, it should be noted that while system 100 is shown as including only a single server 120, it should of course be understood that system 100 may include multiple servers. In particular, system 100 may, in some embodiments, include a clock server configured to synchronize the clocks of the plurality of devices 110(a)-(f), such as by transmitting synchronized clock values ​​to each of the plurality of devices 110(a)-(f) (and optionally determining adjustments to video capture parameters for a group or all of the plurality of devices 110(a)-(f) and / or optionally determining clock offsets for some or all of the plurality of devices 110(a)-(f)), and a “grabber” server configured to receive video data from each of the plurality of devices 110(a)-(f).

[0076] Additionally, while the above description focuses on the operation of the plurality of devices 110(a)-(f), it is further noted that the system 100 may, in other examples, include an additional device equipped with a camera. Indeed, such a device may be the same type as the plurality of devices 110(a)-(f); for example, the additional device and the plurality of devices 110(a)-(f) may be smartphones. Furthermore, video capture by the additional device may be synchronized using the same or different techniques for the plurality of devices 110(a)-(f), or may not be synchronized at all.

[0077] The system 100 of FIG. 1 may be particularly (but by no means exclusively) suited for use in capturing video of sporting activities because the devices 110, especially if portable, can be easily installed in a variety of locations and orientations to capture video of the sporting activity. Such an installation of the system 100 is illustrated in FIG. 3 , which shows the system 100 deployed to analyze a basketball game. Similar to the portable electronic devices illustrated in FIG. 1 , each of the portable electronic devices 100(a)-(g) illustrated in FIG. 3 has a corresponding different viewpoint 114 from which to capture video. As illustrated, most of the multiple portable electronic devices 100(a)-(f) are positioned at various fixed positions and orientations around the basketball court. However, one of the devices 100(g) is worn by a participant in the sporting activity and provides a perspective of the sporting activity not available with a conventional camera. The participants may be players (e.g., allowing data characterizing player movements to be collected directly using accelerometers, gyros, etc. mounted on the portable electronic devices, e.g., as part of an IMU), or they may be referees, umpires, etc. (e.g., to assist in automated refereeing). Of course, while the deployment example shown in FIG. 3 shows only one of the devices 100(g) worn by a participant in the sporting activity, it should be understood that in other deployments of the system, multiple (or even all) of the portable electronic devices 100(a)-(g) may be worn by a participant in the sporting activity, rather than being deployed in a fixed location and orientation. Furthermore, in some examples, some of the portable electronic devices 100(a)-(g) may be held by a coach or a fan (e.g., a fan's own personal device may function as a portable electronic device 100(a)-(g) in the system 100).Furthermore, while FIG. 3 depicts a basketball game, it will be understood that this is for illustrative purposes only, and that the system 100 of FIG. 1 is suitable for deployment in many other types of sporting activities, and indeed in non-sporting environments such as non-sporting live events (e.g., concerts, comedy shows, or plays) or non-sporting practice sessions (e.g., music practice or theater rehearsals).

[0078] Returning to the system of Figure 1, it should be noted that in some examples, server 120 may be located in the same physical location as portable electronic devices 110(a)-(f). For example, in a situation where system 100 is deployed at a sporting event, server 120 may be located in a server room at the venue where the sporting event is taking place, as shown in Figure 3. Alternatively, server 120 may be located in a truck parked at the venue. However, in other examples, server 120 may be a remote server or a cloud server.

[0079] Furthermore, if system 100 includes multiple servers, the servers do not need to be co-located. For example, if system 100 includes a grabber server and a clock server, the grabber server may be co-located with multiple devices 110(a)-(f) (or vice versa). For example, in a situation where system 100 is deployed in a sporting event, the grabber server may be located in a server room at the venue where the sporting event is being held or in a truck parked at the venue, and the clock server may be a remote server or a cloud server (or vice versa), as shown in FIG. 3 .

[0080] Attention is now directed to Figure 4, which illustrates a method 400 for synchronizing video capture by respective cameras of multiple devices according to a further aspect of the present disclosure. Particular embodiments of method 400 may incorporate principles and features taught above with reference to any embodiment of system 100 of Figures 1-3. For example, method 400 may synchronize video capture by respective cameras of multiple devices 110(a)-(f) according to any of the techniques described above with reference to Figures 1-3.

[0081] As shown in block 401 of FIG. 4, the method comprises determining a camera period for cameras of a plurality of devices. The determining of block 401 may implement any of the techniques for determining a camera period (T) described above with reference to FIGS. 1-3. For example, the determining may comprise triggering at least one group of a plurality of devices to capture video and measuring a camera period for each device in the group while the group of devices is capturing video. In such an example, the determination of the camera periods for the cameras of the plurality of devices may be made based on the camera periods measured for the group of devices.

[0082] 4, the method 400 further comprises determining 402 a camera synchronization time, including by adding an integer multiple of the camera period (T) to the synchronized clock value. The determining of block 402 may similarly implement any of the techniques for determining the camera synchronization time described above with reference to FIGS. 1-3.

[0083] As further shown in FIG. 4, the method 400 also comprises capturing a video 403 at each of the plurality of devices using a camera of each of the plurality of devices.

[0084] 4, the method 400 further includes, at each of the plurality of devices 110(a)-(f), when the device determines that the camera synchronization time has elapsed, determining 404 a camera offset by calculating the difference between the timestamp time of a current frame captured by the device and the camera synchronization time. Determining 404 the camera offset of each device 110(a)-(f) may employ any of the techniques described above with reference to FIGS. 1-3, and particularly FIGS. 2A and 2B.

[0085] As also shown in FIG. 4 , method 400 further includes adjusting 405 one or more video capture parameters of one or more devices of the plurality of devices based on a respective camera offset of the one or more devices. Adjusting 405 one or more video capture parameters of one or more devices of the plurality of devices 110(a)-(f) may similarly employ any of the techniques described above with reference to FIGS. 1-3 . For example, adjusting may include, for each device, triggering the device to capture frames with an exposure time determined based on the device's camera offset. Additionally or alternatively, adjusting may include, for each device, temporarily increasing or decreasing the device's capture duration based on the device's camera offset. Additionally or alternatively, adjusting may include, for each device, stopping video capture by the device for a period of time determined based on the device's camera offset.

[0086] Although blocks 401-405 are shown in sequence in FIG. 4, it will be understood that the actions of blocks 401-405 are not necessarily performed in the order shown, and furthermore, it is by no means necessary that the actions of blocks 401-405 be performed in sequence.

[0087] More generally, it will be understood that any feature described in connection with any one embodiment may be used alone or in combination with other features described, and may also be used in combination with one or more features of any other embodiment, or in combination with any other embodiment. Moreover, equivalents and modifications not described above may also be employed without departing from the scope of the invention as defined in the appended claims.

[0088] Additional Embodiments of the Disclosure While various specific embodiments have been described above, further embodiments of the present disclosure are described in the following sections. 1. A system for capturing video, comprising: a plurality of electronic devices, each of which includes a respective camera; at least one server configured to transmit synchronized clock values ​​to each of the plurality of devices and receive video data from each of the plurality of devices; Equipped with the system is configured to determine a camera synchronization time, including by adding an integer multiple of a camera period to a synchronized clock value; Each of the plurality of devices Capturing video using the device's camera; determining that a camera synchronization time has elapsed, and responsively determining a camera offset by calculating a difference between a timestamp time of a current frame captured by the device and the camera synchronization time; adjusting one or more video capture parameters of the device based on the camera offset; The system is configured to: 2. Each of the plurality of devices comparing the camera offset to a threshold; adjusting the one or more video capture parameters of the device based on the amount by which the camera offset exceeds the threshold; and Item 1. The system of item 1, configured to: 3. The system, further configured to determine one or more additional camera synchronization times, including, for each of the one or more additional camera synchronization times, by adding the integer multiple of the camera period to a previous camera synchronization time; Each of the plurality of devices capturing video using the camera of the device; The system of any one of clauses 1 or 2, further configured to: for each of one or more additional camera synchronization times, determine that the additional camera synchronization time has elapsed; and in response, determine a camera offset by calculating the difference between the timestamp time of a current frame captured by the device and the additional camera synchronization time; and then adjust one or more video capture parameters of the device based on the camera offset. 4. The system of any one of clauses 1 to 3, wherein each of the multiple devices is configured such that adjusting one or more video capture parameters by the device comprises capturing frames with an exposure time determined based on a camera offset of the device. 5. The system described in any one of clauses 1 to 4, wherein each of the multiple devices is configured such that adjusting one or more video capture parameters by the device comprises temporarily increasing or decreasing the capture period of the device based on the camera offset of the device. 6. The system of any one of clauses 1 to 5, wherein each of the plurality of devices is configured such that adjusting one or more video capture parameters by the device comprises stopping video capture by the device for a period of time determined based on a camera offset of the device. 7. The system described in any one of items 1 to 6, wherein at least one server comprises a clock server, and the clock server is configured to transmit the synchronized clock value to each of the multiple devices. 8. The system of claim 7, wherein the clock server is configured to transmit the synchronized clock value to each of the plurality of devices using Internet Protocol. 9. At least one server: Remembering the current server time; sending a request for the current device time to each of the plurality of devices; configured to each of the plurality of devices configured to transmit a current device time of the device to the at least one server in response to receiving the request; The system of any one of clauses 1 to 8, wherein at least one server is further configured to determine a first clock offset for each of the plurality of devices using a current server time, a current device time of the device, and a transmission time of a communication between the at least one server and the device. 10. At least one server: determining a second clock offset for each of the plurality of devices; determining a clock drift for each of the plurality of devices by comparing a first clock offset and a second clock offset of the device; Item 10. The system of item 9, configured to: 11. The system of any one of clauses 1 to 10, wherein at least one server comprises a grabber server, and each of the plurality of devices is configured to transmit video data to the grabber server. 12. The system of claim 11, wherein the plurality of devices are configured to transmit video data to the grabber server using Internet Protocol. 13. The system of any one of clauses 1 to 12, wherein the plurality of devices comprises at least one portable device. 14. The system of claim 13, wherein at least one portable device comprises a smartphone. 15. The system described in any one of clauses 1 to 14, wherein at least one server is configured to determine camera periods for cameras of a plurality of devices, and includes triggering at least one group of the plurality of devices to capture video and measuring the camera period of each device in the group while the group of the plurality of devices is capturing video, and wherein the at least one server determines the camera periods for the cameras of the plurality of devices based on the measured camera period for the group of devices. 16. A method for synchronizing video capture by respective cameras of multiple devices, comprising: determining camera periods for cameras of a plurality of devices; determining a camera synchronization time, the determining including by adding a synchronized clock value to an integer multiple of the camera period; capturing video at each of the plurality of devices using a respective camera of the plurality of devices; At each of the plurality of devices, when the device determines that the camera synchronization time has elapsed, determining a camera offset by calculating a difference between a timestamp time of a current frame captured by the device and the camera synchronization time; adjusting one or more video capture parameters of one or more of the plurality of devices based on the respective camera offsets of the one or more devices; A method comprising: 17. The method of claim 16, further comprising, at each of a plurality of devices, comparing a camera offset of the device to a threshold, and adjusting one or more video capture parameters of the device based on the amount by which the camera offset of the device exceeds the threshold. 18. Determining one or more additional camera synchronization times, including, for each of the one or more additional camera synchronization times, by adding the integer multiple of the camera period to a previous camera synchronization time; capturing video using a camera of each of a plurality of devices; for each of the plurality of devices, and for each of the one or more additional camera synchronization times, when the device determines that the additional camera synchronization time has elapsed, determining a camera offset by calculating a difference between a timestamp time of a current frame captured by the device and the additional camera synchronization time, and then adjusting one or more video capture parameters of the device based on the camera offset; 18. The method of any one of clauses 16 or 17, further comprising: 19. The method of any one of clauses 16 to 18, wherein the adjusting comprises, for each of one or more devices, triggering the device to capture a frame at an exposure time determined based on the device's camera offset. 20. The method of any one of clauses 16 to 19, wherein the adjusting comprises, for each of one or more devices, temporarily increasing or decreasing the capture period of the device based on the camera offset of the device. 21. The method of any one of clauses 16 to 20, wherein the adjusting comprises, for each of one or more devices, stopping video capture by the device for a period of time determined based on the device's camera offset. 22. The method according to any one of clauses 16 to 21, further comprising transmitting the synchronized clock value from a clock server to each of a plurality of devices. 23. The method of clause 22, wherein the transmitting of the synchronized clock values ​​is performed using the Internet Protocol. 24. storing a current server time of at least one server; transmitting, by at least one server, a request for a current device time to each of a plurality of devices; At each of the plurality of devices, in response to receiving the request, transmitting a current device time of the device; determining a first clock offset for each of the plurality of devices using a current server time, a current device time of the device, and a transmission time of a communication between at least one server and the device; Item 24. The method according to any one of Items 16 to 23, further comprising: 25. Determining a second clock offset for each of the plurality of devices; determining a clock drift for each of the plurality of devices by comparing a first clock offset and a second clock offset of the device; Item 25. The method of item 24, further comprising: 26. The method of any one of clauses 16 to 25, further comprising transmitting video data from each of the plurality of devices to a grabber server. 27. The method of any one of paragraphs 16 to 26, wherein the transmitting of video data is performed using the Internet Protocol. 28. The method of any one of clauses 16 to 27, wherein the plurality of devices comprises at least one portable device. 29. The method of claim 28, wherein at least one portable device comprises a smartphone. 30. Determining the camera period of the cameras of the plurality of devices comprises triggering at least one group of the plurality of devices to capture video; and measuring the camera period of each device of the group while the group of the plurality of devices is capturing video; 30. The method of any one of clauses 16 to 29, wherein determining the camera period of the cameras of the plurality of devices is based on measured camera periods for a group of devices. 31. A device for capturing video, comprising: at least one processor; a camera; and a program that, when executed by the at least one processor, causes the device to: receiving synchronized clock values ​​from at least one server; determining a camera synchronization time, the determining including by adding an integer multiple of the camera period to a synchronized clock value; capturing video using at least one camera; determining that a camera synchronization time has elapsed, and responsively determining a camera offset by calculating a difference between a timestamp time of a current frame captured by the device and the camera synchronization time; adjusting one or more video capture parameters based on the camera offset; and transmitting the video to at least one server; and a computer-readable storage medium comprising instructions for causing the device to execute the method. 32. The instructions, when executed by at least one processor, cause a device to: comparing the camera offset to a threshold; adjusting the one or more video capture parameters based on the amount by which the camera offset exceeds the threshold; Item 32. The device according to item 31, which causes the device to execute the above. 33. The instructions, when executed by at least one processor, cause a device to: determining one or more additional camera synchronization times, the determining including, for each of the one or more additional camera synchronization times, by adding the integer multiple of the camera period to a previous camera synchronization time; Using the camera to capture video; for each of the one or more additional camera synchronization times, when the device determines that the additional camera synchronization time has elapsed, determining a camera offset by calculating a difference between a timestamp time of a current frame captured by the device and the additional camera synchronization time, and then adjusting one or more video capture parameters of the device based on the camera offset; 33. The device according to claim 31, wherein the device executes the above. 34. A device described in any one of clauses 31 to 33, wherein adjusting one or more video capture parameters by the device comprises capturing frames with an exposure time determined based on a camera offset of the device. 35. A device described in any one of clauses 31 to 34, wherein adjusting one or more video capture parameters by the device comprises temporarily increasing or decreasing a capture period of the device based on a camera offset of the device. 36. A device described in any one of clauses 31 to 36, wherein adjusting one or more video capture parameters by the device comprises stopping video capture by the device for a period of time determined based on a camera offset of the device. 37. A device according to any one of clauses 31 to 36, wherein the instructions, when executed by at least one processor, cause the device to transmit video to at least one server using Internet Protocol. 38. A device according to any one of clauses 31 to 37, wherein the device is a portable electronic device. 39. The device according to any one of clauses 31 to 38, wherein the device is a smartphone. 40. Execution of instructions by at least one processor of a device for capturing video causes the device to: receiving synchronized clock values ​​from at least one server; determining a camera synchronization time, the determining including by adding an integer multiple of the camera period to a synchronized clock value; capturing video using at least one camera; determining that a camera synchronization time has elapsed, and responsively determining a camera offset by calculating a difference between a timestamp time of a current frame captured by the device and the camera synchronization time; adjusting one or more video capture parameters based on the camera offset; and transmitting the video to at least one server; 1. A computer program product comprising instructions for causing a computer to execute: 41. The instructions, when executed by at least one processor, cause a device to: comparing the camera offset to a threshold; adjusting the one or more video capture parameters based on the amount by which the camera offset exceeds the threshold; Item 41. A computer program product according to Item 40, which causes the computer program product to execute the following: 42. The instructions, when executed by at least one processor, cause a device to: determining one or more additional camera synchronization times, the determining including, for each of the one or more additional camera synchronization times, by adding the integer multiple of the camera period to a previous camera synchronization time; Using the camera to capture video; for each of the one or more additional camera synchronization times, when the device determines that the additional camera synchronization time has elapsed, determining a camera offset by calculating a difference between a timestamp time of a current frame captured by the device and the additional camera synchronization time, and then adjusting one or more video capture parameters of the device based on the camera offset; 42. The computer program product of claim 40 or 41, causing the computer program product to execute the following: 43. The computer program product of any one of clauses 40 to 42, wherein adjusting one or more video capture parameters by the device comprises capturing frames with an exposure time determined based on a camera offset of the device. 44. A computer program product described in any one of clauses 40 to 43, wherein adjusting one or more video capture parameters by the device comprises temporarily increasing or decreasing a capture period of the device based on a camera offset of the device. 45. The computer program product of any one of clauses 40 to 44, wherein adjusting one or more video capture parameters by the device comprises stopping video capture by the device for a period of time determined based on a camera offset of the device. 46. ​​The computer program product of any one of clauses 40 to 45, wherein the instructions, when executed by at least one processor, cause the device to transmit video to at least one server using Internet Protocol.

Claims

1. A system for capturing video, Multiple electronic devices, each equipped with its own camera, At least one server configured to transmit synchronized clock values ​​to each of the multiple devices and to receive video data from each of the multiple devices Equipped with, The system is configured to determine the camera synchronization time, which includes determining the synchronized clock value by adding an integer multiple of the camera period, Each of the aforementioned plurality of devices To capture video using the camera of the aforementioned device, The camera synchronization time is determined to have elapsed, and in response, the camera offset is determined by calculating the difference between the timestamp time of the current frame captured by the device and the camera synchronization time. Adjusting one or more video capture parameters of the device based on the camera offset. A system configured to do so.

2. Each of the aforementioned plurality of devices Comparing the aforementioned camera offset with a threshold, Adjusting one or more video capture parameters of the device based on the amount by which the camera offset exceeds the threshold. The system according to claim 1, configured to do the following.

3. The aforementioned system The system is further configured to determine one or more additional camera synchronization times, each of which includes adding an integer multiple of the camera period to the immediately preceding camera synchronization time. Each of the aforementioned plurality of devices To capture video using the camera of the aforementioned device, The system according to claim 1, further configured to determine, for each of the one or more additional camera synchronization times, that the additional camera synchronization time has elapsed, and in response to that, determine a camera offset by calculating the difference between the timestamp of the current frame captured by the device and the additional camera synchronization time, and then adjust one or more video capture parameters of the device based on the camera offset.

4. Each of the plurality of devices adjusts one or more video capture parameters by the device. To capture a frame with an exposure time determined based on the camera offset of the device, Temporarily increasing or decreasing the capture period of the device based on the camera offset of the device, and The system according to any one of claims 1 to 3, configured to include at least one of the following: stopping video capture by the device for a period of time determined based on the camera offset of the device.

5. The aforementioned at least one server, To remember the current server time, To send a request for the current device time to each of the aforementioned multiple devices. It is configured to do so, Each of the plurality of devices is configured to transmit the current device time of the device to the at least one server in response to receiving the request. The system according to any one of claims 1 to 3, wherein the at least one server is further configured to determine a first clock offset for each of the plurality of devices using the current server time, the current device time of the device, and the transmission time of communication between the at least one server and the device.

6. The aforementioned at least one server, Determining a second clock offset for each of the aforementioned multiple devices, The clock drift is determined for each of the plurality of devices by comparing the first clock offset and the second clock offset of the aforementioned device. The system according to claim 5, configured to do the following.

7. The system according to any one of claims 1 to 3, wherein at least one of the devices is a smartphone.

8. The system according to any one of claims 1 to 3, wherein the at least one server is configured to determine the camera period of the cameras of the plurality of devices, and includes triggering at least one group of the plurality of devices to capture video, and measuring the camera period of each device in the group while the group of the plurality of devices is capturing video, and the at least one server determines the camera period of the cameras of the plurality of devices based on the measured camera period for the group of devices.

9. A method for synchronizing video captures from the cameras of multiple devices, Determining the camera cycle of the cameras of the aforementioned plurality of devices, Determining the camera synchronization time, which includes determining the synchronized clock value by adding an integer multiple of the camera period, Using the respective cameras of the aforementioned multiple devices, video is captured on each of the aforementioned multiple devices. In each of the plurality of devices, when the device determines that the camera synchronization time has elapsed, it determines the camera offset by calculating the difference between the timestamp of the current frame captured by the device and the camera synchronization time. Adjusting one or more video capture parameters of one or more of the multiple devices based on the respective camera offsets of one or more of the devices. A method that includes [a certain feature].

10. The method according to claim 9, comprising comparing the camera offset of each of the plurality of devices with a threshold, and adjusting one or more video capture parameters of the device based on the amount by which the camera offset of the device exceeds the threshold.

11. Determining one or more additional camera synchronization times, which includes determining each of the one or more additional camera synchronization times by adding a value that is an integer multiple of the camera period to the immediately preceding camera synchronization time, Capture video using the camera of each of the aforementioned multiple devices, For each of the plurality of devices and for each of the one or more additional camera synchronization times, when the device determines that the additional camera synchronization time has elapsed, the camera offset is determined by calculating the difference between the timestamp of the current frame captured by the device and the additional camera synchronization time, and then one or more video capture parameters of the device are adjusted based on the camera offset. The method according to claim 9, further comprising:

12. The adjustment described above applies to each of the one or more devices, Triggering the device to capture a frame for an exposure time determined based on the camera offset of the device, Temporarily increasing or decreasing the capture period of the device based on the camera offset of the device, and The method according to any one of claims 9 to 11, comprising at least one of stopping video capture by the device for a period of time determined based on the camera offset of the device.

13. A device for capturing video, comprising at least one processor, a camera, and, when executed by the at least one processor, the device, Receiving synchronized clock values ​​from at least one server, Determining the camera synchronization time, which includes adding an integer multiple of the camera period to the synchronized clock value, Capture video using at least one of the aforementioned cameras, The camera synchronization time is determined to have elapsed, and in response, the camera offset is determined by calculating the difference between the timestamp time of the current frame captured by the device and the camera synchronization time. Adjusting one or more video capture parameters based on the aforementioned camera offset, The video is transmitted to the at least one server. A device comprising a computer-readable storage medium having instructions for executing a function.

14. When the instruction is executed by the at least one processor, the device: Comparing the aforementioned camera offset with a threshold, Adjusting one or more video capture parameters based on the amount by which the camera offset exceeds the threshold. The device according to claim 13, which causes to perform the following.

15. When the instruction is executed by the at least one processor, the device: Determining one or more additional camera synchronization times, which includes determining each of the one or more additional camera synchronization times by adding the integer multiple of the camera period to the immediately preceding camera synchronization time, The camera used to capture video, For each of the one or more additional camera synchronization times, when the device determines that the additional camera synchronization time has elapsed, it determines a camera offset by calculating the difference between the timestamp of the current frame captured by the device and the additional camera synchronization time, and then adjusts one or more video capture parameters of the device based on the camera offset. The device according to claim 13, which causes to perform the following.

16. The device adjusts one or more of the video capture parameters. To capture a frame with an exposure time determined based on the camera offset of the device, Temporarily increasing or decreasing the capture period of the device based on the camera offset of the device, and The device according to any one of claims 13 to 15, comprising at least one of stopping video capture by the device for a period of time determined based on the camera offset of the device.

17. The device according to any one of claims 13 to 15, wherein when the instruction is executed by the at least one processor, the device causes the device to transmit the video to the at least one server using the Internet Protocol and / or to receive the synchronization clock value from the at least one server.

18. When the instruction is executed by at least one processor of the device for capturing video, the device: Receiving synchronized clock values ​​from at least one server, Determining the camera synchronization time, which includes adding an integer multiple of the camera period to the synchronized clock value, Capture video using at least one of the aforementioned cameras, The camera synchronization time is determined to have elapsed, and in response, the camera offset is determined by calculating the difference between the timestamp time of the current frame captured by the device and the camera synchronization time. Adjusting one or more video capture parameters based on the aforementioned camera offset, The video is transmitted to the at least one server. A computer program product that contains instructions to execute a command.

19. When the instruction is executed by the at least one processor, the device: Comparing the aforementioned camera offset with a threshold, Adjusting one or more video capture parameters based on the amount by which the camera offset exceeds the threshold. A computer program product according to claim 18, which causes to execute

20. When the instruction is executed by the at least one processor, the device: Determining one or more additional camera synchronization times, which includes determining each of the one or more additional camera synchronization times by adding the integer multiple of the camera period to the immediately preceding camera synchronization time, The camera used to capture video, For each of the one or more additional camera synchronization times, when the device determines that the additional camera synchronization time has elapsed, it determines a camera offset by calculating the difference between the timestamp of the current frame captured by the device and the additional camera synchronization time, and then adjusts one or more video capture parameters of the device based on the camera offset. A computer program product according to claim 18, which causes to execute

21. The device adjusts one or more of the video capture parameters. To capture a frame with an exposure time determined based on the camera offset of the device, Temporarily increasing or decreasing the capture period of the device based on the camera offset of the device, and A computer program product according to any one of claims 18 to 20, comprising at least one of stopping video capture by the device for a period of time determined based on the camera offset of the device.

22. The device according to claim 13, or the computer program product according to claim 18, wherein the device is a smartphone.

Images