Smartphone Motion Tracking System

Abstract

A system for processing videos of sports activities is disclosed. The system includes a plurality of portable electronic devices each having at least one camera for capturing videos of sports activities, and one or more aggregation data processing devices. Each portable electronic device is configured to use its camera to generate video data representing a sports activity, perform processing of the video data to generate video analysis data, and transfer the video analysis data to the aggregation data processing device. The aggregation data processing device is configured to perform processing of the video analysis data to generate activity analysis data. An apparatus for holding the portable electronic devices is also disclosed, and the apparatus includes a housing in which the portable electronic device can be attached and stored, and an active cooling system for cooling the portable electronic device.

Description

Technical Field

[0001] The present invention relates to a method and system for processing videos, particularly videos of sports activities such as sports events and practice sessions. In the embodiments disclosed herein, videos of sports activities are captured from multiple viewpoints.

Background Art

[0002] Sports are an essential source of entertainment and leisure for millions of people around the world. The use of video in sports has revolutionized the way teams and athletes analyze their performance, plan strategies, and improve. Video analysis enables a detailed examination of gameplay, techniques, and tactics, providing valuable insights that can lead to improved performance on the field. Coaches can study the strengths and weaknesses of their teams, identify patterns, and make data-driven decisions to optimize strategies. Players can analyze their own performance, identify areas for improvement, and refine their skills precisely.

[0003] Video technology has also transformed the fan experience, allowing spectators to relive and analyze key moments, gain a deeper understanding of the game, and participate in discussions and debates with other fans.

Summary of the Invention

[0004] According to a first aspect of the present disclosure, there is provided a system for processing videos of sports activities, comprising a plurality of portable electronic devices each having at least one camera for capturing videos of sports activities from the perspective of the electronic device, and one or more aggregated data processing devices in data communication with the plurality of portable electronic devices. Each portable electronic device is configured to generate video data representing a sports activity using at least one camera, perform processing of the video data to generate video analysis data, and transfer the video analysis data to one or more aggregated data processing devices. The one or more aggregated data processing devices are configured to perform processing of the video analysis data from the plurality of portable electronic devices to generate activity analysis data.

[0005] In an embodiment, at least some of the portable electronic devices are mobile devices. Additionally or alternatively, at least some of the portable electronic devices can be smartphones or tablet computing devices.

[0006] According to another aspect of the present disclosure, there is provided a system for analyzing data related to sports activities, comprising one or more aggregated data processing devices. The one or more aggregated data processing devices are configured to receive video analysis data generated by a respective portable electronic device that processes video data generated by at least one camera of the portable electronic device from each of the plurality of portable electronic devices, and perform processing of the video analysis data from the plurality of portable electronic devices to generate activity analysis data.

[0007] According to yet another aspect of the present disclosure, there is provided a portable electronic device comprising at least one processor, at least one camera, and a computer-readable storage medium containing instructions, wherein when the instructions are executed by the at least one processor, the portable electronic device is caused to generate video data representing a sports activity using the at least one camera, perform processing of the video data to generate video analysis data, and transfer the video analysis data to one or more aggregation data processing devices.

[0008] According to another aspect of the present disclosure, there is provided a computer program product containing instructions, wherein when the instructions are executed by at least one processor of a portable electronic device, the portable electronic device is caused to generate video data representing a sports activity using at least one camera of the portable electronic device, perform processing of the video data to generate video analysis data, and transfer the video analysis data to one or more aggregation data processing devices.

[0009] According to yet another aspect of the present disclosure, there is provided an apparatus for holding a portable electronic device comprising at least one camera, the apparatus comprising a housing in which the portable electronic device is attachable, the housing having an interior for storing the portable device, and an active cooling system for cooling the portable electronic device.

[0010] In some embodiments, the housing stores the portable electronic device in a sealing manner. Additionally or alternatively, the housing may cover the touch screen of the stored portable electronic device, thereby making it inaccessible.

[0011] In an embodiment, the active cooling system comprises a fan. Additionally or alternatively, the active cooling system may comprise a pump and one or more conduits, and during use of the device, the pump is used to circulate a coolant through the one or more conduits. Additionally or alternatively, the active cooling system may include a thermoelectric cooling system.

[0012] In the above aspects and embodiments, sports activities are referred to, but in systems, methods, and devices according to other aspects, the activities can be non-sports activities such as non-sports live events (e.g., concerts, comedy shows, or plays) or non-sports practice sessions (e.g., music practice or theater rehearsals).

[0013] Other features and advantages will become apparent from the following description, given by way of example only, with reference to the accompanying drawings.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Modes for Carrying Out the Invention

[0015] Embodiments of the present application relate to the automatic analysis of videos of sports activities, particularly when videos of sports activities are captured from multiple viewpoints.

[0016] Referring first to FIG. 1, FIG. 1 is a schematic diagram showing a video processing system according to an exemplary embodiment. As shown, system 100 includes a plurality of portable electronic devices 110(a)-(f), and each of the plurality of portable electronic devices 110(a)-(f) includes at least one camera 112. Also as shown, each of the portable electronic devices 110(a)-(f) is arranged to capture video from a respective different viewpoint 114. In the particular example shown, system 100 includes six portable electronic devices 110(a)-(f). However, this is of course not essential. With just two or three portable electronic devices 110(a)-(f), acceptable performance can be achieved. Conversely, due to the scalable nature of system 100, it is possible to use a larger number (e.g., ten or more) of portable electronic devices 110(a)-(f) as part of system 100.

[0017] In the particular embodiment shown in FIG. 1, each portable electronic device 110 is a smartphone. However, this is by no means essential, and in another embodiment, some or all of the portable devices 110(a)-(f) can be any suitable type of different portable electronic device 110 having, for example, a tablet computing device, or one or more cameras and sufficient processing power to operate as described herein.

[0018] Modern portable electronic devices 110 having cameras (specifically, smartphones, tablet computing devices, etc.) are readily available, the system 100 can be easily extended by adding another portable electronic device 110(a)-(f), the system 100 can be easily upgraded by leveraging future advancements in the functionality of portable electronic devices, it is simple to configure for use in the system 100 by installing appropriate software on the portable electronic device 110 and using the wireless and / or wired connectivity of the portable electronic device 110, and it is possible to capture video at typically high resolution and / or high frame rate using the camera of such a device and provide significant processing power.

[0019] In the system 100 of FIG. 1, each of the portable electronic devices 110(a)-(f) is configured to perform processing of video data generated by the camera 112 of the device (e.g., by appropriate programming of the processor of each device) to generate video analysis data. As will be discussed in more detail below, the processing can, for example, identify objects or features in the video and / or track motions in the video, and the video analysis data can represent not the video but rather the objects, features, motions, etc. identified as such. Thus, the video analysis data does not include any data representing a video of a sports activity; instead, the video analysis data can represent only the analysis derived from the video data generated by, for example, the camera 112 of the device. Thereby, the bandwidth requirements of the system 100 can be significantly reduced. Further, in some embodiments, the plurality of portable electronic devices 110(a)-(f) can be configured not to transmit any data representing a video of a sports activity to the server 120.

[0020] However, in another embodiment, certain data representing a video of a sports activity may be transmitted to the server by the portable electronic devices 110(a)-(f). Still, in many cases, such video data is expected to have a much lower bit rate than the video data generated by the device's camera 112 so as to reduce the bandwidth requirements of the system 100.

[0021] More generally, it is assumed that the average bit rate for data transfer from each of the portable electronic devices 110(a)-(f) to the server 120 can be much lower than the bit rate of the video data generated by the device's camera 112. For example, the average bit rate for data transfer from each of the portable electronic devices 110(a)-(f) to the server 120 can be less than 50%, less than 25%, less than 10%, less than 5%, or even less than 1% of the bit rate of the video data generated by the device's camera 112.

[0022] Returning to FIG. 1, it can be noted that the system 100 further includes a server 120. As shown by the straight lines connecting the portable electronic devices 110(a)-(f) to the server 120 in FIG. 1, the server 120 is in data communication with the portable electronic devices 110(a)-(f). This enables the portable electronic devices 110(a)-(f) to transmit the video analysis data they each generate to the server 120. The server 120 processes the aggregated video analysis data received from the portable electronic devices 110(a)-(f) to generate activity analysis data.

[0023] In some examples, the server 120 can be located at the same physical location as the portable electronic devices 110(a)-(f). For example, the server 120 can be located in a server room at the venue where the sports activity is taking place. Alternatively, the server 120 can be located in a truck parked at the venue. However, in yet another example, the server 120 can be a remote / cloud server.

[0024] The straight lines connecting the portable electronic devices 110(a)-(f) of FIG. 1 form a loop, but this is merely exemplary and it should be understood that it does not imply any particular topology regarding the data connection between the portable electronic devices 110(a)-(f) and the server 120. (For example, the server 120 is a hub) Various topologies such as hub and spoke topology, mesh topology (especially using wireless connections, in which case the portable electronic devices 110(a)-(f) can be connected to, for example, one or more access points and / or to each other), tree topology (for example, multiple access points or switches are connected to a main line (trunk), and one or more portable electronic devices 110(a)-(f) are connected to each switch (branch)) are considered suitable for such data connections.

[0025] Furthermore, the data connection does not have to be a physical connection, and in some embodiments, the portable electronic devices 110(a)-(f) can communicate wirelessly with the server 120, for example using 5G (however, in other embodiments, communication via 4G or Wi-Fi can be utilized). Nevertheless, in embodiments where at least some of the data connections between the portable electronic devices 110(a)-(f) and the server 120 are physical connections, it is envisioned that such physical connections can be realized (at least in part) using, for example, Power over Ethernet. Advantageously, Power over Ethernet can provide not only data connectivity but also power to the portable electronic devices 110(a)-(f). When the processing of video data performed by the portable electronic devices 110(a)-(f) is particularly computationally intensive and thus consumes significant power, it may be appropriate to enable a continuous supply of power to the portable electronic devices 110(a)-(f).

[0026] The system 100 of FIG. 1 can be utilized, for example, for automated officiating, health and fitness measurements, player performance analysis, and / or metaverse or next-generation media applications. Such applications may, in some cases, require ultra-high-resolution video, ultra-high frame rates, and / or a very large number of camera angles. Usually, this results in significant bandwidth requirements and setup costs for the system.

[0027] The inventors have recognized that the processing power provided by modern portable electronic devices enables such devices to perform complex processing of video data that would typically be carried out by one or more servers in a conventional system. As a result of a novel architecture in which at least some of the video processing is distributed among the plurality of portable electronic devices 110(a)-(f) rather than being performed centrally at the server 120 (or at a plurality of such servers), the system 100 of FIG. 1 can exhibit improved performance, improved scalability, reduced setup costs, and / or reduced bandwidth requirements compared to a conventional system in which a plurality of cameras are connected to one or more servers for processing video data.

[0028] In some embodiments, the processing of video data by each of the portable electronic devices 110(a)-(f) may include using at least one neural network. Processing using neural networks requires significant processing power, especially in embodiments where the neural network is applied frame by frame. Thus, processing using neural networks is a prominent example of the type of processing that can be performed (and distributed among) by the plurality of portable electronic devices 110(a)-(f) in the system 100 of the figure, rather than, for example, at a server. Neural networks can be used for various purposes such as object identification / recognition and feature extraction.

[0029] In some embodiments, the video data processing performed by each of the portable electronic devices 110(a)-(f) utilizes an object identification / recognition algorithm. In some cases, the object identification / recognition algorithm may utilize a neural network such as YOLO or R-CNN. The neural network for object identification / recognition may be specifically used to identify participants in a sports activity (e.g., players, referees, umpires, etc.), body parts of the participants (e.g., the head, hair, torso, feet of the participant, and / or the shoulders, knees, waist, and / or other joints of the participant), and / or sporting objects used in the activity (e.g., a ball in ball sports such as soccer or basketball, a puck in hockey, etc., an object contested in the sports activity, or equipment used by a player to play a specific sports activity such as a stick, club, racket, etc.). The neural network for object identification / recognition may be trained, for example, on images or videos of sports activities labeled to indicate the position of such objects as participants, body parts of the participants, and / or sporting objects as discussed above.

[0030] In some embodiments, the processing of video data by each of the portable electronic devices 110(a)-(f) may include identifying one or more participants in a sports activity. Participant identification may utilize, for example, the output of an object identification / recognition algorithm as described above. In some cases, participant identification may include detecting a jersey number. The jersey number may be mapped to a player name. In that case, the jersey number may be mapped to a player name using, for example, official rosters. Jersey number detection may be performed for each frame. Per-frame jersey number detection may be performed using a support vector machine (SVM), which is a supervised machine learning model used for classification. The SVM may be trained, for example, using a training set of manually marked jersey numbers from videos of the same type of sports activity.

[0031] In some embodiments, the video analysis data generated by each of the portable electronic devices 110(a)-(f) may include motion data representing the actions of the participants in the sports activity, the body parts of such participants, and / or the sports objects used in the sports activity, as viewed from the perspective 114 of the respective portable electronic device 110(a)-(f). Such motion data may be generated at least in part using an object identification / recognition algorithm as described above. For example, a time track may be generated for each object by tracking the position of each object detected by the object identification / recognition algorithm within each video frame.

[0032] When the video analysis data generated by each of the portable electronic devices 110(a)-(f) includes motion data, the processing of the aggregated video analysis data performed by the server 120 may include determining 3D motion data based on the motion data received from the plurality of portable electronic devices 110(a)-(f). In cases where the motion data generated by each of the portable electronic devices 110(a)-(f) is two-dimensional, i.e., the data represents the motion of an object within a frame of the video captured by the device, the processing performed by the server 120 may, for example, utilize triangulation to determine the 3D position of the object (such as a participant in a sports activity, a body part of such a participant, or a sports object used in the sports activity). However, in some cases, certain assumptions may be made about the motion of the participant or the sports object related to the sports activity, so the motion data generated by each of the portable electronic devices 110(a)-(f) may be three-dimensional. In such cases, the processing performed by the server 120 may, for example, combine the respective estimations of the 3D positions of the objects (such as a participant in a sports activity, a body part of such a participant, or a sports object used in the sports activity) provided by the portable electronic devices 110(a)-(f). In some embodiments, the respective estimations may be combined in a weighted manner, for example, based on the probability distribution or confidence level of each estimation, based on the distance of the object from each camera / device, and / or based on the occlusion of each camera.

[0033] In some embodiments, the processing of the video data by each of the portable electronic devices 110(a)-(f) may utilize a compression algorithm. The compression algorithm may be specifically used to compress non-video data included in video analysis data such as motion data, and / or data for identifying a participant in a sports activity.

[0034] In some embodiments, the video data processing performed by each of the portable electronic devices 110(a)-(f) utilizes a spatiotemporal pattern recognition algorithm. Various methods for spatiotemporal pattern recognition are described in U.S. Patent Application Publication No. 2021-0240992, which was assigned to the assignee of the present application and is incorporated herein by reference. As described therein, the spatiotemporal pattern recognition algorithm may, in some embodiments, extract certain "events". The extracted events may be events corresponding to a particular understanding of events within an overall sports event, such as "pick and roll" or "blitz" in the case of basketball. Such events may correspond to actual events during a game and may be the subject of various metrics, analytic tools, and visualizations around the events. Event recognition may be based on pattern recognition by machine learning.

[0035] In another embodiment, spatiotemporal pattern recognition may be performed by the server 120 based on, for example, video analysis data (received from the portable electronic devices 110(a)-(f)) that includes motion data and / or data identifying participants in a sports activity.

[0036] In some embodiments, the video data processing performed by each of the portable electronic devices 110(a)-(f) may utilize a feature extraction algorithm such as a feature pyramid network or a convolutional neural network. Feature data generated by such a feature extraction algorithm may be passed to the server 120, and the server 120 may aggregate the feature data from the plurality of portable electronic devices 110(a)-(f) and supply the aggregated feature data into a deep neural network. The deep neural network may perform identification based on, for example, the aggregated feature data.

[0037] In some cases, it may be important for server 120 to determine the time offset between each of the video analysis data received from multiple portable electronic devices 110(a)-(f). For example, if the video analysis data generated by each of the portable electronic devices 110(a)-(f) includes motion data, in order for server 120 to determine three-dimensional motion data about a particular object (e.g., a participant, a part of a participant, or a sports object, as discussed above) based on the motion data received from two (or three or more) of the portable electronic devices 110(a)-(f), server 120 typically needs to be able to temporally align the motion data from two or more of the portable electronic devices 110(a)-(f). To assist in doing so, techniques similar to those disclosed in U.S. Patent Application No. 18 / 346,355, assigned to the assignee of the present application and incorporated herein by reference, may be utilized. Thus, for example, the video analysis data received from each of the portable electronic devices 110(a)-(f) may include data representing the time shown on the game clock at various instants, data indicating the timing and / or brightness of a camera flash, or data from which the timing and / or brightness of a camera flash can be determined (e.g., data representing the overall pixel brightness or average pixel brightness), and / or data representing changes in an image and / or pattern shown on one or more electronic displays present in the sports activity, or data from which such changes can be determined (e.g., data representing changes in the brightness of some or all of the pixels of a video captured by the portable electronic device corresponding to the electronic display).

[0038] Although the system 100 of FIG. 1 is shown as including only a single server 120, it will be understood, of course, that the system 100 may include multiple servers. Further, in some embodiments, devices other than the server may process some or all of the aggregated data from the plurality of portable electronic devices 110(a)-(f). Specifically, in some examples, it is envisioned that a portable electronic device not currently in use for capturing video of a sports activity may be used to process some or all of such aggregated data. For example, if the system includes a number of portable electronic devices, the burden of processing the aggregated data from the plurality of portable electronic devices 110(a)-(f) that are capturing video of a sports activity may be dynamically shifted to a portable electronic device that is not currently capturing video of the sports activity.

[0039] Since device 110 is portable, it should be further noted that device 110 can be easily installed in various positions and orientations to capture videos of sports activities. This is shown in FIG. 2, which is a perspective view of system 100 when arranged to analyze a basketball game. Similar to the portable electronic device shown in FIG. 1, each of the portable electronic devices 100(a)-(g) shown in FIG. 2 has a corresponding, different viewing point 114 and captures videos from that different viewing point 114. As shown, most of the plurality of portable electronic devices 100(a)-(f) are arranged in various fixed positions and orientations around the basketball court. However, one of the devices 100(g) is worn by a participant in the sports activity and provides a perspective of the sports activity that is likely not available with conventional cameras. The participant can be a player (e.g., enabling the direct collection of data characterizing the player's motion using, for example, an accelerometer, gyroscope, etc. mounted as part of the portable electronic device) or can be a referee, umpire, etc. (e.g., to assist in automatic umpiring). Of course, in the exemplary arrangement shown in FIG. 2, only one of the devices 100(g) is shown as being worn by a participant in the sports activity, but it should be understood that in another arrangement of the system, a plurality (or all) of the portable electronic devices 100(a)-(g) can be worn by participants in the sports activity rather than being arranged in fixed positions and orientations. Further, in some embodiments, some of the portable electronic devices 100(a)-(g) can be held by a coach or even a fan (e.g., the fan's own personal device can function as one of the portable electronic devices 100(a)-(g) of system 100). Additionally, although a basketball game is shown in FIG. 2, this is merely exemplary, and it should be understood that the system 100 of FIG. 1 is suitable for placement in many other types of sports activities.

[0040] Next, referring to FIG. 3, FIG. 3 is an isometric view of an exemplary embodiment of an apparatus 200 for holding portable electronic devices that can be used in the system 100 of FIG. 1. In the particular example shown, the portable electronic devices 110(a)-(f) are smartphones. However, this is merely exemplary, and the apparatus 200 can be configured directly to hold, for example, a tablet computer instead.

[0041] As can be seen from the figure, the apparatus 200 includes a housing 210 within which the portable electronic devices 110(a)-(f) can be mounted. The housing 210 stores the portable electronic devices 110(a)-(f) therein. In particular, in embodiments of the apparatus 200 intended to be disposed externally, the housing 210 can store the portable electronic devices 110(a)-(f) in a sealed manner so that liquid and dust cannot enter the housing 210 and interfere with the operation of the portable electronic devices 110(a)-(f). Further, the housing 210 can cover the touchscreens of the stored portable electronic devices 110(a)-(f), thereby rendering them inaccessible. This prevents direct interaction with the portable electronic devices 110(a)-(f) that may impair the main function of capturing and processing video data in the system of FIG. 1.

[0042] As also shown in FIG. 3, the apparatus 200 further includes an active cooling system 220 for cooling the portable electronic devices 110(a)-(f). The cooling system 220 is active in the sense that it is powered. The active cooling system 220 removes heat from the portable electronic devices 110(a)-(f) that is generated as a result of computationally complex (and thus energy-intensive) video processing performed by the portable electronic devices 110(a)-(f). In the particular embodiment shown in FIG. 3, the cooling system 220 includes a fan 220. In the particular illustrated embodiment, the fan 220 blows air over the portable electronic devices 110(a)-(f) to directly cool the portable electronic devices 110(a)-(f).

[0043] In the embodiment shown in FIG. 3, the apparatus 200 further includes a removable lens cover 230 that is disposed over the cameras of the portable electronic devices 110(a)-(f) when the portable electronic devices 110(a)-(f) are mounted within the housing 210. In some embodiments, the lens cover 230 may be configured to house an external lens, which may provide, for example, additional magnification and / or a wider field of view.

[0044] Turning now to FIG. 4, FIG. 4 is an isometric view of another exemplary embodiment of an apparatus 200' for holding a portable electronic device that may also be used in the system 100 of FIG. 1. In the embodiment shown in FIG. 4, the active cooling system includes not only a fan 220 but also a heat exchanger 225, and the apparatus 200' shown in FIG. 4 is generally the same as the apparatus shown in FIG. 3, except that the fan 220 blows air over the heat exchanger 225. In the particular example shown, the heat exchanger 225 includes a plurality of metal (e.g., aluminum) fins.

[0045] In some embodiments, the devices 200, 200' may comprise a mounting feature that enables the device to be fixed on a conventional camera mounting plate or tripod. In another embodiment, the device 200 may comprise a clip or strap or other fastening device that enables the device to be worn by a participant in a sports activity.

[0046] In some embodiments, the active cooling system may comprise a liquid cooling system, such as a closed-loop liquid cooling system. The liquid cooling system may include a pump and one or more conduits, and during use of the devices 200, 200', the pump is used to circulate a coolant through the one or more conduits. The conduits may pass through or near a heat exchanger, and a fan (such as fan 220) blows air over the heat exchanger. The conduits may also pass through a heat exchanger (such as a metal plate, for example a copper plate) within the housing 210, where heat is transferred from the portable electronic devices 110(a)-(f) to the coolant in the heat exchanger.

[0047] In yet another embodiment of the devices 200, 200', the active cooling system may be or may include a thermoelectric cooling system.

[0048] Next, referring to FIG. 5, FIG. 5 is a schematic cross-sectional view of an apparatus for holding the portable electronic devices shown in FIG. 3. The cross-sectional view enables viewing of the portable electronic devices 110(a)-(f) stored within the housing 210 of the apparatus 200. In the particular illustrated embodiment, the apparatus 200 includes a Power over Ethernet connector 240 that is accessible from outside the housing. This enables the apparatus (and thereby the portable electronic devices 110(a)-(f)) to be connected to the server 120 using an Ethernet cable. As described above, advantageously, Power over Ethernet provides not only data connectivity but also power to the portable electronic devices 110(a)-(f). Also as shown in FIG. 5, the apparatus 200 further includes a device connector 250 (such as a USB connector) for connecting to the portable electronic devices 110(a)-(f). As can be seen from FIG. 3, the device connector 250 is disposed inside the housing 210 and is electrically connected to the Power over Ethernet connector 240 so that when the portable electronic devices 110(a)-(f) are connected to the device connector 250, data and power can be communicated from the Power over Ethernet connector 240 to the portable electronic devices 110(a)-(f) via the device connector 250. As further shown in FIG. 5, the apparatus further includes a converter unit 260 that converts the Power over Ethernet signal into a form compatible with the portable electronic devices 110(a)-(f), such as a USB signal. Advantageously, the power delivered by the Power over Ethernet connection can be used to power other components of the apparatus 200, specifically an active cooling system and the like.

[0049] In the embodiments described above, while referring to FIGS. 1-5, videos of sports activities are captured. It should be noted that many embodiments may be adapted to capture and analyze videos of non-sports activities such as non-sports live events (e.g., concerts, comedy shows, or plays) and non-sports practice sessions (e.g., music practice or theater rehearsals).

[0050] It should be understood that any feature described in connection with any one embodiment may be used alone or in combination with any other feature described, and may further be used in combination with one or more features of any other embodiment of the embodiments, or any combination of other embodiments of the embodiments. Additionally, equivalents and modifications not described above may also be utilized without departing from the scope of the invention as defined in the appended claims.

Claims

Claim 1 A system for processing videos of sports activities, comprising: a plurality of portable electronic devices each having at least one camera for capturing videos of sports activities from the perspective of the electronic device; one or more aggregated data processing devices in data communication with the plurality of portable electronic devices; wherein each portable electronic device is configured to: generate video data representing the sports activity using the at least one camera; perform processing of the video data to generate video analysis data; transfer the video analysis data to the one or more aggregated data processing devices; and wherein the one or more aggregated data processing devices are configured to: perform processing of the video analysis data received from the plurality of portable electronic devices to generate activity analysis data. A system according to claim 1, wherein the processing of the video data by each portable electronic device includes using at least one neural network. A system according to claim 1 or 2, wherein the processing of the video data by each portable electronic device includes using an object recognition algorithm. A system according to any one of claims 1 to 3, wherein the processing of the video data by each portable electronic device includes using a spatio-temporal pattern recognition algorithm. A system according to any one of claims 1 to 4, wherein the processing of the video data by each portable electronic device includes using a feature extraction algorithm. Claim 6 The video analysis data generated by each portable electronic device includes motion data, and the motion data represents at least one motion of: one or more sports objects used by a participant in the sports activity, one or more participants in the sports activity, and one or more body parts of the participant in the sports activity, as viewed from the perspective of the portable electronic device. A system according to any one of claims 1 to 5, wherein the processing of the video analysis data includes determining three-dimensional motion data based on the motion data received from the plurality of portable electronic devices. Claim 8 ​ ​ ​ ​ ​ ​ ​ ​ The system of claim 7, wherein determining the three-dimensional motion data includes using a triangulation algorithm.

9. The system according to any one of claims 1 to 8, wherein the video analysis data includes participant identification data, and the participant identification data indicates the identification of one or more participants in the sports activity.

10. Each portable electronic device is further configured to transfer reduced bitrate video data to the one or more aggregated data processing devices, and the reduced bitrate video data has a lower bitrate than the video data representing the sports activity generated by the at least one camera. The system according to any one of claims 1 to 9.

11. The system according to any one of claims 1 to 10, wherein the plurality of portable electronic devices are configured not to transmit any data representing a video of the sports activity to the one or more aggregated data processing devices.

12. The system according to any one of claims 1 to 11, wherein at least some of the one or more aggregated data processing devices are servers.

13. The system according to any one of claims 1 to 12, wherein at least some of the portable devices are smartphones.

14. The system according to any one of claims 1 to 13, wherein at least some of the portable electronic devices receive power and communicate data with one or more aggregated data processing devices using respective Power over Ethernet connections.

15. The system according to any one of claims 1 to 14, wherein at least some of the portable devices communicate data with the one or more aggregated data processing devices using 5G.

16. The system according to any one of claims 1 to 15, wherein the plurality of portable electronic devices includes at least five portable electronic devices.

17. A system for analyzing data related to a sports activity, comprising one or more aggregated data processing devices, wherein the one or more aggregated data processing devices are Receiving, from each of a plurality of portable electronic devices, video analysis data generated by the portable electronic device that processes video data generated by at least one camera of the portable electronic device, performing processing of the video analysis data from the plurality of portable electronic devices to generate activity analysis data A system configured as follows.

18. The video analysis data generated by each portable electronic device includes motion data, and the motion data is from the perspective of the portable electronic device, one or more sports objects used by the participants in the sports activity, and one or more participants in the sports activity one or more body parts of the participants in the sports activity The system according to claim 17, wherein the system represents at least one motion among them.

19. The system according to claim 18, wherein the processing of the video analysis data includes determining three-dimensional motion data based on the motion data received from the plurality of portable electronic devices.

20. The system according to claim 19, wherein determining the three-dimensional motion data includes using a triangulation algorithm.

21. The system according to any one of claims 17 to 20, wherein the video analysis data includes participant identification data, and the participant identification data indicates the identification of one or more participants in the sports activity.

22. The system according to any one of claims 17 to 21, wherein at least some of the one or more aggregated data processing devices are servers.

23. The system according to any one of claims 17 to 22, wherein at least some of the one or more aggregated data processing devices communicate with the portable electronic device using respective Power over Ethernet connections.

24. The system according to any one of claims 17 to 23, wherein at least some of the one or more aggregated data processing devices communicate with the portable electronic device using 5G.

25. A portable electronic device comprising at least one processor, at least one camera, and a computer-readable storage medium containing instructions, wherein when the instructions are executed by the at least one processor, the portable electronic device is caused to generate video data representing a sports activity using the at least one camera, perform processing of the video data to generate video analysis data, transfer the video analysis data to the one or more aggregated data processing devices, a portable electronic device.

26. The portable electronic device according to claim 25, wherein the processing of the video data by the portable electronic device includes using at least one neural network.

27. The portable electronic device according to claim 25 or 26, wherein the processing of the video data by the portable electronic device includes using an object recognition algorithm.

28. The portable electronic device according to any one of claims 25 to 27, wherein the processing of the video data by the portable electronic device includes using a spatio-temporal pattern recognition algorithm.

29. The portable electronic device according to any one of claims 25 to 28, wherein the processing of the video data by the portable electronic device includes using a feature extraction algorithm.

30. The video analysis data generated by the portable electronic device includes motion data, and the motion data is from the perspective of the portable electronic device, one or more sports objects used by the participants in the sports activity, one or more participants in the sports activity, and one or more body parts of the participants in the sports activity representing at least one motion of, the portable electronic device according to any one of claims 25 to 29.

31. The portable electronic device according to any one of claims 25 to 30, wherein the video analysis data includes participant identification data, and the participant identification data indicates the identification of one or more participants in the sports activity.

32. A computer program product containing instructions, wherein when the instructions are executed by at least one processor of a portable electronic device, the portable electronic device is caused to Using at least one camera of the portable electronic device to generate video data representing a sports activity, Performing processing of the video data to generate video analysis data, A computer program product that transfers the video analysis data to the one or more aggregated data processing devices.

33. The computer program product according to claim 32, wherein the processing of the video data by the portable electronic device includes using at least one neural network.

34. The computer program product according to claim 32 or 32, wherein the processing of the video data by the portable electronic device includes using an object recognition algorithm.

35. The computer program product according to any one of claims 32 to 34, wherein the processing of the video data by the portable electronic device includes using a spatio-temporal pattern recognition algorithm.

36. The computer program product according to any one of claims 32 to 35, wherein the processing of the video data by the portable electronic device includes using a feature extraction algorithm.

37. The video analysis data generated by the portable electronic device includes motion data, and the motion data is from the perspective of the portable electronic device, One or more sports objects used by the participants in the sports activity, One or more participants in the sports activity, and One or more body parts of the participants in the sports activity The computer program product according to any one of claims 32 to 36, which represents at least one motion of.

38. The computer program product according to any one of claims 32 to 37, wherein the video analysis data includes participant identification data, and the participant identification data indicates the identification of one or more participants in the sports activity.

39. An apparatus for holding a portable electronic device having at least one camera, A housing in which the portable electronic device can be mounted, the housing enclosing the portable device inside, and An active cooling system for cooling the portable electronic device An apparatus comprising.

40. The apparatus according to claim 39, further comprising a Power over Ethernet connector accessible from outside the housing and a device connector for connecting to the portable electronic device, wherein the device connector is disposed inside the housing and electrically connected to the Power over Ethernet connector, such that when the portable electronic device is connected to the device connector, data and power can be communicated from the Power over Ethernet connector to the portable electronic device via the device connector.

41. The apparatus according to claim 40, wherein the active cooling system is electrically connected to the Power over Ethernet connector so as to receive power via the Power over Ethernet connector.

42. The apparatus according to any one of claims 39 to 41, wherein the active cooling system includes a fan.

43. The apparatus according to any one of claims 39 to 42, wherein the active cooling system includes a pump and one or more conduits, and during use of the apparatus, the pump is used to circulate a coolant through the one or more conduits.

Images