Data processing apparatus, wireless data capture device and methods

Abstract

A data processing apparatus comprising circuitry configured to: receive, during a sports event, characteristic data indicative of a characteristic of each of a plurality of wireless data capture devices for capturing sports data during the sports event; determine, using the characteristic data, a subset of the wireless data capture devices to transmit the captured sports data over a bandwidth-limited wireless communication path; and control transmission of control data to the subset of the wireless data capture devices to control the subset of the wireless captured devices to transmit the captured sports data over the bandwidth-limited communication path.

Description

[0001] DATA PROCESSING APPARATUS, WIRELESS DATA CAPTURE DEVICE AND METHODS

[0002] BACKGROUND

[0003] Field of the Disclosure

[0004] The present disclosure relates to a data processing apparatus, a wireless data capture device and methods.

[0005] Description of the Related Art

[0006] The “background” description provided is for the purpose of generally presenting the context of the disclosure. Work of the presently named inventors, to the extent it is described in the background section, as well as aspects of the description which may not otherwise qualify as prior art at the time of filing, are neither expressly or impliedly admitted as prior art against the present disclosure.

[0007] The progression of cloud computing and remote production has created a scenario where there is a desire to transmit more data from a sports event (e.g. data captured by an increasing number of cameras and / or sensors) for off-site processing than present communications infrastructure (e.g. cellular and / or wireless local area networks) can support. This effectively caps the amount of data that can be transmitted at any moment in time, thereby potentially creating undesirable latency. For video capture, a capped data rate also restricts how many unique cameras and therefore unique perspectives can be deployed. This is a particular problem for live sports broadcasting, for example, in which low latency is required for viewers to experience the broadcast without perceiving any significant delay and in which viewers expect to be able to view content from many different views.

[0008] There is therefore a desire to improve traffic management and delivery efficiency of sports data to enable such data to be processed in a timely and effective manner despite these limits of communications infrastructure.

[0009] SUMMARY

[0010] The present disclosure is defined by the claims.

[0011] BRIEF DESCRIPTION OF THE DRAWINGS

[0012] Non-limiting embodiments and advantages of the present disclosure are explained with reference to the following detailed description taken in conjunction with the accompanying drawings, wherein:

[0013] Fig. 1 schematically shows an example sports event in which data is collected;

[0014] Fig. 2 schematically shows an example implementation of the present technology; Figs. 3A and 3B schematically show components of a data processing apparatuses;

[0015] Fig. 4 schematically shows an example of how control data is generated;

[0016] Fig. 5 shows a simple example table associating control data characteristics and ranked object identifiers;

[0017] Fig. 6 shows a simple example method executed by a mobile device; and

[0018] Figs. 7A and 7B show further example methods.

[0019] Like reference numerals designate identical or corresponding parts throughout the drawings.

[0020] DETAILED DESCRIPTION OF THE EMBODIMENTS

[0021] Fig. 1 shows an example sports event in which data is collected. There is a desire for this data to be transmitted to a remote processing location (e.g. a cloud or remote production service, not shown) for processing and / or distribution (e.g. for broadcast to viewers and / or for viewing by a remote sports official). For example, greater data processing power, more efficient data processing and / or data processing at lower cost is often available by using a remote processing location such as a cloud service.

[0022] In this example, the sports event is a soccer match occurring on a soccer pitch 101. There are two teams of players. Players 106A are players on the first team (Team A) and players 106B are players on the second team (Team B). Three on-site match officials 105 (a referee and two assistant referees) officiate the match (optionally, with assistance from one or more remote match officials). The position of the soccer ball 104 is also shown.

[0023] A plurality of fixed cameras 102 are positioned around the pitch to capture images (in particular, video images represented by video data) of the match. Since the cameras 102 are fixed, they may be configured to transmit captured image data via a wired connection (e.g. an Ethernet ® connection).

[0024] A plurality of mobile cameras 107 are also provided. In this example, the mobile cameras 107 are wearable cameras worn by each player and each on-site match official (meaning there are 25 such mobile cameras overall, in this example). In this example, each mobile camera 107 attaches with straps 108 to the torso 109 (although other configurations such as head-mountable cameras or the like may be used). Like the fixed cameras 102, the mobile cameras 107 capture images (in particular, video images represented by video data) and, because they are worn by each player and match official (rather than being fixed like cameras 102), the captured images provide a different and dynamic perspective of the match.

[0025] However, since the cameras are wearable, they are configured to transmit captured image data via a wireless connection (e.g. a cellular and / or wireless local area network, WLAN, connection) rather than a wired connection. In this case, wireless base stations 103 (e.g. cellular base stations such as gNodeBs and / or WLAN base stations such as Wi-Fi ® access points) are positioned in a vicinity of the pitch 101 to transmit signals to and / or receive signals from each of the mobile cameras 107. Wireless connections typically have lower available bandwidth, especially when wireless network capacity is shared between multiple users (e.g. users in the audience as well as the service provider of the mobile cameras 107). This means it may not be possible for all data captured by the mobile cameras 107 to be transmitted over the wireless network in real time (particularly if, for example, each of the 25 cameras is simultaneously attempting to transmit High Definition or Ultra High Definition video images over the network). There is therefore a need to prioritise which data is transmitted from which camera depending on what is happening in the soccer match.

[0026] Fig. 2 shows an example implementation of the present technology to the soccer match of Fig. 1.

[0027] One or more wireless base stations 103 (e.g. a cellular or WLAN base station) transmit signals to and receives signal from the mobile cameras 107. The signals transmitted to the mobile cameras 107 comprise control data 201. The signals received from the mobile cameras 107 comprise captured data 202. A wireless communication path is thus established between the mobile cameras 107 and wireless base station(s) 103. This wireless communication path is bandwidthlimited.

[0028] In the below examples, a single wireless base station 103 is discussed for simplicity. However, it will be appreciated the same teachings are applicable when multiple wireless base stations are used. In this case, for example, each mobile camera 107 is served by one of the multiple wireless base stations and each wireless base station transmits control data to and receives captured data from the mobile camera(s) it serves.

[0029] The captured data 202 comprises video, audio and / or sensor data 202 captured by the mobile cameras 107 during the sports event. It may therefore be referred to as sports data. For simplicity, in the below examples, video data is discussed in detail. However, it will be appreciated that similar principles to those described may be applied to audio and / or sensor data. Audio data is captured by a microphone comprised by or connected to each mobile camera 107, for example. Sensor data is captured by one or more sensors comprised by or connected to each mobile camera 107, for example. An example of sensor data is physiological data such as heart rate data (captured by a heart rate sensor such as an electrocardiographical or photoplethysmographical sensor), positioning data (e.g. Global Navigation Satellite System, GNSS, data captured by a GNSS sensor I receiver) and / or accelerometer data (captured by an accelerometer). It will also be appreciated that the captured data 202 may comprise only audio and / or sensor data 202 (that is, no video data). Although the described examples refer to mobile cameras, the present technology is thus applicable more generally to mobile devices (e.g. wearable devices) configured to capture and wireless transmit data during the match for remote processing. A mobile camera 107 must thus be referred to, more generally, as a mobile device 107 or wireless data capture device / apparatus 107.

[0030] The wireless base station 103 is in communication with a control apparatus 204 via a communication path 205. In an example, the control apparatus 204 is itself wirelessly connected to the wireless base station 103 (and therefore the communication path 205 is a wireless communication path). In this case, the control apparatus 204 may be served by the wireless base station 103 in the same way that the mobile cameras 107 are served by the wireless base station.

[0031] In another example, the control apparatus 204 is connected (via a wired or wireless connection) to a wider network (such as the internet) to which the wireless base station 103 (and any other wireless base stations of the wireless network via which the control data 201 is transmitted to and the captured data 202 is received from the mobile cameras 107) is also connected. In this case, the communication path 205 is a network communication path. For instance, the communication path 205 may be a Wi-Fi or Ethernet communication path connecting the control apparatus 204 to the wider network via a suitable access point or router.

[0032] The communication path 205 allows control data 201 to be generated by and transmitted from the control apparatus 204. The control data 201 is then routed to the wireless base station 103 for transmission to the mobile camera devices 107. The communication path 205 also allows network status data (such as the total bandwidth available to the mobile cameras 107 and / or to each mobile camera 107) to be transmitted from the wireless network (e.g. from wireless base station 103) to the control apparatus 204. Alternatively, the control apparatus 204 may be configured in advance with a fixed network status (e.g. a fixed total bandwidth available to the mobile cameras 107 and / or to each mobile camera 107 as guaranteed by a service level agreement with the network provider).

[0033] The control apparatus 204 is in communication with a tracking apparatus 203. The tracking apparatus tracks the position on the pitch 101 (e.g. x, y and z coordinates) of objects including each of the players 106A and 106B on each team, officials 105 and ball 104. In an example, the tracking uses images (in particular, video images) captured by the fixed cameras 102 which are provided as inputs to any suitable known tracking system (such as the SkeleTRACK system from Hawk-Eye ® Innovations) to generate the real time position (indicated as position data) of each tracked object. This is provided (e.g. via a suitable wired or wireless connection) as tracking data 206 to the control apparatus 204.

[0034] Based on the tracking data 206, the control apparatus 204 generates the control data 201 which is to be transmitted to the mobile cameras 107. Example method(s) of generating the control data 201 from the tracking data 206 are described later.

[0035] The control data 201 indicates to each of the mobile cameras 107 whether captured data is to be transmitted by that camera, which captured data is to be transmitted (e.g. video data, audio data and / or sensor data) and / or one or more transmission characteristics of the transmitted data.

[0036] The transmission characteristic(s) are adjustable according to the transmission bandwidth available to each mobile camera 107. In general, the transmission characteristic(s) adjust the rate of data transmission by each mobile camera 107. For video data, the transmission characteristic(s) may include a frame resolution (e.g. with a higher resolution when more bandwidth is available and a lower resolution when less bandwidth is available), a frame rate (e.g. with a higher frame rate when more bandwidth is available and a lower frame rate when less bandwidth is available) and / or a compression technique (e.g. with a lower compression rate, lossless technique used when more bandwidth is available and a higher compression rate, lossy technique used when less bandwidth is available) of the transmitted video data, for example.

[0037] For audio data, the transmission characteristic(s) may include a sample rate (e.g. with a higher sample rate when more bandwidth is available and a lower sample rate when less bandwidth is available) and / or a compression technique (e.g. with a lower compression rate, lossless technique used when more bandwidth is available and a higher compression rate, lossy technique used when less bandwidth is available) of the transmitted audio data, for example.

[0038] For sensor data, the transmission characteristic(s) may include a rate at which the sensor data is transmitted, for example. For instance, if heart rate data is transmitted, then every detected heart rate value may be transmitted when more bandwidth is available whereas only one in every n detected heart rate values (where n > 1) may be transmitted when less bandwidth is available.

[0039] In an example, the control apparatus 204 generates control data packets for each mobile camera 107 comprising the control data for that mobile camera 107. Each control packet comprises a mobile identifier of the mobile camera 107 it is intended for and the control data for that mobile camera 107. The mobile identifier may be any identifier which enables the mobile cameras 107 to be distinguished from each other. For example, the mobile identifier may be a User Equipment (UE) Identifier (ID) such as an International Mobile Subscriber Identity (IMSI) or Subscription Permanent Identifier (SUPI). The mobile identifier may also be a relevant part of or be derived from the IMSI or SUPI. For example, the Mobile Subscriber Identification Number (MSIN) of each mobile camera 107 may be used as the mobile identifier.

[0040] In an example, the control apparatus 204 may receive, as part of the tracking data, a respective object identifier (object ID) of each player and official who is tracked. These object identifiers are identifiers generated by the tracking apparatus 203. In this case, a lookup table is made available to the control apparatus 204 associating each object identifier associated with a player or official with the mobile identifier of the mobile camera 107 worn by that player or official. The lookup table is populated in advance, for example, when mobile cameras 107 are handed out to the players and officials before the match. The lookup table thus enables object identifiers used by the tracking apparatus 203 to be converted to mobile identifiers for the transmission of control data to the mobile cameras 107 by the wireless network.

[0041] Captured data 202 received by the wireless network via the base station(s) 103 is then transmitted (e.g. over a wider network such as the internet) to the necessary location(s) for processing. In this example, the transmission and processing of video data 211 of the captured data 202 is considered. The video data 211 is transmitted, in this example, to a broadcast apparatus 210 and an archive apparatus 212. The broadcast apparatus 210 receives the video data 211 and (under the control of broadcast director 208, for example) outputs at least a portion of the video data 211 for broadcast as live and / or replay video data 209. For example, the live and / or replay video data 209 is output for broadcast to consumer devices 213 such as televisions, laptops, smartphones or tablets via radio frequency (RF) broadcast and / or streaming over a network such as the internet.

[0042] The live and / or replay video data 209 is also output for viewing by the broadcast director 208 who may use it to feedback editorial data 207 to the control apparatus 204. The editorial data 207 comprises, for example, one or more identifiers of player(s) and / or official(s) wearing mobile camera(s) 107 whose captured data is to be prioritised over that determined automatically by the control apparatus 204 using the tracking data 206. This allows the broadcast director 208 to override the automatic prioritisation carried out by the control apparatus 204 if, for example, an unexpected event occurs in the game (e.g. an altercation between players at the opposite end of the pitch from the ball) which is not detectable from the tracking data 206 in a timely manner. The video captured by the fixed cameras 102 is also made available to the broadcast director 208 (and any team they have) to help them be aware of such unexpected events. Furthermore, if video data from one of the mobile cameras 107 is classed as “on air” or “preview” video data (indicating, respectively, the video data currently being broadcast and the video data which may be broadcast next), the editorial data 207 may cause the video data from this “on air” or “preview” mobile camera to be prioritised over that determined automatically by the control apparatus 204. Video data received from one or more mobile cameras 107 (e.g. “on air” or “preview” mobile video data) may be output for display with fixed camera video data (e.g. in a split screen or picture-in-picture (PIP) format) to facilitate control by the broadcast director 208, for example.

[0043] In an example, the live and / or replay video data 209 is a portion of video data 211 with one or more predetermined broadcast characteristics (as determined by the broadcast apparatus 210) which make the video suitable for immediate broadcast. For example, the video data may be received as packets of video data indicating the identifier (e.g. Ul ID) of the mobile camera 107 from which it was captured and a timestamp at which it was captured. Only video data packets with a timestamp within a predetermined time period (e.g. 5, 10 or 30 seconds) of the current time are then provided as live and / or replay video data 209. In addition, the live and / or replay video data 209 may have predetermined transmission characteristic(s) to ensure a minimum broadcast quality threshold is adhered to. For example, only a portion of the video data 211 transmitted using lossless compression with a minimum frame resolution and / or frame rate may be classed as live and / or replay video data 209.

[0044] All received video data 211 is also transmitted to the archive apparatus 212 for storage. The stored video data includes both the live and / or replay video data 209 and a remaining, lower priority portion of the received video data 211 (that is, the portion of the video data which did not meet the broadcast characteristic(s)). In an example, the lower priority video data is temporarily stored locally on each mobile camera 107 and included in the captured data 202 only when there is sufficient bandwidth for transmission of this data (e.g. after the match, during a break in play such as at half time or when the mobile camera 107 is in range of a higher bandwidth but lower range wireless base station such as a Wi-Fi base station in the entrance tunnel of a stadium or placed behind the goal(s)). The lower priority video data may thus be received some time after it was captured but this is acceptable since its lower priority status means it is less likely to be timesensitive. The stored video data 211 may also be made available for output to devices 213 for use in a number of potential applications.

[0045] Such applications may include consumer and / or business applications could include, for example, medical review, coaching review, marketing I public relations review (e.g. by club and / or individual player staff) prior to distribution to social media, in-game officiating reviews (e.g. Video Assistant Referee, VAR), post-game citing (e.g. serious foul play that was missed or evidence for an appeal), officials performance review (e.g. soccer referee peer review by another senior referee) or security review (e.g. aftermath of pitch invasions, crowd trouble or the like). As previously mentioned, although only video data 211 is shown in Fig. 2 as being stored in the archive apparatus 212, it will be appreciated that, if other types of data (e.g. audio and / or sensor data) are captured by the mobile devices 107, this data may also be stored in the archive apparatus 212. This may be particular useful for certain applications. For example, for a medical review, sensor and audio data may be as important as video data in completing a meaningful and accurate review.

[0046] Figs. 3A and 3B show example components of the control apparatus 204 and a mobile device 107, respectively.

[0047] The control apparatus 204 is a data processing apparatus I device comprising a processor 301 for executing electronic instructions, a memory 302 (e.g. volatile memory) for storing the electronic instructions to be executed and electronic input and output information associated with the electronic instructions, a storage medium 303 (e.g. non-volatile memory) for long term (persistent) storage of information, a communication interface 304 for sending information to and / or receiving information from one or more other apparatuses (e.g. wireless base station(s) 103) and a user interface 305 (e.g. a touch screen, a non-touch screen, button(s), a keyboard and / or a mouse) for receiving commands from and / or outputting information to a user. Each of the processor 301 , memory 302, storage medium 303, communication interface 304 and user interface 305 are implemented using appropriate circuitry, for example. The processor 301 controls the operation of each of the memory 302, storage medium 303, communication interface 304 and user interface 305.

[0048] The mobile device 107 is a data processing apparatus I device comprising a processor 306 for executing electronic instructions, a memory 307 (e.g. volatile memory) for storing the electronic instructions to be executed and electronic input and output information associated with the electronic instructions, a storage medium 308 (e.g. non-volatile memory) for long term (persistent) storage of information, a communication interface 309 for sending information to and / or receiving information from one or more other apparatuses (e.g. wireless base station(s) 103) and a user interface 310 (e.g. a touch screen, a non-touch screen and / or button(s)) for receiving commands from and / or outputting information to a user. The mobile device 107 also comprises one or more of an imaging device 311 (e.g. an imaging sensor such as a Complementary Metal-Oxide- Semiconductor (CMOS) sensor for capturing images using one or more lenses (not shown)), sensor(s) 312 (e.g. heart rate sensor, GNSS sensor I receiver and / or accelerometer) and a microphone 313 (for capturing audio data). Each of the processor 306, memory 307, storage medium 308, communication interface 309, user interface 310, imaging device 311 , sensor(s) 312 and microphone 313 are implemented using appropriate circuitry, for example. The processor 308 controls the operation of each of the memory 307, storage medium 308, communication interface 309, user interface 310, imaging device 311 , sensor(s) 312 and microphone 313. In the given examples, the mobile device 107 is a mobile camera and therefore comprises the imaging device 311.

[0049] Fig. 4 shows how the control apparatus 204 generates the control data 201 to be transmitted to each of the mobile cameras 107 worn by the players 106A and 106B and officials 105 in Fig. 1. The control data 201 is generated for each mobile camera 107 by ranking the mobile cameras based on the tracking data 206 of the ball 104 and of the players 106A and 106B and officials 105 wearing the mobile cameras 107. There are 25 mobile cameras 107 in this example and thus the cameras are ranked from 1 (most important or highest priority) to 25 (least important or lowest priority). The rankings are shown in Fig. 4 and are based on one or more characteristics of the ball 104, players 106A and 106B and officials 105 derivable from the tracking data 206. Examples of such characteristics (at least some of which may be used in combination with each other) are given below. The players 106A and 106 and officials 105 may be referred to more generally as participants of the sports event.

[0050] One example characteristic derivable from the tracking data 206 is a location of the ball 104 on the pitch 101. For example, this allows participants in a same area of the pitch as the ball (e.g. the same half) to be prioritised over participants in a different area of the pitch (e.g. the other half).

[0051] Another example characteristic is the location of each participant in relation to the ball (e.g. the distance from the ball of each participant). For example, this allows participants closer to the ball to be prioritised over participants further from the ball.

[0052] Another example characteristic is the absolute location of each player. This allows, for example, an area of the pitch in which each player is located to be determined and prioritisation to be based on these areas. For example, if there are multiple pitch zones in the form of channels running along the length of the pitch (i.e. from one goal to the other) and a player with the ball (determined as the player closest to the ball, for example) is located in either of the outermost channels (that is, the left wing or rightwing channel) and in a half of the opposing team, then it may be determined that a cross ball is likely to be played. Prioritisation is thus given to players in the penalty area of the opposing team, since these are the players most likely to be the recipients of such a cross ball.

[0053] Another example characteristic is the absolute location of each official. For example, this allows the assistant referee in the half of the pitch in which the ball is located to be prioritised over the assistant referee in the other half of the pitch. Officials may also be prioritised over players when the ball 104 is out of play (i.e. when it is determined the ball is no longer positioned within the outer boundary lines of the pitch 101).

[0054] Another example characteristic is which team is in possession of the ball (based on, for example, the team of the player closest to the ball). For example, this allows players on the team in possession to be prioritised over players on the other team.

[0055] Another example characteristic is a location of the current offside line (which moves with the second-to-last defending player of a team in that team’s half of the pitch 101). This allows, for example, the prioritisation of players of an attacking team in a defending team’s half who are closer to and / or moving towards the current offside line over players further from and / or moving away from the current offside line.

[0056] Another example characteristic is the role and / or team of each player. It is noted the tracking data 206 identifies each participant whose position is tracked using a respective object identifier. Information such as the team of a player and / or a role of a player may therefore be looked up (e.g. in a suitable lookup table accessible to the tracking apparatus 203) using that player’s object identifier. Knowing the role and / or team of each player allows, for example, a player on a particular team with a particular role to be prioritised over other players in particular in-game situations. For instance, a goalkeeper of the defending team may be prioritised over other players during certain set piece events (e.g. corner kicks or free kicks of the attacking team) or if, for example, they leave the defending teams penalty area while the ball is still in play.

[0057] Another example characteristic is the trajectory and / or speed of the participants and / or ball. This involves using the tracking data captured at multiple successive times (e.g. in multiple successive images captured by the fixed cameras 102) to calculate the trajectory (e.g. current direction of travel) and / or speed of tracked objects. This allows, for example, faster participants travelling in the direction of the ball (who are more likely to be involved in a current attacking or defending activity) to be prioritised over slower participants and / or those moving away from the ball. Pose data of participants (e.g. obtained using SkeleTRACK) may also be used to help determine whether a participant is moving towards or away from the ball, for example.

[0058] Other data associated with the object identifier of each participant (e.g. as again stored in a lookup table accessible to the control apparatus 204) may also be used with the tracking data 206. For instance, a playing style of each player (indicated as one of a plurality of predetermined playing styles such as “fast dribbler”, “accurate passer” or “robust defender”) may be used together with other suitable information (such as team information and ball and / or player position information) to prioritise particular players over others. For example, a player of a defending team classified as a “robust defender” may be prioritised over other players when the ball is in that defending team’s penalty area (due to the likelihood of that player engaging in a tackle against an attacking player currently with the ball). In another example, a player of an attacking team in possession of the ball and classified as an “accurate passer” may be given a lower priority than other players on the attacking team within a passing range of the player (due to the likelihood the player of that player quickly and accurately passing the ball to one of the other players). It will be appreciated these are only examples and other types of participant characteristic(s) may be recorded against the object identifier of each participant and used appropriately.

[0059] In the example of Fig. 4, the ranking has been calculated as follows. It will be appreciated this is only one example ranking technique and other suitable ranking technique(s) (using the one or more characteristics derivable from the tracking data) may be used.

[0060] The player in possession of the ball (determined as the player closest to the ball, for example), for example, is given rank 1 .

[0061] The defending goalkeeper is given rank 2. In an example, this is conditional on the ball being in a predetermined area of the pitch (e.g. the final third) comprising the goal defended by the defending goalkeeper. This condition is met in the example of Fig. 4.

[0062] One or more players deemed most likely to be involved in an attack are respectively assigned the next successive ranks. For instance, all player(s) on the team of the player in possession of the ball (deemed the attacking team) and within the predetermined area of the pitch (e.g. the final third) comprising the goal defended by the defending goalkeeper are assigned the next ranks. In this case, there are four such players who are respectively assigned ranks 3, 4, 5 and 6. These ranks may be assigned based on, for example, each player’s proximity to the ball (so a player closer to the ball is ranked before a player further from the ball). This is the case here, where the player closest to the ball is given rank 3, the next closest player is given rank 4, the next closest player is given rank 5 and the furthest player is given rank 6.

[0063] One or more players deemed most likely to be involved in a defence are respectively assigned the next successive ranks. For instance, all player(s) not on the team of the player in possession of the ball (and therefore deemed as being on the defending team) and within the predetermined area of the pitch (e.g. the final third) comprising the goal defended by the defending goalkeeper are assigned the next ranks. In this case, there are three such players who are respectively assigned ranks 7, 8, 9 and 10. These ranks may again be assigned based on, for example, each player’s proximity to the ball (so a player closer to the ball is ranked before a player further from the ball). This is the case here, where the player closest to the ball is given rank 7, the next closest player is given rank 8, the next closest player is given rank 9 and the furthest player is given rank 10.

[0064] The referee is given rank 11 .

[0065] The assistant referee for the half of the pitch in which the ball is currently located is given rank 12.

[0066] All remaining outfield players of the attacking and defending teams are then respectively assigned the next successive ranks. These ranks may again be assigned based on, for example, each player’s proximity to the ball (so a player closer to the ball is ranked before a player further from the ball). This is the case here, where the remaining players are respectively ranked from 13 to 23 with higher ranked players (e.g. rank 13) being a smaller distance from the ball to lower ranked players (e.g. rank 23).

[0067] The assistant referee for the half of the pitch in which the ball is not currently located is given rank 14.

[0068] Finally, goalkeeper of the attacking team is given rank 25.

[0069] The assigned ranks are updated over time (e.g. each time updated tracking data 206 indicating the latest position of each participant and the ball is received from the tracking apparatus 203) so that, at any given time, the object ID associated with each participant is associated with a rank. The rankings are then used to determine the control data 201 to be transmitted to each of the mobile cameras 107 associated with that object ID.

[0070] An example of a characteristic of the control data 201 associated with each ranked object ID is provided in Fig. 5.

[0071] In this example, the total bandwidth available to the mobile cameras 107 for transmitting video data is such that, at any one time, two mobile cameras are able to transmit video data at a highest quality (e.g. at a highest frame resolution and frame rate, such as 1920 x 1080 pixels at 60 frames per second) and another two mobile cameras are able to transmit video data at a reduced quality (e.g. at a reduced frame resolution and / or frame rate, such as 720 x 480 pixels at 24 frames per second). There is not sufficient bandwidth available for transmission of video data by the remaining mobile cameras and therefore these cameras are controlled to enter a dormant mode in which video data is recorded (e.g. in storage medium 308) but not transmitted. It is noted all cameras may capture video data at the highest quality at all times (whether or not the video data is actually being transmitted at the highest quality).

[0072] In this example, first control data is thus sent to the two highest ranked cameras (associated with respective object IDs A11 and B1) to control them to enter a first “highest quality” mode in which video data is transmitted at the highest quality (and therefore a first, higher, transmission rate). The two highest ranked cameras therefore form a first subset of the cameras. Second control data is sent to the next two highest ranked cameras (associated with respective object IDs A7 and A8) to control them to enter a first “reduced quality” mode in which video data is transmitted at the reduced quality (and therefore a second, lower, transmission rate). These two next highest ranked cameras therefore form a second subset of the cameras. Third control data is sent to the remaining cameras to control them to enter a “dormant” mode in which no video data is transmitted. As the ranking of the cameras changes, the control data then updates the mode of each camera as appropriate to change which cameras transmit video data (and which cameras transmit at the highest quality and which at the reduced quality) and which do not transmit video data.

[0073] In an example, the ranking of each camera may be retained for a minimum predetermined period of time (e.g. 1 , 3 or 5 seconds) before being available for update to prevent hysteresis in camera switching. For example, this helps alleviate multiple, unnecessary, camera ranking switches (and the associated control data overhead) when two players are in similar proximity to the ball and, as the players move around, the player closest to the ball keeps changing.

[0074] In an example, the video data captured by each camera is stored and transmitted as timestamped video data packets. The cameras may use a synchronised clock (e.g. synchronised based on timestamps included in the transmitted control data 102) so the timestamps across different cameras are synchronised. The archive apparatus 212 records the timestamps and quality (e.g. highest quality or reduced quality) of the video data packets received from each camera. At the end of the match (or during a break in play) and / or when more bandwidth becomes available, the highest quality video data from each camera which has been captured but not transmitted (e.g. for times when the camera was in reduced quality or dormant mode) is transmitted to the archive apparatus 212. The archive apparatus thus receives all video data packets not previously received and can ensure, based on the timestamp of each packet from each camera, that no captured video data is missing.

[0075] Fig. 6 shows an example method carried out by each mobile camera 107 (e.g. by processor 301). Although video data I video data packets are discussed in this example, it will be appreciated similar principles may be applied to other types of captured data I data packets (e.g. audio and / or sensor data I data packets).

[0076] At step 601 , video data that can be transmitted to the wireless network as a video data packet is captured.

[0077] At step 602, it is determined whether the mobile camera 107 is in a mode enabling immediate transmission of the video data (that is, transmission of the video data as a video data packet in response to capture of the video data so the video data is transmitted as soon as possible after capture). Such a mode allows, for example, effective live or real time capture and transmission of video data as captured data 202 to the wireless network. The mobile camera 107 is in such a mode if it has received control data 201 causing it to enter the “Highest quality” or “Reduced quality” mode, for example. On the other hand, the mobile camera 107 is not in such a mode if it has received control data 201 causing it to enter the “Dormant” mode.

[0078] If, at step 602, it is determined the mobile camera 107 is in the mode enabling immediate transmission, at step 603, it is determined whether the immediate transmission mode is the highest quality (HQ) mode (as indicated by the control data 201). If so, at step 606, the video data at the highest resolution and frame rate is transmitted in a video data packet as captured data 202. If not, at step 607, it is determined that the immediate transmission mode is the reduced quality (RQ) mode (as indicated by the control data 201). At step 604, the video data at the reduced resolution and / or frame rate is transmitted in a video data packet as the captured data 202. If, at step 602, it is determined the mobile camera 107 is not in the mode enabling immediate transmission, at step 605, the video data is stored (e.g. as a video data packet at the highest resolution and frame rate).

[0079] At step 607, it is determined whether additional bandwidth is available to transmit the stored video data (together with any previously-stored and not-yet-transmitted video data).

[0080] For example, at step 607, it is determined whether control data 202 has been received indicating that stored video data may now be transmitted to the wireless network (e.g. as may occur at the end of the match or during a break in play). Thus, in addition to the control data indicating the transmission mode (e.g. highest quality, reduced quality or dormant mode), it may also indicate when stored video data may be transmitted. This may be referred to as an archive transmission mode. In an example, fourth control data is used to initiate the archive transmission mode. It may also be used to end the archive transmission mode (e.g. when a break in play ends and play resumes).

[0081] Alternatively, or in addition, at step 607, it is determined whether an additional connection (e.g. a Wi-Fi connection) has been established allowing transmission of the stored video data without using the limited bandwidth (e.g. of a cellular network) allocated for immediate transmission of captured data 202. In other words, it is determined whether any additional data path (e.g. Wi-Fi as well as cellular) has become available.

[0082] If additional bandwidth for transmission of the stored video data is available then, at step 608, the stored video data is transmitted and stored by the archive apparatus 212. Otherwise, the method returns to step 601. At step 608, only a portion of video data stored by the mobile camera 107 (e.g. in storage medium 308) may be transmitted (e.g. starting with the stored video data packet with the earliest capture timestamp), depending on the amount of additional bandwidth and how long the additional bandwidth is available (as determined, for instance, by the time period between the archive transmission mode being enabled and disabled and / or the time period over which the additional connection is established (e.g. as determined by how long a player stands within range of a Wi-Fi access point)).

[0083] The present technology thus allows data captured during a sports event to be prioritised to make more effective use of the limited bandwidth available for transmission of that data. It is particularly applicable to data captured by mobile devices such as mobile cameras which must be transmitted wirelessly and allows the technical benefits of remote processing of such data (e.g. for sports broadcasting and / or tracking) to be realised even when there are many data capture and transmission devices. The present technology thus enables more efficient and flexible use of bandwidth-limited communication paths for the transmission of sports data.

[0084] Although the above examples relate to soccer, it will be appreciated the same principles may be applied to any sport in which multiple devices need to collect and transmit data. For example, similar principles may be applied to other sports comprising multiple participants equipped with data capture equipment (e.g. hockey, rugby, racing, etc.). Accordingly, it will be appreciated that the example ranking technique of Fig. 4 is only an example and that, although at least some steps of the same ranking technique may also be applicable to sports such as hockey or rugby (where, for instance, player proximity to the ball is likely to be highly correlated with the relevance of the data currently being captured forthat player), different ranking techniques for different sports may be appropriate (and the control apparatus 204 thus configured accordingly).

[0085] The ranking of devices may also take into account different information to that of the tracking data 206 (in addition to, or instead of, the tracking data 206). For example, a suitable machine learning classification model may be applied by each mobile camera 107 to video, audio and / or sensor data captured by the camera to determine a likely relevance of that data to the current state of the game. For example, captured data may be classified as “low relevance” or “high relevance” and notification data may be transmitted to the wireless network when any device captures data classified as “high relevance”. Device(s) with data classified as “high relevance” may then be ranked higher by the control apparatus 204 than device(s) with data classified only as “low relevance”. The notification data may be a single flag transmitted as part of a regularly transmitted beacon signal of each mobile camera 107, for example, thereby ensuring the portion of the limited bandwidth used for transmission of the notification data is negligible.

[0086] In an example, a stream of video data captured by each mobile camera may be periodically classified as “low relevance” or “high relevance” depending on, for example, the number of participants detected in the video data (e.g. so video data is classified as “high relevance” if there is at least a predetermined threshold number of objects classified as “human” in a predetermined number of consecutive video frames and “low relevance” otherwise), whether or not a ball is detected in the video data (e.g. so video data is classified as “high relevance” if an object is classified as “ball” in a predetermined number of consecutive video frames and “low relevance” otherwise) and / or a facial expression of one or more participants in the video data (e.g. so video data is classified as “high relevance” if a face of a participant in a predetermined number of consecutive video frames is classified as “angry” and “low relevance” otherwise).

[0087] Complementary data such as audio data and / or sensor data may also be used for the video data classification. For instance, a louder crowd noise (e.g. where noise classified as “cloud” exceeds a predetermined loudness threshold in dB), the detection of certain words being uttered (e.g. certain expletives or abusive language) and / or an increasing heart rate while a player is travelling below a predetermined speed (as determined by the tracking data 206) may be indicative that a participant has been involved in foul play and / or an altercation with another participant. In this case, the video stream mobile camera 107 worn by the participant concerned by may classified as “high relevance”.

[0088] Furthermore, in addition to (or instead of) the tracking data 206 derived from images captured by the fixed cameras 102 being used to determine the ranking of participants, other data may be used. For instance, location data (e.g. global navigation satellite system, GNSS, data) and / or acceleration data generated by each mobile device 107 and / or sensor(s) in the ball itself 104 may be used (and, again, periodically transmitted to the wireless network using a beacon signal for use by the control apparatus 204) to determine the location, trajectory and / or speed of each participant and / or the ball. This allows the mobile cameras 107 to be prioritised with less (or no) reliance on the tracking data 206, for example.

[0089] Third party data streams may also be taken into account (e.g. as generated by a third party generation apparatus). For example, instead of or in addition to the tracking data 206, a third party data stream based on the collection of different data associated with the match (e.g. real time player statistics or the like) may be taken into account by the control apparatus 204. For instance, players with in-game statistics indicating higher overall performance over the course of the game (e.g. based on total distance ran, amount of time with the ball, number of passes or the like) may be prioritised above players with in-game statistics indicating lower overall performance over the course of the game.

[0090] Such data may be used with the example ranking method described above in place of proximity to the ball, for example. Thus, for instance, for players deemed most likely to be involved in an attack (e.g. players with rankings 3, 4, 5 and 6), rather than ranking these players based on proximity to the ball, they may be ranked based on performance statistics. Alternatively, they may be ranked based on both proximity and performance statistics (e.g. which each of proximity and performance being weighted accordingly to determine the final rank).

[0091] Thus, in general, data indicative of any suitable characteristic for determining the relevance (and therefore the prioritisation and / or ranking for wireless data transmission) of data captured by each mobile device 107 may be used. This may be referred to, more generally, as characteristic data and may include, for example, one or more of the tracking data 206, machine learning classification data and play statistical data exemplified above.

[0092] Figs. 7A and 7B show example methods.

[0093] Fig. 7A is an example method executed by the processor 301 of data processing apparatus 204.

[0094] At step 701 , during a sports event, characteristic data is received indicative of a characteristic of each of a plurality of wireless data capture devices for capturing sports data during the sports event.

[0095] At step 702, a subset of the wireless data capture devices to transmit the captured sports data over a bandwidth-limited wireless communication path is determined using the characteristic data.

[0096] At step 703, transmission of control data to the subset of the wireless data capture devices is controlled to control the subset of the wireless captured devices to transmit the captured sports data over the bandwidth-limited communication path.

[0097] Fig. 7B is an example method executed by the processor 301 of wireless data capture device 107. At step 704, control data is received for controlling the wireless data capture device to transmit sports data captured during a sports event over a bandwidth-limited wireless communication path.

[0098] At step 705, sports data is captured during the sports event.

[0099] At step 706, the captured sports data is transmitted over the bandwidth-limited wireless communication path.

[0100] Example(s) of the present disclosure are defined by the following numbered clauses:

[0101] 1 . A data processing apparatus comprising circuitry configured to: receive, during a sports event, characteristic data indicative of a characteristic of each of a plurality of wireless data capture devices for capturing sports data during the sports event; determine, using the characteristic data, a subset of the wireless data capture devices to transmit the captured sports data over a bandwidth-limited wireless communication path; and control transmission of control data to the subset of the wireless data capture devices to control the subset of the wireless captured devices to transmit the captured sports data over the bandwidth-limited communication path.

[0102] 2. A data processing apparatus according to clause 1 , wherein the characteristic data comprises position data of each of the plurality of wireless data capture devices.

[0103] 3. A data processing apparatus according to clause 2, wherein the circuitry is configured to receive position data of a ball of the sports event and determine the subset of wireless data captured devices using the position data of the ball.

[0104] 4. A data processing apparatus according to any preceding clause, wherein the sports data comprises one or more of video data, audio data and sensor data.

[0105] 5 A data processing apparatus according to any preceding clause, wherein: the control data is to control the subset of wireless data capture devices to transmit the captured sports data at a first data transmission rate; and the circuitry is configured to: determine, using the characteristic data, a second subset of the wireless data capture devices to transmit the captured sports data over the bandwidth-limited wireless communication path at a second data transmission rate lower than the first data transmission rate; and control transmission of second control data to the second subset of the wireless data captured devices to control the second subset of the wireless captured devices to transmit the captured sports data over the bandwidth-limited communication path at the second data transmission rate.

[0106] 6. A data processing apparatus according to any preceding clause, wherein the circuitry is configured to control transmission of third control data to any remaining wireless data captured device to control the remaining wireless data capture device to locally store captured sports data.

[0107] 7. A data processing apparatus according to clause 6, wherein the circuitry is configured to control transmission of fourth control data to the remaining wireless data capture device to control the remaining wireless data capture device to transmit the locally stored captured sports data.

[0108] 8. A wireless data capture device comprising circuitry configured to: receive control data for controlling the wireless data capture device to transmit sports data captured during a sports event over a bandwidth-limited wireless communication path; capture the sports data during the sports event; and transmit the captured sports data over the bandwidth-limited wireless communication path.

[0109] 9. A wireless data captured device according to clause 8, wherein the sports data comprises one or more of video data, audio data and sensor data.

[0110] 10 A wireless data capture device according to clause 8 or 9, wherein: the control data is to control the wireless data capture device to transmit the captured sports data at a first data transmission rate; and the circuitry is configured to: receive second control data for controlling the wireless data capture device to transmit the captured sports data over the bandwidth-limited wireless communication path at a second data transmission rate lower than the first data transmission rate; capture sports data during the sports event; and transmit the captured sports data over the bandwidth-limited wireless communication path at the second data transmission rate.

[0111] 11. A wireless data capture device according to any one of clauses 8 to 10, wherein the circuitry is configured to: receive third control data for controlling the wireless data capture device to locally store captured sports data; capture sports data during the sports event; and locally store the captured sports data.

[0112] 12. A wireless data capture device according to clause 11 , wherein the circuitry is configured to: receive fourth control data for controlling the wireless data capture device to transmit the locally stored captured sports data; and in response to receiving the fourth control data, transmit the locally stored captured sports data.

[0113] 13. A computer-implemented data processing method comprising: receiving, during a sports event, characteristic data indicative of a characteristic of each of a plurality of wireless data capture devices for capturing sports data during the sports event; determining, using the characteristic data, a subset of the wireless data capture devices to transmit the captured sports data over a bandwidth-limited wireless communication path; and controlling transmission of control data to the subset of the wireless data capture devices to control the subset of the wireless captured devices to transmit the captured sports data over the bandwidth-limited communication path.

[0114] 14. A computer-implemented method of controlling a wireless data capture device, the method comprising controlling the wireless data capture device to: receive control data for controlling the wireless data capture device to transmit the sports data captured during a sports event over a bandwidth-limited wireless communication path; capture the sports data during the sports event; and transmit the captured sports data over the bandwidth-limited wireless communication path.

[0115] 15. A computer-readable storage medium storing a program for controlling a computer to perform a method according to clause 13 or 14.

[0116] Numerous modifications and variations of the present disclosure are possible in light of the above teachings. It is therefore to be understood that, within the scope of the claims, the disclosure may be practiced otherwise than as specifically described herein.

[0117] In so far as embodiments of the disclosure have been described as being implemented, at least in part, by one or more software-controlled information processing apparatuses, it will be appreciated that a machine-readable medium (in particular, a non-transitory machine-readable medium) carrying such software, such as an optical disk, a magnetic disk, semiconductor memory or the like, is also considered to represent an embodiment of the present disclosure. In particular, the present disclosure should be understood to include a non-transitory storage medium comprising code components which cause a computer to perform any of the disclosed method(s).

[0118] It will be appreciated that the above description for clarity has described embodiments with reference to different functional units, circuitry and / or processors. However, it will be apparent that any suitable distribution of functionality between different functional units, circuitry and / or processors may be used without detracting from the embodiments.

[0119] Described embodiments may be implemented in any suitable form including hardware, software, firmware or any combination of these. Described embodiments may optionally be implemented at least partly as computer software running on one or more computer processors (e.g. data processors and / or digital signal processors). The elements and components of any embodiment may be physically, functionally and logically implemented in any suitable way. Indeed, the functionality may be implemented in a single unit, in a plurality of units or as part of other functional units. As such, the disclosed embodiments may be implemented in a single unit or may be physically and functionally distributed between different units, circuitry and / or processors. Although the present disclosure has been described in connection with some embodiments, it is not intended to be limited to these embodiments. Additionally, although a feature may appear to be described in connection with particular embodiments, one skilled in the art would recognize that various features of the described embodiments may be combined in any manner suitable to implement the present disclosure.

Claims

CLAIMS1 . A data processing apparatus comprising circuitry configured to: receive, during a sports event, characteristic data indicative of a characteristic of each of a plurality of wireless data capture devices for capturing sports data during the sports event; determine, using the characteristic data, a subset of the wireless data capture devices to transmit the captured sports data over a bandwidth-limited wireless communication path; and control transmission of control data to the subset of the wireless data capture devices to control the subset of the wireless captured devices to transmit the captured sports data over the bandwidth-limited communication path.

2. A data processing apparatus according to claim 1 , wherein the characteristic data comprises position data of each of the plurality of wireless data capture devices.

3. A data processing apparatus according to claim 2, wherein the circuitry is configured to receive position data of a ball of the sports event and determine the subset of wireless data captured devices using the position data of the ball.

4. A data processing apparatus according to claim 1 , wherein the sports data comprises one or more of video data, audio data and sensor data.5 A data processing apparatus according to claim 1 , wherein: the control data is to control the subset of wireless data capture devices to transmit the captured sports data at a first data transmission rate; and the circuitry is configured to: determine, using the characteristic data, a second subset of the wireless data capture devices to transmit the captured sports data over the bandwidth-limited wireless communication path at a second data transmission rate lower than the first data transmission rate; and control transmission of second control data to the second subset of the wireless data captured devices to control the second subset of the wireless captured devices to transmit the captured sports data over the bandwidth-limited communication path at the second data transmission rate.

6. A data processing apparatus according to claim 1 , wherein the circuitry is configured to control transmission of third control data to any remaining wireless data captured device to control the remaining wireless data capture device to locally store captured sports data.

7. A data processing apparatus according to claim 6, wherein the circuitry is configured to control transmission of fourth control data to the remaining wireless data capture device to control the remaining wireless data capture device to transmit the locally stored captured sports data.

8. A wireless data capture device comprising circuitry configured to:receive control data for controlling the wireless data capture device to transmit sports data captured during a sports event over a bandwidth-limited wireless communication path; capture the sports data during the sports event; and transmit the captured sports data over the bandwidth-limited wireless communication path.

9. A wireless data captured device according to claim 8, wherein the sports data comprises one or more of video data, audio data and sensor data.10 A wireless data capture device according to claim 8, wherein: the control data is to control the wireless data capture device to transmit the captured sports data at a first data transmission rate; and the circuitry is configured to: receive second control data for controlling the wireless data capture device to transmit the captured sports data over the bandwidth-limited wireless communication path at a second data transmission rate lower than the first data transmission rate; capture sports data during the sports event; and transmit the captured sports data over the bandwidth-limited wireless communication path at the second data transmission rate.

11. A wireless data capture device according to claim 8, wherein the circuitry is configured to: receive third control data for controlling the wireless data capture device to locally store captured sports data; capture sports data during the sports event; and locally store the captured sports data.

12. A wireless data capture device according to claim 11 , wherein the circuitry is configured to: receive fourth control data for controlling the wireless data capture device to transmit the locally stored captured sports data; and in response to receiving the fourth control data, transmit the locally stored captured sports data.

13. A computer-implemented data processing method comprising: receiving, during a sports event, characteristic data indicative of a characteristic of each of a plurality of wireless data capture devices for capturing sports data during the sports event; determining, using the characteristic data, a subset of the wireless data capture devices to transmit the captured sports data over a bandwidth-limited wireless communication path; and controlling transmission of control data to the subset of the wireless data capture devices to control the subset of the wireless captured devices to transmit the captured sports data over the bandwidth-limited communication path.

14. A computer-implemented method of controlling a wireless data capture device, the method comprising controlling the wireless data capture device to: receive control data for controlling the wireless data capture device to transmit the sports data captured during a sports event over a bandwidth-limited wireless communication path; capture the sports data during the sports event; and transmit the captured sports data over the bandwidth-limited wireless communication path.

15. A computer-readable storage medium storing a program for controlling a computer to perform a method according to claim 13 or 14.

Images