Method for delivering personalized video content over a network - Patent Application 20070122997

Abstract

Delivering personalized videos and / or videos containing real-time information. A method for delivering video content over a network includes providing video data to a primary data source, associating control data with the video data, broadcasting the video data along with the associated control data from the primary data source over the network to one or more user devices, providing a media player on each user device, the media player operable to create auxiliary data locally on the respective user device in response to reading the control data while the media player is playing the video data, and creating the auxiliary data locally on the respective user device while the media player is playing the video data locally on the respective user device. The control data defines one or more elements of the auxiliary data created by the media player, including elements of the auxiliary data retrieved from the primary data source and one or more secondary data sources.

Description

[Technical Field]

[0001] The present invention relates to a method and system for distributing video content, particularly over a network. [Background technology]

[0002] Over the past few years, the growth of personalized and contextualized video has grown rapidly. Many organizations specialize in creating and distributing personalized videos. These organizations include some of the largest social media companies in the world. Personalized and contextual videos are created to provide users with a personalized video experience by including the user's personal information in the video.

[0003] To meet the massive demand for personalized videos, companies have created systems that allow templated videos to be combined with data, rendered within controlled servers, and then delivered to users. This method of creating personalized videos is difficult to scale and highly inefficient, as each personalized video must be rendered and stored on the company's servers before delivery, resulting in exponentially increasing rendering costs and time as volume increases. For context, creating one video for each user of the world's largest social media site would require a company to create and store 1.6 billion unique video files. The cost, energy, time, and storage required to meet this demand are prohibitive. This makes it extremely difficult, if not impossible, for companies other than the world's largest to enable personalized video creation at any scale. For example, as the number of users increases, the time, cost, and energy required to create personalized videos increases to meet the demand, as seen in FIG. 1A, generally indicated by reference numeral 10A. These problems increase as the volume, scale, and demand for personalized videos increase. Therefore, these problems create a growing need for the ability to create personalized videos at any scale, quickly, and in a much more efficient and energy-saving manner.

[0004] For every video created, all information that needs to be in the video must be ingested into the video rendering server. This may include secure or personal information. For context, creating personalized videos for all 1.6 billion users of the largest social media network requires retrieving the personal data of those 1.6 billion people and storing it on the video rendering server so that personalized videos can be created. Beyond the technical challenges associated with this, there are also potentially significant concerns regarding data privacy and security, particularly in light of the recent media and government focus on data privacy, as evidenced by recent changes in data privacy laws such as the General Data Protection Regulation (GDPR) introduced in the European Union in 2018.

[0005] These recent privacy laws make it increasingly difficult for first-party services that want to deliver videos with personalized information, even with personalized information about the user featured in real time, to do so while at the same time being extremely difficult, if not impossible, to comply with the GPDR, as to accomplish this, the first-party service would need to pull data from one or more different third parties and / or first-party services.

[0006] As described above, current practices for creating personalized videos on a large scale typically involve rendering / creating each personalized video file before delivery. Thus, to create 100 personalized videos, 100 video files would be created and stored on a server, broadcast to user devices, and stored in the physical memory of the user devices. There is a need to provide a means to reduce the time, effort, and cost of delivering personalized video content.

[0007] The problems with current practice are best evidenced when considering real-world applications. For example, consider a situation where a company wants to create location-based videos displaying the names of local pubs that served a particular brand of beer during a rugby or soccer match. To do this, all third-party data, including the locations and pub names, must be collected. From here, a separate, personalized video must be rendered for each pub, which can be a significant undertaking, especially if the locations are major cities like London, New York, or Dublin. For each variation of the base video, a new video, including the third-party data, must be created. Currently available technology requires all videos to be rendered before the user can watch them within the server. This method of video delivery hinders data scalability, time, and relevance. Ultimately, existing methods limit the ability to deliver truly personalized video and / or video containing real-time information.

[0008] It is a desire of the present invention to overcome the above-highlighted shortcomings. Summary of the Invention

[0009] Accordingly, a first aspect of the present invention provides a method for delivering video content over a network, the method comprising: providing video data to a primary data source; associating control data with the video data; broadcasting the video data together with the associated control data from the primary data source over the network to one or more user devices; providing a media player on each user device, the media player operable to create auxiliary data locally on the respective user device in response to reading the control data while the media player is playing the video data; and creating the auxiliary data locally on the respective user device while the media player is playing the video data locally on the respective user device, the control data defining one or more elements of the auxiliary data created by the media player, including elements of the auxiliary data retrieved from the primary data source and one or more secondary data sources such that there is no direct data exchange between the primary data source and the secondary data sources. Advantageously, therefore, the present invention provides a method for creating auxiliary data to be overlaid on top of video data created locally on a user device, wherein the created auxiliary data may be based on information retrieved from either the primary data source and / or the secondary data source without any direct data exchange between the primary and secondary data sources, thus ensuring that information about the user of the user device on which the video is played and the auxiliary data is created locally in real time is kept private, with the provider of personal information from either the primary or secondary data source only knowing the content they provided to the media player.

[0010] A second aspect of the present invention provides a system for delivering video content over a network, the system including a primary data source, one or more user devices, and one or more secondary data sources, wherein the primary data source is configured to associate control data with video data provided to the primary data source, the primary data source is configured to broadcast the video data and associated control data for reception by the one or more user devices, the user devices providing thereon a media player configured to create auxiliary data locally on the respective user device in response to reading the control data as the video is played on the user device, and the control data defines one or more elements of the auxiliary data created by the media player locally on the user device, the element of the auxiliary data being retrieved from the primary data source and the one or more secondary data sources.

[0011] Preferably, the control data includes metadata, such as, for example, a data exchange format or a data storage format.

[0012] Ideally, the control data comprises a machine-readable markup language.

[0013] Preferably, the control data includes instructions defining elements of the auxiliary data, the elements of the auxiliary data including one or more of: a layout of the auxiliary data relative to the video data; a type of auxiliary data to be provided in association with the video data; a first location from which the auxiliary data is retrieved from the primary data source and / or secondary data source; a time at which the auxiliary data is provided in association with the video data; and / or an action to be performed on the auxiliary data when video playback ends.

[0014] Ideally, the action taken on the auxiliary data when video playback ends includes ceasing rendering of the auxiliary data on the media player.

[0015] Preferably, the control data further defines a second location from which the auxiliary data is retrieved from the primary data source and / or the secondary data source if the auxiliary data is not available at the first location.

[0016] Ideally, at different times during the playback of the video data, different types of auxiliary data are provided, typically as dictated by control data.

[0017] Preferably, the different types of auxiliary data include customizable text overlays, or graphics, or sound, or secondary video data, or special effects, or a live feed or display of information, or any combination thereof.

[0018] Ideally, the auxiliary data includes user-specific data, and the user-specific data includes data relating to the user of the user device.

[0019] Preferably, the user-specific data includes one or more of the user's location, the user's age, the user's gender, the user's interests or hobbies, the user's language, the user's search history, the user's web history, and / or any other suitable user-specific information.

[0020] Ideally, the user-specific data is stored on one or more of the secondary data source and / or the primary data source and / or the user device and / or the media player.

[0021] Optionally, a secondary data source from which the media player is configured to retrieve one or more elements of the auxiliary data created by the media player is determined based on one or more elements of the user-specific data.

[0022] Preferably, while the media player is playing the video data locally on the respective user device and before creating the auxiliary data locally on the respective user device, the method further comprises authenticating the media player with the secondary data source to enable the media player to retrieve the auxiliary data, preferably user-specific data, from the secondary data source.

[0023] Ideally, authenticating the media player with the secondary data source includes requiring the user to provide consent for the media player to retrieve one or more pieces of auxiliary data from one or more secondary data sources.

[0024] Preferably, authenticating the media player with the secondary data sources includes confirming that the user has previously provided consent for the media player to retrieve one or more pieces of auxiliary data from one or more of the secondary data sources.

[0025] Ideally, the control data includes instructions that define the actions to be taken if user consent is not given or confirmed.

[0026] Preferably, the control data indicates that the video playback on the user device is not performed on a media player, or that predefined auxiliary data is created during playback of the video on a media player.

[0027] Ideally, the type of predefined auxiliary data that is created is defined in the control data.

[0028] Preferably, the predefined auxiliary data is retrieved from a primary data source.

[0029] Ideally, the primary data source and / or secondary data source include cloud and / or local server architecture and / or API services and / or any data storage format files and / or JSON files and / or computing devices and / or any data storage format or other suitable data source.

[0030] Preferably, the media player is configured to create and synchronize the auxiliary data in real time with the video data while the video data is being played on the user device.

[0031] Ideally, the user device comprises a smartphone, tablet, laptop, or any other suitable computing device. [Brief explanation of the drawings]

[0032] The invention will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0033] [Figure 1A] 1 is a graph showing time, cost, and energy usage as the number of users increases for personalized video generation in the prior art; [Figure 1B] 1 is a graph showing time, cost, and energy usage as the number of users increases for personalized video generation in accordance with the present invention; [Figure 1] 1 is a schematic diagram illustrating a system for delivering video content over a network. [Figure 2] FIG. 1 is a flow diagram illustrating the authentication process of the system. DETAILED DESCRIPTION OF THE INVENTION

[0034] The present teachings will now be described with reference to an exemplary video broadcast system. It will be understood that the exemplary broadcast system is provided to aid in understanding the present teachings and should not be construed as limiting in any way. Furthermore, modules or elements described with reference to any one figure may be interchangeable with those of other figures or other equivalent elements without departing from the spirit of the present disclosure.

[0035] Referring now to the drawings, and particularly to FIG. 1 thereof, there is shown a system for delivering video content over a network, generally designated by reference numeral 1, embodying one aspect of the present invention. The system comprises a primary data source 3, in which control data 5 is associated with raw video data, typically for broadcast to one or more user devices 7. The raw video data and associated control data are typically broadcast over a network, typically including the Internet. The primary data source 3 may include a cloud and / or local server architecture and / or an API service and / or any data storage format file and / or a JSON file and / or a computing device and / or any data storage format or other suitable data source. Preferably, the primary data source 3 includes a server 13 having one or more databases 15 provided thereon or otherwise accessible thereto. The user devices 7 include a media player 11 provided thereon, operable to create auxiliary data locally on each user device 5 in response to reading the control data 5 while playing a video. The control data 5 includes information defining one or more elements of auxiliary data to be created or rendered on the user device 7 while the video is being played on the user device 7. The user device 7 includes a computing device, and more preferably, the user device includes a handheld computing device. To this end, the user device may include a smartphone, a tablet, a laptop, or any other suitable computing device.

[0036] The system further includes one or more secondary data sources 9 with which the user device 7, particularly the media player 11, is operable to communicate in order to retrieve information. To this end, the control data 5 typically defines which information to retrieve from both the primary data source 3 and / or the secondary data source 9. The system has several advantages, the most relevant of which is that from the user's perspective, i.e., from the perspective of the user of the user device 7, there is no direct data exchange or communication between the primary data source 3 and the secondary data source 9, meaning that the primary data source 3 is unaware of the data provided by the secondary data source 9 to the media player 11 of the user device 7, and in turn, the secondary data source 9 is unaware of the data provided by the primary data source 3 to the media player 11 of the user device 7. This is particularly advantageous from a privacy perspective, since data provided by either the primary data source 3 or the secondary data source 9 may be personal data about the user that the user wants to keep private. The secondary data source 9 typically includes a cloud and / or local server architecture and / or an API service and / or any data storage format file and / or a JSON file and / or a computing device and / or any data storage format or other suitable data source.

[0037] A further aspect of the present invention provides a method for delivering video content over a network, the method comprising: providing video data to a primary data source 3; associating control data 5 with the video data; broadcasting video data together with control data 5 associated with one or more user devices 7 from a primary data source 3 over a network; providing on each user device 7, the media player 11 being operable to create auxiliary data locally on each user device 7 in response to reading the control data 5 while the media player 11 is playing the video data; generating auxiliary data locally on each user device 7 while the media player is playing the video data locally on each user device 7; Control data 5 defines one or more elements of auxiliary data to be created by media player 11 , including elements of auxiliary data retrieved from primary data source 3 and one or more secondary data sources 9 .

[0038] 1 is configured to implement a method for distributing video content over a network. Furthermore, features of system 1 described further herein are equally applicable with respect to the method.

[0039] The control data 5 is typically associated with the video data of the primary data source 3. In an alternative embodiment, the system may further include a first device (not shown) operable to communicate with the primary data source 3 via wired and / or wireless transmission means. To this end, the first device is operable to broadcast data for reception by the primary data source 3. The control data may be associated with the video data on the first device, typically by the operator of the first device, and the video data and associated control data may then typically be broadcast simultaneously or separately from the first device to the primary data source 3 for subsequent distribution to the user devices 7. Alternatively, if the video data has already been provided to the primary data source 3, i.e., if the first device contains a copy of the video data already available to the primary data source 3, only the associated control data may be broadcast to the primary data source 3 for subsequent distribution. The first device may include a computing device, and more preferably, the first device may include a handheld computing device. To this end, the first device may include a smartphone, tablet, laptop, or any other suitable computing device. The first device may include an application or the like residing thereon that may be used by a user to add specific auxiliary data to the raw video data.

[0040] As previously mentioned, the control data 5 typically includes information defining one or more elements of auxiliary data to be created and applied in real time to live video data during subsequent playback of the video data on the user device 7 via a media player 11 installed or otherwise accessible thereon. To this end, the control data preferably includes metadata, e.g., a data exchange format and / or data storage format, referred to herein as Video Markup Language (VML) or other machine-readable markup language. The control data 5 includes instructions defining one or more of the layout of the auxiliary data relative to the video data, one or more types of auxiliary data to be provided in association with the video data, the timing at which the auxiliary data is provided relative to the video data, and / or the location from which the auxiliary data is retrieved, such as from the primary data source 3 and / or one or more secondary data sources 9. The auxiliary data may include one or more of customizable text overlays, graphics, sound, secondary video data, special effects, a live feed or display of information, or any combination thereof. It should be understood that live feed is intended to mean substantially live, i.e., real time. While the created auxiliary data is typically layered above or below the video to display the synchronized video, it should be understood that within the video broadcasting system 1, the video remains raw video independent of the generated auxiliary data—in other words, video data without accompanying graphics or special effects. As users watch the video using user devices 7, media player 11 synchronously creates the correct auxiliary data, such as high-quality graphics, text, special effects, etc. This auxiliary data is then overlaid by media player 11 onto the raw video on each user device 11, giving the end user the appearance of a single high-quality video file. To overlay the created auxiliary data on top of the video data, the video data may be defined as multiple distinct display segments, and the auxiliary data may be defined as including one or more display segments of video data.It should be understood that auxiliary data is created or rendered on media player 11 only when video playback is initiated on media player 11, and only in the order prescribed by control 5 data, and typically continues to be created only until the point at which video playback is stopped on media player 11.

[0041] The control data 5 typically acts as a placeholder or template that defines the type of data to be inserted or layered on top of the video data and when this occurs, with different elements of auxiliary data being inserted and removed at specific times. It should be understood that all auxiliary data is processed and created or rendered locally on the user device 7 for insertion in relation to the video data by the media player 11. By way of example, the primary data source 3 may be provided with video data containing an advertising video for a particular product or service, and the control data 5 associated with this video data may define the layout of the auxiliary data, i.e., where and when the created auxiliary data should appear relative to the video data. This may take the form of x-y axis coordinate data and defined time slots for the video data. For example: At time x, insert text overlay {user location} at grid location x=10, y=20 for 15 seconds duration;

[0042] Typically, the control data 5 associated with the video data of the primary data source defines a placeholder for subsequent information retrieved from the secondary data source 9 and / or the primary data source 3. To this end, the control data 5 may further indicate where the placeholder information can be obtained. As indicated in the above statement, the auxiliary data created can be tailored to the user, such as {user location}. The primary data source 3 may also be aware of secondary data sources 9 to which the user can provide relevant personal information. Typically, such secondary data sources 9 may include one or more social media platforms, including one or more of Facebook, Google Plus, Twitter, Instagram, Snapchat, or any other suitable social media platform or API. Thus, the control data 5 defines from which secondary data source 9 the user's location information can be obtained. This can be achieved by providing a general web address or the like, with further information being provided by the user device 7, typically by the media player 11 thereon. Alternatively, the control data 5 may be more specific as to where the information can be obtained based on information about the user of the user device already available to the primary data source 3, as follows: INSERT {USER LOCATION} FROM https: / / facebook.com / usernumber12345 / locationdata.html;

[0043] Following association of the control data with the video data, the video data and control data 5 are then broadcast for reception by user devices 7 on which media player 11 is installed or otherwise accessible. The media player 11 in reading the control data 5 is configured to retrieve user location information from one or more secondary data sources 9 and insert this at time x into grid location x=10, y=20 for a duration of 15 seconds as described above. This might include, for example, where {user location} indicates that the user is in London and where to display a text advertisement for a product or service located in London.

[0044] Additionally, the control data 5 can also define an action to take if the data is not available or accessible at a specified location. For example, if the control data 5 indicates that an auxiliary data element, e.g., user location, is retrieved from a first location in the secondary data source 9 and the media player 11 attempts to retrieve the user location from the first location but an error occurs and the information cannot be accessed at this location, the control data may indicate a second location from which the user's location can be obtained, such as Instagram, or a stock location to provide if the information is not available from the second location. Alternatively, the control data may simply indicate an alternative auxiliary data element, e.g., a graphic, to be inserted in place of the user's location. This example represents a significant advantage to the system of the present invention in that the primary data source 3 is not actually provided with user location data; while the primary data source knows where this data can be retrieved, the location data is not directly accessible from the primary data source 3 itself. The control data 5 acts as a pointer or placeholder to indicate what auxiliary data is provided, when it is provided, and from where.

[0045] As described above, auxiliary data created by user device 7 can be tailored to that particular user. The auxiliary data may include user-specific data, such as location, age, gender, interests, or any other suitable user-specific information. This user-specific data may already be available on either primary data source 3 and / or secondary data source 9, in which case control data 5 may define the locations on primary data source 3 and / or secondary data source 9 from which the user-specific data can be obtained as described above. Furthermore, control data 5 may already include some knowledge of the user-specific data, such that the secondary data source from which media player 11 is configured to retrieve one or more elements of the auxiliary data created by media player 11 is determined based on this user-specific data. For example, the secondary data source associated with a particular user may differ based on the user's current location, age, or other user-specific data. Preferably, personal information about the user is stored in secondary data source 9 that is inaccessible to primary data source 3. However, in order for media player 11 to be able to retrieve this user-specific data, the user may need to provide consent as to which primary data source 3 and / or secondary data source 9 information can be retrieved from and what personal information can be retrieved therefrom; for this purpose, user authentication may be required to enable this personal information to be retrieved from the primary 3 and / or secondary 9 data sources. Thus, the user may be prompted to consent to media player 11 retrieving one or more elements of auxiliary data from the primary data source 3 and / or secondary data source 9. Advantageously, this provides the user with a greater level of control over what data, in particular personal data, can be accessed for the generation of auxiliary data.

[0046] For example, if the user-specific data includes location data, the control data 5 may indicate that this information can be retrieved from one of the secondary data sources 9, such as secondary data source Alpha 17 or secondary data source Beta 18 or secondary data source Gamma 19. Upon accessing media player 11, before playing a video on user device 7, the user may be requested to consent to media player 11 communicating and retrieving personal data, in this case location data, from secondary data source Alpha 17; if the user consents, this location data may be retrieved from secondary data source Alpha 17, and location-dependent auxiliary data may be created by the media player for layering over the video data directly on user device 7. If the user indicates that they do not consent to their location information being retrieved by media player 11, the control data may indicate stock location or alternative auxiliary data to be created and presented.

[0047] 2, a flow diagram illustrating an authentication process 100 is shown in which a user 101 may access a media player 11 and be prompted to watch video data having associated control data 5 on a user device 7. Upon accessing the user device 7, the user 101 starts or otherwise initiates the media player on the user device 103, and upon initialization or when attempting to access a particular video, the user is prompted to provide authentication 105. The user 101 will typically be prompted to provide consent for the media player 11 to retrieve PPI about the user from one or more secondary data sources 9 and / or primary data sources 3; this may be in the form of a single query, or alternatively, the user 101 may be presented with a list of secondary data sources 9 or different types of personal information, such as age, gender, location, hobbies, brand preferences, etc., which the user can individually select to retrieve information from one or more secondary data sources 9 and one or more types of personal information. If the user 101 does not provide consent for the media player 11 to access any secondary data sources 9 and / or any type of personal information, the media player 11 may indicate a failure and, as a result, not play the video requested by the user 101. The user 101 may then be re-prompted to provide consent for the media player 11 to retrieve information from the secondary data sources 9. Additionally or alternatively, if the user 101 is presented with a selection of secondary data sources 9 and / or different types of personal information and does not provide consent for the media player 11 to retrieve the information, when re-prompted for consent, the user 101 may be offered a different selection of secondary data sources 9 and / or different types of personal information for which to provide consent for them to be retrieved.

[0048] Now, if the user 101 consents to the media player 11 retrieving PPI for the user from one or more secondary data sources 9 and / or primary data sources 3, the media player 11 is configured to read the control data 107 and retrieve elements of the auxiliary data to be created from the primary data source 109 and the secondary data source 111. Following receipt thereof, video playback is initiated on the user device 7, and auxiliary data is created 113 locally on the user device 7 in real time. From the foregoing, it should be understood that when elements of the auxiliary data to be created are referred to as being "retrieved" or "fetched" from the primary data source 109 and the secondary data source 111, this auxiliary data is typically in the form of metadata that the media player 11 uses to render and create the auxiliary data locally on the user device 7.

[0049] Additionally, the control data 5 typically indicates what happens to the generated auxiliary data when playback of the video ends; in particular, the control data 5 typically indicates that once the video ends, the generated auxiliary data and / or raw video data stops being rendered or created on the media player 11 of the user device 7 so that it is no longer viewable on the user device 7. This advantageously provides additional privacy for the user, as the generated auxiliary data is not viewable to subsequent users of the user device 7, and auxiliary data created based on the user's personal information is not stored on the user device 7. This is also highly advantageous from a data storage perspective, as the auxiliary data is only created locally during playback of the video data and rendering stops when playback of the video data stops, so local memory on the user device 7 is fairly limited and therefore carefully stored, and the memory utilized for creating the auxiliary data is kept to a minimum, typically only temporary memory on the user device 7, such as cache memory, thus freeing up local data storage for other tasks.

[0050] Advantageously, the disclosed arrangement, particularly the independence of data exchange between the primary data source 3 and the secondary data source 9, ensures that data compliance laws are upheld while providing for the creation of large volumes of personalized videos containing personally identifiable information (PII) and third-party data. Large volumes of videos can be created without individually rendering each one on a server, thereby avoiding the processing or storage of PII from user-authenticated secondary data sources 9. The control data 11 instructs the media player 11 how and where to retrieve data from the primary data source 3 and / or secondary data source 9 and render them with the video locally on the user device 7 in real time. The control data 5 further typically includes an instruction file containing one or more algorithms or other suitable instruction means configured to retrieve or fetch data from the primary data source 3 and / or secondary data source 9 and instruct the media player 11 to seamlessly add the fetched data to the video. The control data 5 can only be read by the media player 11 and cannot be used outside of the media player 11. When a user authenticates access to a secondary data source 9 on device 7, typically including, but not limited to, a third-party API service such as Facebook or LinkedIn, data obtained from the secondary data source 9 is never sent to primary data source 3. Media player 11 is initiated when the user begins watching a video. Data received from different sources, primary data source 3 and / or secondary data source 9, is dynamically rendered as a single video. Dynamic data rendering occurs on user device 7 within media player 11; no PII or data from third-party sources is shared outside of media player 11. Furthermore, content within media player 11 exists only while the user is watching a video. Media player 11 and its content are discarded when the user finishes watching the video on the device and closes the player.

[0051] It should be understood that auxiliary data created in association with a user's personal data is not simply a visual representation of the personal data; this data can further be utilized as a building block for more relevant and targeted auxiliary data. For example, if the personal data includes location data, the auxiliary data created based on this location data can take a variety of forms. First, if the user data indicates that the location is in France, the text overlays may all be displayed in French. Further, if the location data indicates that the location is in Strasbourg, France, the auxiliary data may include graphics and / or text containing advertisements for services and products available in that area, i.e., Strasbourg. Furthermore, if the location data indicates that the user is in France, the auxiliary data may include a live feed of information such as, but not limited to, weather, news, sports, stock market information, and the like. This live feed of information may be in the form of a ticker or other suitable display means overlaid above or below the live video data.

[0052] Advantageously, this not only addresses data privacy concerns, but also demonstrates the versatility of the disclosed invention in that from a commercial standpoint, the system and method are operable to provide users with targeted and relevant information that will be highly advantageous to the primary data sources 3, secondary data sources 9 and the enterprises they represent in practical applications.

[0053] Because the control data 5 typically includes metadata, the actual auxiliary data is not broadcast from either the primary data source 3 or the secondary data source 9; instead, the metadata defines the auxiliary data, which is created locally on the user device 7 by the media player 11. This is particularly advantageous from a data storage and bandwidth perspective, since adding auxiliary data to a video typically requires rendering separate video files containing the auxiliary data into a single file and broadcasting it to the user device as auxiliary data. The auxiliary data may contain significant amounts of graphics, video, and other forms of auxiliary data, making these personalized video files significantly larger in size for broadcast to the user device and consuming significant amounts of bandwidth. In contrast, in the present invention, the raw video data is broadcast along with the associated control data 5, including the metadata, resulting in a much smaller file size and therefore more bandwidth-efficient. Furthermore, because the auxiliary data is represented only by metadata, which serves as a pointer to its local creation on the user device 7, there is no need to store the auxiliary data in the primary data source 3 and / or the secondary data source 9, which is also much more advantageous from a data storage perspective. As previously mentioned, conventional methods of creating personalized digital videos require significant computing power, requiring the generation and storage of each personalized adaptation of each video for each user. Before a user can watch a video, the data required for the personalized elements of the video must be obtained, processed, and used to generate the video. The more users who provide personalized video, the greater the cost, time required, and energy usage to generate and store all the video adaptations.The present invention is far more advantageous in this regard as all processing is done locally on the user device 7 when the video is only desired to be viewed, as opposed to the pre-distribution and storage approach commonly practiced; this advantage is illustrated in Figure 1B of the drawings generally indicated by reference numeral 10B, which shows a graph illustrating how time, cost and energy usage remain substantially fixed as the number of users requiring personalized video increases.

[0054] Traditionally, computer programs are used to generate personalized videos using data relevant to target users. This generation typically occurs on one or more servers, which are costly to set up, run, and maintain. The more videos that need to be generated, the greater the financial burden resulting from data storage on the servers. For a large number of users, if each user has a limited timeframe for generating videos, more servers may be required to split the generation work and perform tasks in parallel, allowing the total workload to be handled within this timeframe. This requires additional cost, time, and energy consumption. The time and energy usage required to generate a video are positively correlated with the file size of the master video file, the amount of data to be processed for personalization, the effects that need to be applied to the video, and the processing power of the computing environment. Each rendered video must be stored in data storage accessible by the server so that users can download or stream the personalized video, which continues to increase cost and energy usage even after the video is generated. Storing and distributing large amounts of video requires the use of data centers, which are expensive to run and procure, and require large amounts of energy.

[0055] Under normal circumstances, servers are constantly running and consuming power. In traditional methods of creating personalized videos, this can consume a large amount of energy. Because the video is produced before the user is able to view it, the video may not be viewable at all, wasting power and financial resources. In an era where energy conservation is more important than ever and businesses are constantly scrutinized for their environmental practices, this traditional method can be problematic. The proposed method and system of the present invention address these deficiencies of current practices by enabling personalized videos to be generated for users in real time, locally on the user's device 7. This means that the video is generated only when the user intends to view it. The video is not physically stored anywhere on the user's device 7; all processing and rendering is performed on the user's device 7, thereby saving financial resources and energy and eliminating the need for prior rendering or processing.

[0056] Advantageously, the methods and systems described herein provide a means by which videos can be constructed on a user device 7 that can take disconnected data from multiple sources and render them into one seamless video. The video can be personalized for every user who watches the video based on all of the user's preferences, such as when and where they watch the video, and data from services that may contain the data. In this invention, data and video are interconnected and rendered in real time on the user's consumption device. This enables the creation of an infinite amount of personalized video, with real-time information potentially embedded in the video. Video consumers have full control and visibility into what first- and third-party data is being used, knowing that no data is shared between connected services. Furthermore, the invention enables the creation of an infinite amount of personalized video without creating and storing a rendered video for each iteration.

[0057] The media player 11 aggregates all the information collected from the primary data sources 3 and secondary data sources 9 and combines it with the control data 5 originally broadcast from the primary data sources 3 to create raw video data that it then renders into a "personalized" video for the user. This process works by taking real-time data and rendering it, at the point where it is consumed, ensuring that the video is updated in real time as the user watches the video as the data changes, while at the same time ensuring that the data is personal to the user as defined by the control data 5.

[0058] All data acquisition and processing occurs on the client-side consuming device. Data is never shared between the primary data source 3 and the secondary data source 9. This enables data compliance, as no additional data is stored or created using aggregated data. In this invention, the first-party service, i.e., the primary data source 3, can provide personalized services based on the data available to the user at the time of consumption, which is known through the control data 5. This control data 5 knows the point in the video where data should be retrieved, but also provides a means to adapt when the media player 11 does not have access to the desired data retrieval location, i.e., a "negative input" scenario. This adjustment within the video can be made on the fly. Adjustments are not limited to video adjustments, audio adjustments, data adjustments, or location at the point of consumption. This is facilitated by the control data 5's ability to function as a placeholder. At any point in time, requested data is interlaced with the video by the control data 5, which defines all the elements of auxiliary data to be retrieved and created.

[0059] In this manner, the media player 11 retrieves data from the primary data source 3 and secondary data source 9 based on the control data 5 and renders the auxiliary data while the video is playing locally on the user device 7, eliminating the creation of excessive data. This process allows the primary data source 3 to provide a personalized and / or real-time video experience without having to process or store information from the secondary data source 9 or store it in the video itself. The personalized data displayed within the video at the time of consumption exists only while the user is "watching" the video on the consuming device 7. Within the user-side media player 11, the data is connected in real time within the media player 11. Because the data never leaves the client consuming device 7, no external data or PII exists outside of the user's device 7. Data processing occurs client-side on the user device 7, not on a server, specifically not on the primary data source 3 or secondary data source 9. Because the video and generated auxiliary data are from disconnected sources, this enables the ability to distribute videos containing personally identifiable information while preventing third-party services, i.e., secondary data sources 9, from viewing or processing data from services that are not their own.

[0060] Due to the real-time nature of the present invention, this allows for both content retrieval and invalidation. This means that the control data 5 can determine what happens when the stream is "completed" and what defines completion, which typically indicates the end of video playback. As previously mentioned, this also means that any generated auxiliary data can be invalidated after the video is completed. For example, this could include when the video has been viewed once, when the auxiliary data and / or video data is deleted from the user device 7 after the end of the video is reached, or alternatively, if the media player 11 is embedded within a web page, whenever the established session between the user and the media player 11 ends, such as when the web page is closed. This advantageously adds an additional layer of security and control over the data that the secondary data source 9 can access.

[0061] It will be understood that what is described herein is an exemplary system for delivering video content, and while the present teachings have been described with reference to exemplary arrangements, it will be understood that the present teachings are not intended to be limited to such arrangements, as modifications can be made without departing from the spirit and scope of the present teachings.

[0062] Although exemplary features of a distributed network system according to the present teachings have been described, it will be understood that such an arrangement should not be construed as limiting the present invention to such features. The methods of the present teachings can be implemented in software, firmware, hardware, or a combination thereof. In one mode, the methods are implemented in software as executable programs and executed by one or more special or general-purpose digital computers, such as personal computers (PC, IBM-compatible, Apple-compatible, or otherwise), personal digital assistants, workstations, minicomputers, or mainframe computers. Steps of the methods may be implemented by a server or computer on which the software modules reside or reside in part. Generally, in terms of hardware architecture, such computers include a processor, memory, and one or more input and / or output (I / O) devices (or peripherals) communicatively coupled via a local interface, as will be well understood by those skilled in the art. The local interface can be, for example, but not limited to, one or more buses or other wired or wireless connections, as known in the art. The local interface may comprise additional elements, such as controllers, buffers (caches), drivers, repeaters, and receivers, to enable communication. Additionally, the local interface may include address, control, and / or data connections to enable appropriate communication between the aforementioned components. The processor may be programmed to perform the functions of the first, second, third, and fourth modules, as described above. The processor is a hardware device for executing software, particularly software stored in a memory.The processor may be a custom-made or commercially available processor, a central processing unit (CPU), a coprocessor among several processors associated with a computer, a semiconductor-based microprocessor (in the form of a microchip or chipset), a microprocessor, or generally any device for executing software instructions.

[0063] The memory is associated with the processor and may include any one or combination of volatile memory elements (e.g., random access memory (RAM such as DRAM, SRAM, SDRAM, etc.)) and non-volatile memory elements (e.g., ROM, hard drive, tape, CD-ROM, etc.). The memory may also incorporate electronic, magnetic, optical, and / or other types of storage media. The memory may have a distributed architecture where various components are located remotely from each other, yet are still accessed by the processor.

[0064] The software in the memory may include one or more individual programs, each of which includes an ordered list of executable instructions for implementing logical functions to implement the functionality of a module. In the examples described above, the software in the memory includes one or more components of a method and is executable by a suitable operating system (O / S).

[0065] The present disclosure may include components provided as a source program, an executable program (object code), a script, or any other entity that includes a series of instructions to be executed. In the case of a source program, the program must be translated via a compiler, assembler, interpreter, etc., which may or may not be contained in memory, in order to operate properly in conjunction with the O / S.

[0066] Furthermore, methodologies implemented in accordance with the teachings may be expressed as (a) an object-oriented programming language having classes of data and methods, or (b) a procedural programming language having routines, subroutines, and / or functions, such as, but not limited to, C, C++, Pascal, Basic, Fortran, Cobol, Perl, Java, and Ada.

[0067] It should be noted that if the method is implemented in software, such software can be stored on any computer-readable medium for use by or in connection with a computer-related system or method. In the context of this teaching, a computer-readable medium is an electronic, magnetic, optical, or other physical device or means that can contain or store a computer program used by or in connection with a computer-related system or method. Such a configuration can be embodied in any computer-readable medium for use by or in connection with an instruction execution system, apparatus, or device, such as a computer-based system, a processor-containing system, or other system that can fetch, process, and execute instructions from the instruction execution system, apparatus, or device. As used herein, a "computer-readable medium" can be any means that can store, convey, propagate, or carry a program for use by or in connection with an instruction execution system, apparatus, or device. Such a computer-readable medium can be, for example, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, device, or propagation medium. Any process description or block in the figures should be understood as representing a module, segment, or portion of code that contains one or more executable instructions for implementing a particular logical function or step in the process, as understood by those skilled in the art.

[0068] It should be emphasized that the above-described embodiments of the present teachings, particularly any "preferred" embodiments, are possible examples and are set forth merely for a clear understanding of the principles. Many variations and modifications can be made to the above-described embodiments without substantially departing from the spirit and principles of the present disclosure. All such modifications are intended to be included within the scope of this disclosure and the present invention and protected by the following claims.

[0069] The invention is not limited to the embodiments described herein, but can be amended or modified without departing from the scope of the invention.

Claims

1. 1. A method for distributing video content over a network, said method comprising: providing video data to a primary data source; associating control data with said video data; broadcasting the video data along with associated control data from the primary data source to one or more user devices over the network; providing a media player on each user device, the media player operative to create auxiliary data locally on the respective user device in response to reading the control data while the media player is playing the video data, the created auxiliary data being overlaid on top of the video data; creating the auxiliary data locally on each user device while the media player is playing the video data locally on the respective user device; the control data defines one or more elements of the auxiliary data to be created by the media player, including elements of the auxiliary data retrieved from the primary data source and one or more secondary data sources; the auxiliary data includes user-specific data, and the user-specific data includes data relating to a user of the user device; the secondary data source from which the media player is configured to retrieve the one or more elements of the auxiliary data created by the media player is determined based on one or more elements of the user-specific data; The method, wherein the auxiliary data retrieved from the secondary data source is maintained only on the user device, and the auxiliary data is automatically deleted when the media player finishes playing the video data.

2. The method of claim 1 , wherein the control data comprises metadata.

3. The method of claim 2 , wherein the control data includes a data exchange format and / or a data storage format.

4. The control data includes instructions defining the elements of the auxiliary data, the elements of the auxiliary data comprising: the layout of the auxiliary data relative to the video data; one or more types of auxiliary data provided in association with said video data; at least a first location where the auxiliary data is retrieved from the primary data source and / or secondary data source; the time at which the auxiliary data is provided in relation to the video data; and / or and an action performed on the auxiliary data when video playback ends.

5. The method of claim 4 , wherein the action performed on the auxiliary data when the video playback ends includes ceasing to create the auxiliary data locally on the media player.

6. The method of claim 4 , wherein the control data further defines a second location from which the auxiliary data is retrieved from the primary and / or secondary data sources if the auxiliary data is not available at the first location.

7. The method of claim 4 , wherein the one or more types of auxiliary data are provided at different times during playback of the video data.

8. the one or more types of auxiliary data, Customizable text overlays, graphic, sound, secondary video data, special effects, and / or The method of claim 1 , comprising one or more of: a live feed or display of information.

9. 10. The method of claim 1, wherein the user-specific data includes one or more of a user's location, a user's age, a user's gender, a user's interests or hobbies, a user's language, a user's search history, a user's web history, and / or any other suitable user-specific information.

10. The method of claim 1 , wherein the user-specific data is stored on one or more of the secondary data source and / or the primary data source, and / or a user device and / or a media player.

11. 10. The method of claim 1, wherein while the media player is playing the video data locally on the respective user device, and prior to creating the auxiliary data locally on the respective user device, the method further comprises authenticating the media player with the secondary data source so that the media player can retrieve the auxiliary data from the secondary data source.

12. 12. The method of claim 11 , wherein authenticating the media player with the secondary data sources comprises requesting the user to provide consent for the media player to retrieve one or more elements of the auxiliary data from one or more of the secondary data sources.

13. 12. The method of claim 11, wherein authenticating the media player with the secondary data sources includes verifying that the user has previously provided consent for the media player to retrieve one or more pieces of the auxiliary data from one or more of the secondary data sources.

14. The method of claim 12 , wherein the control data includes instructions defining actions to be taken if the user's consent is not given or confirmed.

15. 15. The method of claim 14, wherein the control data indicates that the video playback on the user device is not performed on the media player, or that predefined auxiliary data is created during playback of the video on the media player.

16. The method of claim 15 , wherein the control data defines the type of predefined auxiliary data to be created.

17. The method of claim 15 , wherein the predefined auxiliary data is retrieved from the primary data source.

18. 10. The method of claim 1, wherein the primary and / or secondary data sources comprise cloud and / or local server architectures, and / or API services, and / or any data storage format files and / or JSON files, and / or computing devices and / or any data storage format, or other suitable data sources.

19. 2. The method of claim 1, wherein the media player is configured to create the auxiliary data in real time and synchronize it with the video data while the video data is being played on the user device.

20. The method of claim 1 , wherein the user device comprises a smartphone, tablet, laptop, or any other suitable computing device.

21. 1. A system for delivering video content over a network, said system comprising: Primary data sources, one or more user devices; one or more secondary data sources; the primary data source is configured to associate control data with video data provided to the primary data source; the primary data source is configured to broadcast the video data and associated control data for reception by the one or more user devices; the user devices include media players provided with the user devices, the media players configured to create auxiliary data locally on the respective user devices in response to reading the control data when the video is played on the user devices, the created auxiliary data being overlaid on the video data; the control data defines one or more elements of the auxiliary data to be created by the media player locally on the user device, including elements of the auxiliary data retrieved from the primary data source and the one or more secondary data sources; the auxiliary data includes user-specific data, and the user-specific data includes data relating to a user of the user device; the secondary data source from which the media player is configured to retrieve the one or more elements of the auxiliary data created by the media player is determined based on one or more elements of the user-specific data; The system, wherein the auxiliary data retrieved from the secondary data source is maintained only on the user device, and the auxiliary data is automatically deleted when the media player finishes playing the video data.

Images