Method and system for accessing a digital 3D model of a scene

Abstract

Described herein is a method and a system for providing access for a user to an interactive digital 3D model of a scene or event relating to a temporally changing digital content (4000), the method comprising the steps of: providing the temporally changing digital content for display on a first device (4001), the digital content including one or more scenes or events; generating an indicator at times corresponding to the one or more scenes or events, the indicator being associated with a link (4002); allowing activation of the link on a second device (4003); in response to the activation of the link, rendering on a display of the second device, an interactive 3D model associated with the scene or event, wherein the interactive 3D model includes one or more selectable objects (4004); and allowing the user on the second user device to select one or more of the selectable objects to direct the second device to a data portal to interact with the selected object(s) (4005).

Description

METHOD AND SYSTEM FOR ACCESSING A DIGITAL 3D MODEL OF A SCENEFIELD OF THE INVENTION

[0001] The present application relates to a method and system for accessing digital content in an interactive manner as it is displayed on a device via a user’s personal device.

[0002] Embodiments of the present invention are particularly adapted for use in accessing a digital 3D model with handheld mobile devices such as a smartphone. However, it will be appreciated that the invention is applicable in broader contexts and other applications.BACKGROUND

[0003] Presently, there is no satisfactory means of providing a user with the ability to interact with temporally changing content such as a displayed video or scene, or electronic advertising content through a secondary device such as their mobile device. Current systems require a user to either look up a webpage or copy a weblink into a browser on their personal computer or smart phone. QR codes allow a person to access online content from an image, but this content is typically a static website and not dynamic in nature.

[0004] Such barriers to entry often result in a user not accessing content for lack of convenience. Therefore, a means of providing easier access to content relevant to a user is desirable.

[0005] Any discussion of the background art throughout the specification should in no way be considered as an admission that such art is widely known or forms part of common general knowledge in the field.SUMMARY OF THE INVENTION

[0006] In accordance with a first aspect of the present invention, there is provided a method for providing access for a user to an interactive digital 3D model of a scene or event relating to a temporally changing digital content, the method comprising the steps of: a) providing the temporally changing digital content for display on a first device, the digital content including one or more scenes or events; b) generating an indicator at times corresponding to the one or more scenes or events, the indicator being associated with a link; c) allowing activation of the link on a second device;d) in response to the activation of the link, rendering on a display of the second device, an interactive 3D model associated with the scene or event, wherein the interactive 3D model includes one or more selectable objects; and e) allowing the user on the second user device to select one or more of the selectable objects to direct the second device to a data portal to interact with the selected object(s).

[0007] In one embodiment, the temporally changing digital content includes a digital billboard.

[0008] In one embodiment, the temporally changing digital content includes video content.

[0009] In one embodiment, the video content includes a television show.

[0010] In one embodiment, the video content includes on-demand video content.

[0011] In one embodiment, the video content includes a live broadcast video event.

[0012] In one embodiment, the video content includes a live broadcast video event.

[0013] In one embodiment, the video content includes online video.

[0014] In one embodiment, the video content includes social media video.

[0015] In one embodiment, the data portal provides updated content of the link independent of the one or more scenes or events including factors such as when the content is viewed or a demographic of the user.

[0016] In one embodiment, the data portal provides updated content of the link based on other factors such as a season.

[0017] In one embodiment, wherein the data portal includes a webpage containing a list of the selected objects.

[0018] In one embodiment, the webpage provides functionality to purchase the selected objects.

[0019] In one embodiment, the webpage is a third party webpage hosted by an owner of the selected objects.

[0020] In one embodiment, the interaction with the selected object(s) includes a purchase.

[0021] In one embodiment, the first and the second devices are the same device.

[0022] In one embodiment, the activation of the link includes capturing a QR code, HTML link, or a soft button on a touchscreen.

[0023] In one embodiment, the one or more selectable objects are indicative of products that can be purchased.

[0024] In one embodiment, the temporally changing digital content is pre-recorded and wherein the indicator is generated by inserting a trigger at timecodes of the pre-recorded content corresponding to a scene or event.

[0025] In one embodiment, the temporally changing digital content is broadcast live and wherein the indicator is inserted into the temporally changing digital content in real-time or near real-time through a push operation by an operator.

[0026] In one embodiment, upon activating the link, information about the user is acquired.

[0027] In one embodiment, the indicator is displayed on the first device.

[0028] In one embodiment, the indicator is displayed on the second device.

[0029] In one embodiment, the interactive 3D model is generated from a digital scan of a real environment associated with the scene or event using a 3D scanner.

[0030] In one embodiment, the interactive 3D model is generated from a plurality of images of a real environment and fed to a generative machine learning model.

[0031] In one embodiment, at least a subset of the one or more selectable objects are generated from a digital scan of a real object using a 3D scanner.

[0032] In one embodiment, at least a subset of the one or more selectable objects are generated from a plurality of images of a real object and fed to a generative machine learning model.

[0033] In one embodiment, the interactive 3D model is configurable.

[0034] In accordance with a second aspect of the present invention, there is provided a system for providing access for a user to an interactive digital 3D model of a scene or event relating to a temporally changing digital content, the system comprising: a) a first device, adapted to display the temporally changing digital content, the digital content having a narrative including one or more scenes or events;b) at least one processor adapted to: generate an indicator at times corresponding to a display of at least one of the scenes or events on the first device, the indicator being associated with a link; c) a second user device, adapted to receive the link and allowing the user to activate the link; wherein, in response to the activation of the link, the second user device is configured to render on a display of the second user device, an interactive 3D model of the scene or event, wherein the interactive 3D model includes one or more selectable objects and allow the user on the second user device to select one or more of the selectable objects to direct the second device to a data portal to interact with the selected object(s).

[0035] In one embodiment, the first device includes a digital billboard.

[0036] In one embodiment, the temporally changing digital content includes video content.

[0037] In one embodiment, the video content includes a television show.

[0038] In one embodiment, the video content includes on-demand video content.

[0039] In one embodiment, the video content includes a live broadcast video event.

[0040] In one embodiment, the video content includes online video.

[0041] In one embodiment, the video content includes social media video.

[0042] In one embodiment, the data portal provides an updated content of the link dependent on the times corresponding to the display of the at least one of the scenes.

[0043] In one embodiment, the data portal updates the content of the link based on other factors such as a season.

[0044] In one embodiment, the data portal includes a webpage containing a list of the selected objects.

[0045] In one embodiment, the webpage provides functionality to purchase the selected objects.

[0046] In one embodiment, the webpage is a third party webpage hosted by an owner of the selected objects.

[0047] In one embodiment, the interaction with the selected object(s) includes a purchase.

[0048] In one embodiment, the first and the second devices are the same device.

[0049] In one embodiment, the activation of the link includes capturing a QR code, HTML link, or a soft button on a touchscreen.

[0050] In one embodiment, the one or more selectable objects are indicative of products that can be purchased.

[0051] In one embodiment, the temporally changing digital content is pre-recorded and wherein the indicator is generated by inserting a trigger at timecodes of the pre-recorded content corresponding to a scene or event.

[0052] In one embodiment, the indicator is inserted into the temporally changing digital content in real-time or near real-time through a push operation by an operator.

[0053] In one embodiment, upon activating the link, acquiring information about the user.

[0054] In one embodiment, the indicator is displayed on the first device (e.g. TV).

[0055] In one embodiment, the indicator is displayed on the second device (e.g. smartphone).

[0056] In one embodiment, wherein the interactive 3D model is configurable.

[0057] In one embodiment, the interactive digital 3D model is comprised of one or more configurable modules defining the one or more selectable objects within the interactive digital 3D model.BRIEF DESCRIPTION OF THE FIGURES

[0058] Example embodiments of the disclosure will now be described, by way of example only, with reference to the accompanying drawings in which:Figure 1 shows a schematic diagram of an embodiment of the invention;Figure 2 shows a further schematic diagram of an embodiment of the invention;Figure 3 shows yet a further schematic diagram of an embodiment of the invention;Figure 4 shows a flow chart illustrating the primary steps in a method for providing access for a user to an interactive digital 3D model of a scene or event relating to a temporally changing digital content;Figure 5 shows a flow chart in accordance with an embodiment of the invention; andFigure 6 shows a sequence chart in accordance with an embodiment of the invention.DESCRIPTION OF THE INVENTIONOverview

[0059] Described herein is a system and method for providing access for a user, such as a consumer to an interactive digital 3D model of a scene or event relating to a temporally changing digital content. Embodiments of the invention described herein relate primarily to a scene / event in the form of a digital program such as a television series with the associated temporally changing digital content being the digital audio and video signal recorded for that program and displayable on electronic devices. However, other forms of digital content may include websites, social media posts, e-books, online articles, podcasts and digital advertisements. The method and system are accessed by the user from conventional household devices such as televisions and mobile devices (e.g. smartphones or tablet computers) having displays and user interfaces. However, it will be appreciated that various alternative embodiments are possible, as will be outlined below.

[0060] An aspect of the invention is exemplified schematically in Figures 1 to 3. In Figure 1 , a schematic is provided to exemplify a system 1000 in accordance with an embodiment of the invention. The system 1000 will be described in conjunction with method 4000, which may be implemented on the system. The system 1000 provides access for a user 106 to an interactive digital 3D model 108 of a scene or event relating to a temporally changing digital content. The temporally changing digital content may include video or digital images that change with time as will be discussed below. The temporally changing digital content may also include digital audio data as well as various metadata. The interactive digital 3D model may be generated utilising image processing methods such as generative Al and will be discussed in more detail below.

[0061] An example of a scenario in which the temporally changing digital content occurs at a variable rate may include a digital billboard. In this scenario, the digital content may update based on the nature of the content being displayed on the billboard or in accordance with an advertiser’s marketing campaign.

[0062] For the purposes of this disclosure, the temporally changing digital content will be in the context of video and optionally audio also, such as a program accessible through a video streaming service or a terrestrial television broadcast. The scenes or events represent specificscenes or events that occur within the narrative of the program. By way of example, a kitchen cooking reality program may include one or more scenes associated with a particular kitchen having a set of kitchen products, appliances and / or ingredients laid out therein. Subsequent scenes of that program may include different kitchens and / or different arrangements of kitchen products, appliances and / or ingredients. A different event may represent a different cooking scenario set in the same kitchen as a previous scenario.

[0063] The model will include objects such as the kitchen products, appliances and / or ingredients. An interactive digital 3D model 108 of that kitchen can be generated as described in more detail below. However, it will be appreciated that the present invention may be used in other contexts such as electronic billboards, online content such as social media content, YouTube videos and the like. The scene or event represents a subset of the overall temporally changing digital content (in the example case, a kitchen cooking reality program), which may include a plurality of scenes or events.

[0064] Referring to Figures 1 and 4, at step 4001 , the system 1000 includes the provision of initially providing the temporally changing digital content for display on a first device 100, such as a television set, with the digital content including one or more scenes or events. This temporally changing digital content may be provided by way of a digital stream over the internet, on-demand video content, a subscription video service or via a free to air television broadcast as a few examples. However, it will be appreciated that the temporally changing digital content may include non-video sources such as temporally changing images in the form of a digital advertising, such as that displayed on an electronic billboard. The temporally changing digital content may also include audio data and metadata associated with the video content. The metadata may include timecodes indicating a temporal order of images or other data in the temporally changing digital content. Nonetheless, the temporally changing digital content, when in the form of video, may include a number of digital video formats including MPEG-2, MPEG- 4, 3GPP or FLV as some examples.

[0065] Central to all embodiments is the notion that the digital content is temporally changing such that scenes or events within the content come and go. In some embodiments, the scenes or events may be identified by timecode data or one or more timestamps that are inserted into the data stream.

[0066] The first device 100 may include a visual display device as was discussed previously, such as a user’s 106 television set, or a user’s 106 mobile device, such as a mobile / cell phone,in the form of a smart phone, a laptop or desktop computer or a tablet computer as some examples.

[0067] At step 4002, in order to allow the user 106 to interact with objects within the scene, an indicator 104 may be generated and presented to the user 106 on the first device 100 at times corresponding to one or more of the scenes or events. Where the temporally changing digital content includes pre-recorded media, the indicator may be generated by inserting a trigger at timecodes of the pre-recorded content corresponding to a scene or event and embedding the indicator into frames of the content during those times. More preferably, an indicator is generated to be presented to the user 106 during a period of time defined by a “starttime" and an “end-time”. The period between the start-time and end-time corresponds to a display of one of the scenes / events and is when the indicator is presented to the user 106. This start-time and end-time may be set by a creator of the content or by a third party after creation of the content.

[0068] In these scenarios, the one or more indicators may be inserted or embedded into the temporally changing digital content as metadata or other associated data (e.g. a predetermined packet of data) such that it is broadcast, streamed or transmitted together with the content itself.

[0069] Where the temporally changing digital content includes a live broadcast of content, the one or more indicators may be inserted into the temporally changing digital content in realtime or near real-time through a push operation by an operator. The operator may include a content broadcaster or a third party. The push operation may be performed from the same system or server as that hosting the content or may be pushed from a separate system or server to the content-hosted system / server to augment the content with the indicator(s). In this scenario, the indicator is embedded into the temporally changing digital content for the duration of the push operation.

[0070] In the embodiment shown in Figure 1 , the indicator 104 takes the form of a visual indicator 104 which is presented on the screen of the first device 100 at a time corresponding to the one or more scenes or events. A common indicator may be used for multiple scenes / events or different indicators may be used for different scenes / events. As mentioned above, the indicators may be inserted either at post production, or in the case of a live television broadcast, the indicators may be inserted in near real time or real time as the case may be.

[0071] The indicator 104 may be overlayed on a screen of the first device 100 at a predetermined location such as in a bottom corner of the screen. This may occur by modifying the image / video data corresponding to that area of the content or by augmenting, embedding or overlaying the image / video content to include the indicator using known video or image editing methods. If the indicator includes audio data, the indicator may generate an audio signal for output through an audio output of first device 100.

[0072] In some embodiments, the indicator 104 takes the form of a scannable code such as a QR code. In the context of a QR code, the generation of the QR code may involve encoding data into a two-dimensional scannable code which may be achieved using a number of different software solutions for generating QR codes such as “qr code” libraries available in Python or JavaScript as some examples. In other embodiments, the indicator 104 includes a selectable button, such as a “soft button” presented to the screen and associated with a weblink for the user 106 to activate. These latter embodiments are possible where the first device 100 includes an interactive display such as a smartphone or tablet computer. Here the weblink indicator is displayed on the screen of the first device 100 and is a selectable link that can be activated by the user 106 by touching the weblink. In other embodiments, the indicator may take other forms such as a non-visual indicator in the form of an audio signal as one example. It is envisaged, in some embodiments, that the user may activate the weblink by providing an audible instruction to activate the weblink.

[0073] In some embodiments, step 4002 includes the generation of one or more time stamps which are associated with particular events within the video stream. The time stamps may be pre-generated when the digital content is generated or may be added at a later time. In the context of a pre-recorded program, the timestamps may be generated at the time of production or shortly thereafter. In the context of a live streaming program, the time stamps may be inserted into the digital data in real or near-real time by a number of methods that may include a push operation by an operator or a push operation by a server that is being called by an agreed list of scenes / events based on the timecode. The time stamps are preferably in the form of a sequence of characters or encoded information identifying when the event or scene occurred in the program / content. The time stamps may be associated with a timecode of the digital content. The time stamps may be designated by an absolute measure of time (if referenced to a common reference such as a clock signal) or may include a relative measure of time such as from a starttime of the program / content. The time stamps may include sufficient significant figures so as tobe accurate to a fraction of a second. In some embodiments, the time stamps conform with one or more standards such as ISO 8601 .

[0074] For instance, where the temporally changing digital content includes a cooking segment being broadcast on free to air television, the user 106 may wish to investigate a recipe or ingredients being used in the cooking segment as some examples. In this case, time stamps may be associated with key segments of the video stream, allowing the user 106 to interact with a particular scene at a particular time in the video stream. Time stamps may be generated corresponding to a start time and an end time of a scene / event.

[0075] At step 4003, the generation of the indicator 104 then provides an opportunity for the user 106 to activate a link associated with the indicator 104. Where the indicator includes a QR code, the user 106 is able to scan the QR code on a second device 102, which may include a personal device such as a smart phone or a tablet computer. Alternatively, the user 106 may choose not to scan the indicator if the content associated with the scene is not of interest to the user 106 at that particular time stamp.

[0076] In some embodiments, the first device 100 and second device 102 are one and the same device. For example, where the user 106 is viewing the digital content (e.g. social media content) on a device with an interactive interface, such as a smartphone, the indicator 104 is generated on the device with the content being viewed.

[0077] The indicator 104, which, in the embodiment shown takes the form of a QR code 104, then provides a link which is then activated on the user’s 106 second device 102 (e.g. smart phone or table computer). The link may include a HTML or hyperlink that is associated with a visible or non-visible URL or address available over the internet or another network. The link then is activated on the second device 102 (or first device 100 if the user 106 is viewing the content on a smart phone) by the user 106.

[0078] At step 4004, upon activation of the link associated with the indicator 104, the first device is directed to the URL or address associated with the link via a web browser or other executable application. The URL or address is hosted by a server that hosts an interactive 3D model of the scene / event. The interactive 3D model 108 associated with the scene / event is rendered on a display of the second device 102 (or first device 100 where the user 106 is viewing the content on a smart phone or tablet computer). The interactive 3D model 108 is associated with the scene or event which was displayed at the time at which the indicator 104was presented to the user 106. Therefore, the contents of each link will typically be unique for the scene / event associated with a given indicator. The interactive 3D model 108 is preferably hosted on a server that may accessed by the second device 102 over the internet or another network.

[0079] For example, if the indicator 104 was presented at a time where the temporally changing digital content included a particular scene, such as a living room, the interactive 3D model 108 provided in the link may emulate the living room that was presented to the user 106 at that time. The interactive 3D model 108 then allows the user to interact with the scene through the second device 102 in response to an indicator 104 being presented on the first device. The generation of the interactive 3D model 108 will be described in more detail below.

[0080] The interactive 3D model 108 includes one or more selectable objects that preferably correspond to objects that were displayed in the scene / event of the digital content. Furthermore, the interactive 3D model 108 preferably takes the form of a 3D rendering of the scene / event of the digital content. By way of example, where the temporally changing digital content corresponds to a kitchen cooking program, the interactive 3D model may represent a 3D model of the kitchen being displayed at that time. The one or more selectable objects may correspond to cooking items, utensils, appliances or ingredients used in the scene / event.

[0081] As described below, various methods are possible for generating the one or more selectable objects and the interactive 3D model 108 itself. In order to improve the efficiency in rendering the interactive 3D model 108, (especially on slower devices including devices with inefficient computation abilities) certain techniques may be used including decimating the interactive 3D model 108. Decimating the interactive 3D model involves reducing the number of polygons or vertices comprising the interactive 3D model 108 while preserving the overall shape and appearance of the model. The interactive 3D model 108 is then rebuilt utilizing less polygons or vertices, thus load times for the interactive 3D model 108 may be reduced from ten or more seconds on slower devices to a fraction of that amount. Other methods employed to reduce the load time may include, but are not limited to; lowering the resolution of the images comprising the 3D models, removing post processing and refactoring the product and item representations in the interactive 3D model 108.

[0082] Once the interactive 3D model 108 has been rendered on the display of the second device 102, the user can manipulate the display / device to interact with the objects within themodel. This manipulation includes moving a current view or position around the 3D model and / or zooming in or out to view different areas of the rendered scene.

[0083] In some embodiments, the interactive 3D model 108 is comprised of one or more configurable modules which define the one or more selectable objects within the interactive 3D model 108. The one or more configurable modules allow for user editing of aspects of the interactive 3D model 108. The configurable modules, in the form of software modules, may include a set of functions which may be used throughout the interactive 3D model 108. The set of functions may also be stored in a library for use on different 3D models.

[0084] In some embodiments, the act of editing the aspects or properties of the interactive 3D model 108, by the operator of, for example, an online store, may create a new version of that online store. The operator of the online store can also generate a new indicator in the form of a QR code representing that new online store that can be placed onto the temporally changing digital content. This results in keeping the new online store synchronised with the temporally changing digital content, so that the products shown in the temporally changing digital content continue to match the products provided in the online store via the interactive 3D model 108.

[0085] A variety of different software modules may be utilised to edit aesthetic aspects of the interactive 3D model 108 such as rendering modules, texture modules, shader modules and user interface (III) modules as some examples. As previously alluded to, the functionality of the interactive 3D model may also be adjusted for insofar as the III is concerned. For example, an operator of an online store may wish to adjust the properties of items presented in the online store represented by the interactive 3D model 108 or may wish to edit the appearance of certain items or the manner in which a potential customer may interact with the interactive 3D model 108. These properties may include specific items presented for sale within the interactive 3D model 108, such as a particular line of shoes or clothing as some examples, or the means of purchasing those items. In this regard, each of the objects within the interactive 3D model 108 may be selectable for editing with user selectable boundaries being defined within the scene of the interactive 3D model 108. This may be achieved by selecting on the item to be edited, where a menu may be presented upon selection (as will be discussed further below) for editing parameters within the configurable modules to alter the properties of the objects within the interactive 3D model 108. In one example, as previously discussed, selecting a given object within the 3D model 108 may invoke a menu allowing for the customization of the given object or the adjustment of items presented for purchase within that object.

[0086] The objects within the interactive 3D model 108 may be selected by way of a device such as a mouse (in the context of a computer), or remote control when in the context of a television display, or in the context of a handheld device by way of interaction with a touch screen display.

[0087] Finally, at step 4005, the user 106 is able to select one or more of the selectable objects to direct the second device 102 to a data portal to interact with the selected object(s). On a touchscreen or computer interface the selectability may be implemented by defining the objects as ‘soft buttons’ that are defined to have corresponding bounding boxes or borders corresponding approximately to the visual shape of the object. If the position, shape or size of the object changes over time, the corresponding position, shape or size of the soft buttons may change also.

[0088] The object selection process may be by way of a hidden HTML or hyperlink associated with a corresponding URL or address accessed by the internet or another network. By way of example, where the selectable objects represent products that can be purchased, this data portal may include a website associated with and hosted by a merchant selling the products. Where the selectable objects represent educational items (e.g. animals in a documentary program), the data portal may include a website providing educational information on the items.

[0089] Thus, the selectable objects are associated with one or more weblinks (e.g. HTML links) to direct the second device 102 to the data portal. In some embodiments, the data portal may include more than one website or links to multiple websites and may also include a checkout page for purchasing multiple items from different merchants.

[0090] In some embodiments, the user is able to interact with the selectable objects in other ways prior to being directed to the data portal. For example, a user may be able to select the objects to have the interface present more information on the objects, increase their size or rotate them.

[0091] Turning to Figure 2, an embodiment of the invention is shown in which a user 106 is provided with a headset 200 (such as a VisionPro headset) in which the user 106 can interact with a 3D model. In this embodiment, the headset 200 represents both the first device 100 and second device 102. The user 106 is able to select and activate the link associated with the indicator using a physical gesture rather than scanning a code as in previous embodiments.Utilising the headset 200, the user then views the interactive 3D model 108 through the headset 200 and is able to select the selectable objects and navigate the data portal.

[0092] In Figure 3, a further embodiment of the invention is provided in which a video sharing platform (such as YouTube or TikTok) is viewed by a user 106 on the user’s personal device 102. In this embodiment, the first device and the second device are the same device 102 and the user 106 is presented with an indicator 104 on the user’s device 102. The user 106 can then choose to respond to the indicator 104, by either tapping on the indicator or providing a gesture to the device. Based on the interaction with the indicator 104, the user 106 is then able to access the interactive 3D model by way of a link associated with the indicator 104. The link being dependent on the event / scene (and / or optionally an embedded timecode) defining a time at which the indicator 104 was first presented to the user 106. In Figure 3, the labels A, B and C represent different periods of time in which the user is presented with the indicator 104. Different 3D models may be presented to the user which are dependent on the time at which the user interacts with the indicator 104 as it is presented to the user 106.

[0093] Referring now to Figure 5, there is illustrated a flow chart exemplifying a method of providing a 3D model to a user 106 in accordance with the present invention. The embodiment of Figure 5 is in the context of a television show which is presented to a user 400. During the broadcast of the television show, an indicator 104, in the form of a QR code, is presented to the user 106. Data related to the indicator is provided based on a timecode which may be predetermined or provided in real time. Data related to the presentation of the time code include data related to user experience, placement, and the television show time code placement.

[0094] If the user 106 chooses to scan the QR code 402, data relating to the user’s device, (i.e. the second device 102), the user’s IP address, and any previous interactions are sent upon which the user is presented to the interactive 3D model 404, providing a scene related to the temporally changing digital content at the time that the indicator was presented to the user 106, as defined by a time stamp. Once the user has entered the 3D interactive model 406, the user can then browse the 3D model providing items which the user can either investigate for additional information, or the ability to purchase items may be included.

[0095] It is noted that the experiences in the 3D model can alter with time during the progression of the television show 408. If the user wants to purchase a product displayed in the 3D model, the user can then select by way of tapping 410 on the relevant product for purchase displayed in the 3D model. If the user chooses to purchase the selected items, the user canselect the “buy now” or similar icon and proceed to the seller’s website, transitioning from the 3D model back to a 2D webpage for purchase 418. The user can then complete the checkout process 420 and return to the 3D model 422.

[0096] Once the user has selected a product, critical information required for purchase is provided to the user allowing the user to “buy now” or “add to a shopping list” 412. At this point, the user can either select to save for later or buy now 414. Items can be collated in this fashion with the ability for the user to view their “shopping cart” 416 to review potential purchases.

[0097] Being interactive, the interactive 3D model 108 allows a user to browse items within the interactive 3D model 108 by selecting items by selecting an item of interest. This then provides the user with a 2D model, allowing the user to select items within the 2D model to either purchase or enquire about. Based on data acquired on the user via search histories or from 1stparty data etc., more relevant content may be presented to the user based on their demographic, gender, location and other factors.

[0098] Figure 6 provides a flow chart representation from the perspective of a user’s 106 device 102. As can be seen in Figure 6, a user opens their device, which may be a smartphone or other smart device, including a camera capable of scanning codes such as a QR code. The user then scans the QR code 500 presented to the user on a first device 100. The first device then accesses a web address embedded in the QR code providing a 3D model 502 for presentation to the user.Generation of the Interactive 3D Model

[0099] One aspect of the present invention involves the generation of an interactive 3D model which provides selectable objects to be selected by a user. The generation of the interactive 3D model may utilize well known 3D modelling software such as Blender, Autodesk, Maya, 3ds or ZBrush as some examples.

[0100] In one embodiment, the interactive 3D model may be generated using generative artificial intelligence (Al) where an image input on the first device may be captured at a given timestamp and provided to a generative Al system. Generative Al essentially works by creating new content such as text, images or sound based on patterns learned from existing data. These generative Al models are typically based on deep learning architectures such as Generative Adversarial Networks (GANs), Variational Autoencoders (VAEs), and transformer models such as Generative Pre-trained Transformer (GPT). An example of a generative Al system isChatGPT, which can take as an input still or video images and provide an emulation of the still or video images as an output.INTERPRETATION

[0101] The term “temporally changing” is used throughout the description to mean content that changes with time. However, unlike video which changes at a precise rate (for example 24 frames per second), the period related to “temporally changing” may be variable or constant and may be measured in seconds, hours, days, weeks or months etc. For instance, an electronic billboard may have temporally changing content that may be updated when a product changes, or the content may change based on a predetermined sequence.

[0102] Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the specification discussions utilizing terms such as "processing," "computing," "calculating," “determining”, analyzing” or the like, refer to the action and / or processes of a computer or computing system, or similar electronic computing device, that manipulate and / or transform data represented as physical, such as electronic, quantities into other data similarly represented as physical quantities.

[0103] In a similar manner, the term “controller” or "processor" may refer to any device or portion of a device that processes electronic data, e.g., from registers and / or memory to transform that electronic data into other electronic data that, e.g., may be stored in registers and / or memory. A “computer” or a “computing machine” or a "computing platform" may include one or more processors.

[0104] Reference throughout this specification to “one embodiment”, “some embodiments” or “an embodiment” means that a particular feature, structure or characteristic described in connection with the embodiment is included in at least one embodiment of the present disclosure. Thus, appearances of the phrases “in one embodiment”, “in some embodiments” or “in an embodiment” in various places throughout this specification are not necessarily all referring to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner, as would be apparent to one of ordinary skill in the art from this disclosure, in one or more embodiments.

[0105] As used herein, unless otherwise specified the use of the ordinal adjectives "first", "second", "third", etc., to describe a common object, merely indicate that different instances oflike objects are being referred to, and are not intended to imply that the objects so described must be in a given sequence, either temporally, spatially, in ranking, or in any other manner.

[0106] In the claims below and the description herein, any one of the terms comprising, comprised of or which comprises is an open term that means including at least the elements / features that follow, but not excluding others. Thus, the term comprising, when used in the claims, should not be interpreted as being limitative to the means or elements or steps listed thereafter. For example, the scope of the expression a device comprising A and B should not be limited to devices consisting only of elements A and B. Any one of the terms including or which includes or that includes as used herein is also an open term that also means including at least the elements / features that follow the term, but not excluding others. Thus, including is synonymous with and means comprising.

[0107] It should be appreciated that in the above description of exemplary embodiments of the disclosure, various features of the disclosure are sometimes grouped together in a single embodiment, Fig., or description thereof for the purpose of streamlining the disclosure and aiding in the understanding of one or more of the various inventive aspects. This method of disclosure, however, is not to be interpreted as reflecting an intention that the claims require more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive aspects lie in less than all features of a single foregoing disclosed embodiment. Thus, the claims following the Detailed Description are hereby expressly incorporated into this Detailed Description, with each claim standing on its own as a separate embodiment of this disclosure.

[0108] Furthermore, while some embodiments described herein include some but not other features included in other embodiments, combinations of features of different embodiments are meant to be within the scope of the disclosure, and form different embodiments, as would be understood by those skilled in the art. For example, in the following claims, any of the claimed embodiments can be used in any combination.

[0109] In the description provided herein, numerous specific details are set forth. However, it is understood that embodiments of the disclosure may be practiced without these specific details. In other instances, well-known methods, structures and techniques have not been shown in detail in order not to obscure an understanding of this description.[001 10] Similarly, it is to be noticed that the term coupled, when used in the claims, should not be interpreted as being limited to direct connections only. The terms "coupled" and "connected," along with their derivatives, may be used. It should be understood that these terms are not intended as synonyms for each other. Thus, the scope of the expression a device A coupled to a device B should not be limited to devices or systems wherein an output of device A is directly connected to an input of device B. It means that there exists a path between an output of A and an input of B which may be a path including other devices or means. "Coupled" may mean that two or more elements are either in direct physical, electrical or optical contact, or that two or more elements are not in direct contact with each other but yet still co-operate or interact with each other.[001 1 1 ] Embodiments described herein are intended to cover any adaptations or variations of the present invention. Although the present invention has been described and explained in terms of particular exemplary embodiments, one skilled in the art will realize that additional embodiments can be readily envisioned that are within the scope of the present invention.

Claims

What is claimed is:1 . A method for providing access for a user to an interactive digital 3D model of a scene or event relating to a temporally changing digital content, the method comprising the steps of: a) providing the temporally changing digital content for display on a first device, the digital content including one or more scenes or events; b) generating an indicator at times corresponding to the one or more scenes or events, the indicator being associated with a link; c) allowing activation of the link on a second device; d) in response to the activation of the link, rendering on a display of the second device, an interactive 3D model associated with the scene or event, wherein the interactive 3D model includes one or more selectable objects; and e) allowing the user on the second user device to select one or more of the selectable objects to direct the second device to a data portal to interact with the selected object(s).

2. The method of claim 1 , wherein the temporally changing digital content includes a digital billboard.

3. The method of claim 1 , wherein the temporally changing digital content includes video content.

4. The method of claim 3, wherein the video content includes a television show.

5. The method of claim 3, wherein the video content includes on-demand video content.

6. The method of claim 3, wherein the video content includes a live broadcast video event.

7. The method of claim 3, wherein the video content includes online video.

8. The method of claim 3, wherein the video content includes social media video.

9. The method of any one of the preceding claims, wherein the data portal provides updated content of the link independent of the one or more scenes or events including factors such as when the content is viewed or a demographic of the user.

10. The method of claim 9, wherein the data portal provides updated content of the link based on other factors such as a season.1 1 . The method of any one of the preceding claims, wherein the data portal includes a webpage containing a list of the selected objects.

12. The method of claim 1 1 , wherein the webpage provides functionality to purchase the selected objects.

13. The method of claim 1 1 , wherein the webpage is a third party webpage hosted by an owner of the selected objects.

14. The method of claim 1 , wherein the interaction with the selected object(s) includes a purchase.

15. The method of any one of the preceding claims, wherein the first and the second devices are the same device.

16. The method of any one of the preceding claims, wherein the activation of the link includes capturing a QR code, HTML link, or a soft button on a touchscreen.

17. The method of any one of the preceding claims, wherein the one or more selectable objects are indicative of products that can be purchased.

18. The method of any one of the preceding claims, wherein the temporally changing digital content is pre-recorded and wherein the indicator is generated by inserting a trigger at timecodes of the pre-recorded content corresponding to a scene or event.

19. The method of any one of the preceding claims, wherein the temporally changing digital content is broadcast live and wherein the indicator is inserted into the temporally changing digital content in real-time or near real-time through a push operation by an operator.

20. The method of any one of the preceding claims, wherein upon activating the link, acquiring information about the user.21 . The method of any one of claims 1 to 20, wherein the indicator is displayed on the first device.

22. The method of any one of claims 1 to 21 , wherein the indicator is displayed on the second device.

23. The method of any one of the preceding claims, wherein the interactive 3D model is generated from a digital scan of a real environment associated with the scene or event using a 3D scanner.

24. The method of any one of claims 1 to 23, wherein the interactive 3D model is generated from a plurality of images of a real environment and fed to a generative machine learning model.

25. The method of any one of the preceding claims, wherein at least a subset of the one or more selectable objects are generated from a digital scan of a real object using a 3D scanner.

26. The method of any one of claims 1 to 25, wherein at least a subset of the one or more selectable objects are generated from a plurality of images of a real object and fed to a generative machine learning model.

27. The method of any one of the preceding claims, wherein the interactive 3D model is configurable.

28. A system for providing access for a user to an interactive digital 3D model of a scene or event relating to a temporally changing digital content, the system comprising: a) a first device, adapted to display the temporally changing digital content, the digital content having a narrative including one or more scenes or events; b) at least one processor adapted to: generate an indicator at times corresponding to a display of at least one of the scenes or events on the first device, the indicator being associated with a link; c) a second user device, adapted to receive the link and allowing the user to activate the link; wherein, in response to the activation of the link, the second user device is configured to render on a display of the second user device, an interactive 3D model of the scene or event, wherein the interactive 3D model includes one or more selectable objects and allow the user on the second user device to select one or more of the selectable objects to direct the second device to a data portal to interact with the selected object(s).

29. The system of claim 28, wherein the first device includes a digital billboard.

30. The system of claim 28, wherein the temporally changing digital content includes video content.31 . The system of claim 28, wherein the video content includes a television show.

32. The system of claim 28, wherein the video content includes on-demand video content.

33. The system of claim 28, wherein the video content includes a live broadcast video event.

34. The system of claim 28, wherein the video content includes online video.

35. The system of claim 28, wherein the video content includes social media video.

36. The system of any one of claims 28 to 35, wherein the data portal provides an updated content of the link dependent on the times corresponding to the display of the at least one of the scenes.

37. The system of claim 36, wherein the data portal updates the content of the link based on other factors such as a season.

38. The system of any one of claims 28 to 37, wherein the data portal includes a webpage containing a list of the selected objects.

39. The system of claim 38, wherein the webpage provides functionality to purchase the selected objects.

40. The system of claim 38, wherein the webpage is a third party webpage hosted by an owner of the selected objects.41 . The system of claim 28, wherein the interaction with the selected object(s) includes a purchase.

42. The system of any one of claims 28 to 41 , wherein the first and the second devices are the same device.

43. The system of any one of claims 28 to 42, wherein the activation of the link includes capturing a QR code, HTML link, or a soft button on a touchscreen.

44. The system of any one of claims 28 to 43, wherein the one or more selectable objects are indicative of products that can be purchased.

45. The system of any one of claims 28 to 44, wherein the temporally changing digital content is pre-recorded and wherein the indicator is generated by inserting a trigger at timecodes of the pre-recorded content corresponding to a scene or event.

46. The system of any one of claims 28 to 45, wherein the indicator is inserted into the temporally changing digital content in real-time or near real-time through a push operation by an operator.

47. The system of any one of claims 28 to 46, wherein upon activating the link, information about the user is acquired.

48. The system of any one of claims 28 to 47, wherein the indicator is displayed on the first device (e.g. TV).

49. The system of any one of claims 28 to 48, wherein the indicator is displayed on the second device (e.g. smartphone).

50. The system according to any one of claims 28 to 49, wherein, the interactive 3D model is configurable.51 . The method or system of any one of the preceding claims, wherein the interactive digital 3D model is comprised of one or more configurable modules defining the one or more selectable objects within the interactive digital 3D model.

Images