Systems and methods for identifying b-roll conditions in live stream or live rendered content

Abstract

The video stream management system (150) includes a video controller (152) that renders video in real-time. Further, the video stream management system (150) includes a display (102) that is communicatively coupled to the video controller (152) and displays a primary video feed (168) that includes the real-time rendered video. The video controller (152), the display (102), or a combination thereof, embeds a pixel pattern (164) in the primary video feed (168). Additionally, the video feed management system (152) monitors one or more displayed images on the display (102) to identify an error in the primary video feed (168).

Description

[0001] Cross-reference to related applications

[0002] This application claims priority and interest in U.S. Provisional Application No. 62 / 699,739, filed July 18, 2018, entitled “System and Method for Identifying B-Roll Condition in Live Streaming or Live Rendering Content,” which is hereby incorporated in its entirety by reference for all purposes. Background Technology

[0003] This disclosure generally relates to the field of amusement parks. Specifically, embodiments of this disclosure relate to techniques for managing amusement park operations, including managing video streams for rides or attractions.

[0004] In certain settings (such as amusement park settings), certain rides and other equipment have become increasingly interactive. Among other things, this means that certain aspects of rides and equipment may not necessarily be scripted. In fact, as such rides and customer-activated equipment become more dynamic (e.g., rendered in real-time or game-centric), it becomes increasingly difficult to identify when elements of an attraction (e.g., the images displayed at the attraction) are not displaying as intended. Assigning operators to monitor and / or identify unexpected display problems can lead to inaccurate and inefficient park operations. Furthermore, this reliance can result in situations where the customer experience is negatively impacted by equipment malfunctioning or breaking down. Therefore, there is a need for technologies and systems capable of using the displayed media to identify problems and take corrective action to address them. Summary of the Invention

[0005] Certain embodiments corresponding in scope to the originally claimed subject matter are summarized below. These embodiments are not intended to limit the scope of this disclosure, but rather are intended only to provide a brief overview of some of the disclosed embodiments. In fact, this disclosure may include a wide variety of forms that may be similar to or different from the embodiments set forth below.

[0006] In one embodiment, the video streaming management system includes a video controller that renders video in real time. The video streaming management system also includes a display communicatively coupled to the video controller and displaying a main video feed that includes the real-time rendered video. The video controller, the display, or a combination thereof causes pixel patterns to be embedded in the main video feed. The video feed management system monitors one or more displayed images on the display to identify errors in the main video feed.

[0007] In one embodiment, a method for managing a video feed includes embedding a dynamic pixel pattern into frames of a real-time video feed. The dynamic pixel pattern includes a first pixel pattern associated with a first frame and a second pixel pattern associated with a second frame, wherein the first pixel pattern is different from the second pixel pattern. The method also includes using a display to display the real-time video feed having the dynamic pixel pattern. Additionally, the method includes monitoring the image displayed on the display. Furthermore, the method includes identifying an error in the real-time video feed in response to determining that the monitored displayed image includes a displayed pixel pattern that does not match the embedded dynamic pixel pattern. Furthermore, the method includes switching from displaying the real-time video feed to displaying an alternative video feed in response to identifying the presence of an error in the real-time video feed.

[0008] In one embodiment, the video streaming management system includes: one or more sensors that detect customer presence; a video streaming controller; and a display communicatively coupled to the video streaming controller. The video streaming controller renders video in real time based on the detected customer presence to generate a main video stream, and embeds a pixel pattern into the main video stream to generate the video stream. The display receives the video stream, displays the video stream to generate a displayed image, monitors the displayed image to identify errors based on a comparison of the displayed image with the pixel pattern, and generates an error signal based on the identified errors. Attached Figure Description

[0009] These and other features, aspects, and advantages of this disclosure will become better understood when the following detailed description is read with reference to the accompanying drawings, in which the same characters denote the same parts throughout the drawings, wherein:

[0010] Figure 1 This is a schematic diagram of an amusement park ride that includes a scenic spot utilizing a video stream monitoring system, according to an embodiment of the present disclosure;

[0011] Figure 2 It is based on the embodiments of this disclosure. Figure 1 A block diagram of the video stream monitoring system used at the tourist attraction;

[0012] Figure 3 This is a process flowchart of a method for using a video stream monitoring system to monitor alternative video streams and send alternative video streams to a display based on detected unexpected displays, according to embodiments of the present disclosure.

[0013] Figure 4 This is a schematic diagram of an image data frame provided by an interactive or real-time rendered video stream having an embedded dynamic pixel pattern, according to an embodiment of the present disclosure.

[0014] Figure 5 It is based on the embodiments of this disclosure. Figure 4A schematic diagram illustrating an example of an unexpected display on a frame;

[0015] Figure 6 It is based on the embodiments of this disclosure. Figure 4 A schematic diagram of another example of unexpected display on a frame; and

[0016] Figure 7 This is a schematic diagram of image data frames provided by an alternative video stream according to an embodiment of the present disclosure. Detailed Implementation

[0017] One or more specific embodiments of this disclosure will be described below. To provide a concise description of these embodiments, not all features of the actual implementation may be described in the specification. It should be appreciated that, as in any engineering or design project, numerous implementation-specific decisions must be made to achieve the developer's specific goals, which may vary depending on the implementation, such as compliance with system-related constraints and business-related constraints. Furthermore, it should be appreciated that such development efforts may be complex and time-consuming, but for those skilled in the art who benefit from this disclosure, such development efforts will be nothing more than routine tasks of design, fabrication, and manufacturing.

[0018] When describing elements of various embodiments of this disclosure, the articles “a,” “an,” and “the” are intended to mean the presence of one or more of the elements. The terms “comprising,” “including,” and “having” are intended to be inclusive and mean that additional elements may exist in addition to those listed. Furthermore, it should be understood that references to “an embodiment” or “an embodiment” in this disclosure are not intended to be construed as excluding the existence of additional embodiments that also incorporate the described features.

[0019] Amusement parks are characterized by a wide variety of entertainment, such as rides, shows, and games. Different types of entertainment can include interactive or real-time rendered features that enhance the customer experience at the amusement park. Interactive features can include rides or equipment that are activated based on the presence of a customer. For example, a display used as part of the attraction's environment can be interactive, allowing displayed elements or objects to become activated or triggered, changing based on the detected presence of a customer, rather than operating according to a timer and / or as a pre-recorded playback. Changes can include variations in the displayed video environment, including lighting, textures, animations, and so on. However, despite this, changes to the display may preserve some objects or features that might be perceived as unchanged. Therefore, it can be difficult to observe when an interactive aspect of a video stream ceases to be interactive because some features (e.g., the background) may remain unchanged. Alternatively or concurrently, the video stream display can be real-time rendered, making changes to the video display "real-time." Playback can be fast and perceived as real-time. It can also be difficult to observe when a real-time rendered playback ceases to be real-time (e.g., in the case of a stuck display).

[0020] External factors can cause interactive and / or real-time rendered video streams to behave unexpectedly, causing interactive features on the display to cease engaging with customers, or real-time rendered images to stop changing in real time. Consequently, the displayed video stream may no longer provide a seamless viewing experience for customers on rides. Typically, operators may be assigned the task of monitoring and identifying unexpected display behavior. However, relying on a single attraction operator to identify this behavior can distract the operator from other important tasks, such as managing customer flow and ride dispatch. Moreover, in the presence of multiple displays, monitoring in this manner may require multiple operators. Tasks such as identifying problems, providing resolution data, and reviewing video stream data by monitoring personnel can lead to unreasonable delays in ride dispatch and potential ride downtime. Additionally, inconsistent and inaccurate monitoring of the video stream may occur due to various operators, as identifying unexpected behavior may be difficult to determine in interactive or real-time video streams. Therefore, manual video display monitoring can be difficult and inefficient, resulting in poor customer viewing and interaction, and unnecessary wait times between ride dispatches, which can further lead to reduced customer enjoyment and longer queues at the amusement park.

[0021] It should be noted that while the examples provided herein (such as using the proposed techniques to facilitate monitoring of displays provided on rides or as part of the ride environment) can generally be presented in the context of amusement parks and ride attractions, the techniques in this disclosure can be applied to other conditions and / or situations not related to amusement parks. Therefore, the examples presented should be understood as merely reflecting real-world examples of display monitoring systems on rides to provide contexts useful for discussion, and should not be construed as limiting the applicability of this approach. Rather, this approach should be understood as applicable to other situations in which video is displayed.

[0022] Considering the current situation, Figure 1 This is a schematic representation of an amusement park ride 100 that may include a video stream on display 102. For example, in the depicted embodiment, the amusement park ride 100 may include one or more displays 102 along a ride path 104. The ride 100 may be accessed via a customer queue 106, and the ride may be considered ready when a variety of conditions, including display conditions, are met. Display conditions may be conditions that allow for a seamless viewing experience on one or more displays 102. As shown, a ride operator 110 may operate the ride such that customers are signaled to enter the ride carriage 112 (e.g., a vehicle) when the operator determines that the display conditions are met.

[0023] When the interactive and / or real-time rendering features of the video stream are functioning appropriately, display conditions can be met so that features of the video stream (including environment, elements, or animations) are perceived and / or triggered in real time in response to the detected presence or gesture of a customer. Detection of customer presence can include a wide variety of sensing mechanisms.

[0024] In one embodiment, and as depicted, a camera 114 or a series of cameras 114 mounted along the riding device 100 (including along the riding path 104) can detect the presence and / or gestures of a customer. Alternatively or additionally, the camera 114 can be integrated into the display 102. The camera 114 can use a wide variety of detection mechanisms, including but not limited to facial recognition, skeletal tracking, body thermal recognition, and so on. The camera 114 can also capture the customer's movements (e.g., gestures) and use those captured movements to simulate real-time rendering or interaction with elements or animations displayed on the display 102. The video stream displayed on the display 102 can also be a real-time feed of the customer (captured by the camera 114) or a real-time rendered video including the customer's representation.

[0025] In another embodiment, customer presence detection can be performed by one or more sensors, such as a radio frequency identification (RFID) tag 118 incorporated into the carriage 112 or a customer wearable device, or a weight sensor 120 positioned along the carriage track 116. These sensors can be placed or positioned in an area based on where a customer is expected to be present (e.g., on a seat in the carriage 112). The RFID tag 118 can communicate with an electronic reader 119 incorporated into the riding device 100 (e.g., on the carriage track 116 or the carriage 112 (e.g., inside the carriage 112, on the side of the carriage 112, or on the entrance passage of the carriage 112)) to indicate the presence of the RFID tag 118. Thus, as the carriage 112 passes the electronic reader 119, the electronic reader 119 positioned on the riding path 104 (e.g., the carriage track 116 or the carriage 112) can scan the RFID tag 118 located on the carriage 112. Alternatively, weight sensor 120 may be mounted on boarding track 116 and may be used to indicate the presence of carriage 112 on boarding track 116 based on a predetermined weight. Detecting the presence of a customer can trigger real-time rendering or interactive video on display 102. Additionally, the sensor can trigger a customer presence indication to camera 114, thereby activating the camera or narrowing the area where the customer might be detected. Thus, camera 114 can be used alone or in conjunction with other detection mechanisms (e.g., RFID tag 118 or weight sensor 120) to detect and / or track customers.

[0026] Once a customer is identified and tracked, display 102 can change so that objects on display 102 appear to interact with or react to the customer. Animation or live streaming rendering can react to the customer's movement and position relative to display 102. In the illustrated embodiment, a clown is depicted on display 102. The depicted clown can react in response to the detection of a customer and / or their tracked movement (e.g., juggling). Thus, the clown video display can be live streaming video and / or interaction based on customer movement. As previously mentioned, interaction and / or live streaming video (e.g., clown video) may stop interactivity or live display and may rely on either the customer or operator 110 to detect unintended changes in the display. The video streaming management system 150 according to embodiments of this disclosure can be used to automatically detect unintended displays and switch to alternative video sources to ensure a seamless viewing experience.

[0027] In light of the foregoing, the currently disclosed embodiments can determine when the video stream sent to display 102 and / or display 102 itself ceases to display the intended image. That is, in some embodiments, the video stream management system 150 can use a video stream with an embedded pixel pattern to detect unintended display and then switch to an alternative video stream to provide seamless viewing for the best customer experience.

[0028] For the configuration and functions of the video stream management system 150, please refer to [link / reference]. Figure 2 And to understand better, Figure 2 This diagram illustrates a video stream management system 150 used to monitor and switch video streams using the techniques provided herein. The video stream management system 150 includes a video stream controller 152, a monitoring subsystem 154, a display 102, and at least one camera 114. The display 102 may represent one or more displays. Although some descriptions below describe the camera 114, video stream controller 152, monitoring subsystem 154, and display 102 as separate components of the video stream management system 150, either indirectly or directly coupled via wires / cables (this indicates a particular embodiment), it should be noted that the methods and systems can be implemented and realized using any suitable component arrangement (e.g., all components integrated into a single display 102). For example, the monitoring subsystem 154 may also be included in the video stream controller 152, and the video stream controller 152 may be integrated into the display 102. Furthermore, the display 102 may include the camera 114. Thus, in one embodiment, all functions (e.g., detection, monitoring, switching, etc.) can be provided by a single integrated display 102. The display 102 may include multiple pixels.

[0029] As illustrated, one or more cameras 114 may be coupled to video stream controller 152. Cameras 114 may be used to track customers (e.g., detect customer presence and / or capture customer movement), allowing customer movement to be used for interactive or real-time video streaming. Once customer presence or movement is detected and / or tracked, camera 114 may send detection and / or tracking signals to video stream controller 152. Video stream controller 152 may use the camera signals to enable video stream 156 to be sent to display 102. Video stream controller 152 may include memory 158 for storing instructions executable by processor 160.

[0030] Processor 160 may include one or more processing means, and memory 158 may include one or more tangible, non-transitory machine-readable media. By way of example, such machine-readable media may include RAM, ROM, EPROM, EEPROM, or optical disc storage devices, magnetic disk storage devices, or other magnetic storage devices, or any other medium capable of carrying or storing desired program code in the form of machine-executable instructions or data structures and accessible by processor 160 or other processor-based devices. Processor 160 may include a processing core 162 for executing machine-executable instruction algorithms stored in memory 158.

[0031] The stored algorithms may include algorithms for sending video data streams (including embedded pixel patterns 164, interactive video streams 166, real-time rendered video streams 169, and / or alternative video sources 174, such as pre-recorded video streams) to the display 102. The processor 160 may also include a processor-side interface 170 for software applications running on the processing core 162 to interact with hardware components associated with the processor 160 on the riding device 100 (such as the display 102 and camera 114).

[0032] Embedding pixel pattern 164 may include modifying data associated with main video stream 168 to generate video stream 156 provided to display 102. As provided herein, main video stream 168 may refer to a video stream associated with interactive video, such as real-time rendered video stream 169 or interactive video stream 166. Main video stream 168 is embedded with embedded pixel pattern 164 by modifying image data of one or more images (e.g., individual frames of the main video stream) such that the modified one or more images display embedded pixel pattern 164 when displayed. In one embodiment, the image data of one or more images includes information encoding color for each pixel associated with embedded pixel pattern 164. Thus, when modified using embedded pixel pattern 164, one or more images of video stream 156 display different colors for one or more pixels relative to main video stream 168. It should be understood that images in the main video stream may include certain pixels that have already been encoded to display the color associated with embedded pixel pattern 164. That is, one or more source images may already be black at a pixel location corresponding to a black pixel of embedded pixel pattern 164. However, the embedded pixel pattern 164 may be complex and variable enough that the source image is statistically unlikely to display the complete embedded pixel pattern 164 without modification. Furthermore, the embedded pixel pattern 164 may be dynamic and change frame-by-frame, further reducing the alignment between the original source image and the embedded pixel pattern 164. Dynamic characteristics of the embedded pixel pattern 164 may include translating the embedded pixel pattern 164 to different pixel positions across successive frames, changes in the color of all pixels associated with the embedded pixel pattern 164 in subsequent frames, and / or different arrangements of pixels within the embedded pixel pattern 164.

[0033] The video stream controller 152 may also include a switch 172 or a series of switches coupled to the processor 160. Based on instructions executed from the processor 160, the switch 172 can be used to switch the data stream sent to the display 102. As depicted, and in some embodiments, the processor 160 may transmit a main video stream 168 (such as a real-time rendered video stream 169 or an interactive video stream 166) along with an embedded pixel pattern 164 as video stream 156 to the display 102 via a cable (e.g., a High Definition Multimedia Interface (HDMI) cable). The embedded pixel pattern 164 may be embedded in the main video stream 168 and may be used to detect unintended display behavior.

[0034] In addition to providing video stream 156 to display 102, video stream management system 150 can also operate to automatically identify unintended display behavior by switching video streams. For example, video stream management system 150 can switch the displayed video stream 156 to an alternative video source 174, which can be simultaneously streamed to display 102. The switch 172 of video stream controller 152 and / or the display processor 165 of display 102 can control which stream is displayed on display 102 for the riding device 100. In the depicted embodiment, display 102 may be configured to default to video stream 156 and may be switched to alternative video source 174 based on additional instructions sent by video stream controller 152 and processed by display processor 165. Upon detection of unintended display behavior, additional instructions may be sent and executed based on display error detection performed by monitoring subsystem 154.

[0035] Specifically, the monitoring subsystem 154 may include error detection logic 176, a monitoring subsystem memory 180, and a monitoring subsystem processor 182. Algorithms may be used, for example, to identify errors on the display 102 using error thresholds and pattern identifiers. The algorithms may be stored in the monitoring subsystem memory 180 and executed by the monitoring subsystem processor 182.

[0036] As will be discussed in detail below, the pixel comparator 178 of the error detection logic 176 can determine unexpected display. The pixel comparator 178 can be used to compare image pixels of a video stream 156 displayed on the display 102 with one or more pixel patterns stored in the monitoring subsystem memory 180 and / or pixel patterns transmitted via an embedded pixel pattern 164. The monitoring subsystem 154 can relay received display data 184 regarding pixels or pixel patterns to the video stream controller 152. The error detection logic 176 can generate an error signal 186 when an error is identified. The monitoring subsystem 154 can also send the error signal 186 to the video stream controller 152 when the error detection logic 176 detects unexpected display behavior. Unexpected display behavior can be determined based on a comparison of the expected pixel pattern stored in the monitoring subsystem memory 180 with the pixel pattern displayed on the display 102. In some embodiments, information regarding the embedded pixel pattern 164 may be accessible only to authorized users. In such an embodiment, the embedded pixel pattern 164 may be encrypted or accessible only using a key. A key stored in subsystem memory 180 for an authorized user can be used to decrypt or verify the embedded pixel pattern 164, also stored in monitoring subsystem memory 180, into plaintext, which can then be used to identify errors (e.g., unexpected display behavior). The key may be generated based on metadata, network connectivity, and / or memory location (e.g., subsystem memory 180). For example, if the video stream controller 152 is connected to a specific network (such as an unauthorized or unidentified network), the video stream controller 152 may not access the embedded pixel pattern 164 without entering or loading a paired key (e.g., a unique private key for each network connection). Accordingly, the video stream controller 152 may not be able to identify the currently displayed pixel or compare the currently displayed pixel with the expected embedded pixel pattern 164 to identify unexpected display behavior without using the key. Examples of unintended display behavior may include, but are not limited to, dialog boxes that block at least a portion of video frames transmitted by video stream 156, stuck video frames, malfunctions in displaying video stream 156 (e.g., a black screen), and / or incorrect pixel illumination (e.g., pixel color or brightness) at specific pixel locations on display 102. Alternatively or additionally, an error detection algorithm on video stream controller 152 may be used to determine erroneous or unintended display behavior based on received display data 184.

[0037] In an embodiment, comparator 178 operates to count pixel mismatches in pairs between the displayed image and the expected pixel pattern for each frame in video stream 156. A threshold may be set to a threshold number of pixel mismatches, whereby a number of mismatches lower than the threshold indicates a correct video stream 156, and a number of pixel mismatches higher than the threshold indicates an error. For example, there may be some allowable tolerance of 1-2 pixel mismatches to account for non-functional pixels. In an embodiment, comparator 178 may be configured to require that there be pixel mismatches above the threshold in at least a certain number of frames before triggering an error indication and thus triggering a switch to alternative video source 174. Each individual frame or only a subset of frames in video stream 156 may include an embedded pixel pattern 164 (e.g., the expected pixel pattern). The comparator can associate the displayed image with the expected embedded pixel pattern 164. That is, for dynamically embedded pixel patterns 164 (wherein the pixel patterns change between frames of the video stream), the expected embedded pixel pattern 164 for the displayed image of the video stream can be associated with a specific frame identifier, which can be part of the metadata for the video stream 156. The display processor 165 can provide the metadata with the frame identifier to the error detection logic 176 during display so that the expected pixel pattern for that specific frame can be used by the comparator 178.

[0038] Error detection can occur at the receiving device (e.g., display 102) of the video stream 156. Any active rendering or interactive changes to the video stream 156 can occur at the video stream controller 152. However, in other embodiments, the video stream controller 152 and the display 102 are provided as a single, integrated device. In embodiments, the video stream 156 is formed from a series of successive image frames, whereby each image frame is being displayed using multiple pixels of the display 102. Pixel patterns (e.g., embedded pixel patterns 164) can be formed or combined from only a subset of all pixels used in displaying an image using the display 102. For example, the pixel pattern for each frame or image may involve less than 10%, 5%, 2%, or 1% of the available pixels on the display 102. In this way, the pixel comparator 178 of the error detection logic 176 at the receiving device can evaluate only a subset of the available pixels of the display 102 (e.g., only those pixels associated with the expected pixel pattern), thus avoiding complex full image / video comparison operations for faster and more efficient error detection within the video stream 156.

[0039] To illustrate in detail an example of the error detection logic 176 of the monitoring subsystem 154, in Figure 3The present invention describes a process 200 for detecting unexpected displays and / or generating a response to unexpected displays. Generally, process 200 includes: generating (process block 202) a main video stream 168; embedding (process block 204) a pixel pattern 164 in the main video stream 168; sending (process block 206) a video stream 156 including the main video stream 168 with the embedded pixel pattern 164 to a display 102; evaluating (process block 208) one or more images displayed on the display 102 to identify whether an expected pixel pattern is being displayed; and evaluating (decision block 210) whether the displayed image includes an expected pixel pattern within a predetermined threshold. If the pixel pattern identified in the displayed image is within the threshold, the display 102 continues (process block 212) to display the main video stream 168 with the embedded pixel pattern 164 as video stream 156, and if the pixel pattern is not within the threshold, the display 102 switches (process block 214) video stream 156 to alternative video source 174.

[0040] Although process 200 is described using actions in a specific sequence, it should be understood that the described actions may be performed in a different sequence than described, and some described actions may be omitted or not performed at all. Generally, at least some of the steps of process 200 may be implemented at least partially by the video stream management system 150. Specifically, these steps may be implemented at least partially by the processor 160 of the video stream controller 152 or the monitoring subsystem processor 182 of the monitoring subsystem 154, which executes instructions stored in a tangible, non-transitory computer-readable medium (such as monitoring subsystem memory 180). In alternative or additional embodiments, at least some steps of process 200 may be implemented by any other suitable components or control logic, etc.

[0041] Therefore, in some embodiments, the video stream controller 152 can generate (process block 202) a main video stream 168. As previously discussed, the main video stream 168 may include any video data stream, such as an interactive video stream 166, a real-time rendered video stream 169, and / or alternative video sources 174, such as a pre-recorded video bitstream. The main video stream 168 may be pre-determined by the user or operator 110. The type of video stream used may be based on the riding device 100 and / or the target customer. For example, Figure 1 The type of ride for the ride device 100 may include a clown theme and may target younger customers, and therefore, the main video stream 168 may include an interactive video stream 166 that allows the clown to respond to the customer’s gestures.

[0042] In some embodiments, the video stream controller 152 may embed a pixel pattern 164 as an embedded pixel pattern along with the main video stream 168 to create a video stream 156 (process block 204). The embedded pixel pattern 164 may be a data stream that may cause pixels or pixel arrays of display 102 to emit a specific color or a color that is not present. In some embodiments, the pixel pattern may be dynamic, such that the color at a particular pixel of display 102 may change with each frame of the displayed image. In other embodiments, the pixel pattern may be static and remain unchanged between frames.

[0043] After the video stream controller 152 has embedded the embedded pixel pattern 164 into the main video stream 168, the video stream controller 152 can send the data stream as a video stream 156 (process frame 206) to the display 102. Because the pixel pattern is embedded within the video stream 156 (e.g., within one or more individual frames of the main video stream 168), the pixel pattern can be designed so that it is invisible or undetectable to the human eye. In this way, the pixel pattern can only be detected by the video stream management system 150, and customers can continue to enjoy a seamless viewing experience on the riding device 100.

[0044] Once the main video stream 168 with the embedded pixel pattern 164 is sent to the display 102, the monitoring subsystem 154 can determine (process block 208) the pixel pattern being displayed on the display 102. To illustrate the determination of the expected pixel pattern and the displayed image for error detection, Figure 4-6 An image data frame with an embedded pixel pattern 164 is shown. Furthermore, although some of the description below describes the pixel pattern shown alongside the displayed image data (which can be described as facilitating the interpretation of error detection using pixels), it should be noted that the implemented methods and systems can utilize hidden pixel patterns that are not detectable by the human eye. That is, the embedded pixel pattern 164 is designed to be detectable by the video stream management system 150 and its components, but not by the customer.

[0045] As explained, Figure 4The image data frame 220 (frame 1) and the image data frame 222 (frame 2) depict interactive and / or real-time rendered image data with dynamically embedded pixel patterns 224 (e.g., pixel pattern 164) displayed on display 102. Pixel pattern 224 may include at least one pixel displayed on display 102 together with each of image data frames 220 and 222. The first image data frame 220 and the second image data frame 222 may display a real-time rendered image, illustrated herein by way of example as a clown, such that the video stream provides the viewer with the perception of a clown moving in real time. As shown, pixel pattern 224 may include black or colored light emitted by pixels on or around elements of the image (e.g., the clown). For example, pixels of display 102 may be intended to emit black light (e.g., no light), and black light may be described herein as black pixel 226. In the depicted embodiment, black pixel 226 may be intended in the bottom corner of display 102 in the first frame 220.

[0046] However, in a subsequent image data frame (second frame 222), the clown may have changed his actions, and the pixel pattern 224 may also have changed. As depicted in second frame 222, black pixel 226 can be expected to appear in the upper right corner of display 102. It should be understood that, by way of example, the illustrated pixels (e.g., black pixel 226 and / or white pixel 227) are shown as relatively large within display 102. However, each pixel can be sized to correspond to the pixel size of display 102, which is a function of display resolution. Moreover, the illustrated pixels of pixel pattern 224 may also include groups of adjacent pixels forming larger shapes or patterns. Furthermore, while the illustrated pixel pattern 224 is formed by black pixel 226 and white pixel 227, other color combinations are also contemplated. Furthermore, pixel pattern 224 can be chosen to be generally invisible to the viewer. In one example, the pixel pattern may be distributed around segments of frames 220, 222 associated with the background image. In one embodiment, pixel pattern 224 is formed only by non-adjacent pixels to render a pixel pattern 224 that is less discernible to the viewer. In another embodiment, individual pixels of pixel pattern 224 are selected to be a different color from each of the surrounding pixels in the intended image of frames 220, 222. In yet another embodiment, individual pixels of pixel pattern 224 are separated from other pixels in pixel pattern 224 by at least two, three, five, or ten other pixels.

[0047] The display 102 can show an unexpected pixel pattern indicating an unintended display. For illustration purposes, Figure 5An unexpected pixel pattern 225 is depicted on the erroneous first frame '228' (frame 1). Although the erroneous first frame '228' displays the correct, expected frame image, such as the image of first frame 220, the expected black pixel 226 of pixel pattern 224 fails to appear, indicating an incorrect pixel 229. In other embodiments, the unexpected pixel pattern 225 may include, but is not limited to, one or more pixels appearing on display 102 as different colors and / or changed positions. In some implementations, such changes in pixel position or color may be caused by errors in video rendering that may not be visible in the overall displayed image itself. This error may be caused by digital media errors in video stream 156, which may otherwise go undetected by other monitoring mechanisms (such as limited detection of the video output of display 102 via hardware / cable or physical observation by operator 110 or customer) and result in false positives determined by such other monitoring mechanisms. Therefore, the embedded pixel pattern 164 can allow for internal error detection and early detection that might otherwise rely on external data and / or hardware.

[0048] Alternatively or additionally, unintended displays may also include errors in the displayed image itself. For example, if the rendered image gets stuck or fails to change from one image frame to the next, the expected illumination at the pixel location may not appear on display 102. Furthermore, additional images on display 102 (such as dialog boxes or other user prompts) may cause the expected illumination at the pixel location to not appear on display 102.

[0049] For the purpose of explanation, Figure 6 The error is explained in the second frame '230 (frame 2)'. Figure 6The expected image and expected pixel pattern 224 of the second frame 222 are displayed, but with an error dialog box 231 awaiting user input. The dialog box 231 may appear as a result of a change in video output or a temporary loss, etc. Due to the dialog box 231, the expected black pixels 226 of the expected pixel pattern 224 may not appear in the expected position, thus indicating that the image displayed on the display 102 may be an unexpected error. Therefore, determining the displayed pixel pattern 224 can detect unexpected display errors. In embodiments, the detected error can be characterized by distributing the pixel pattern 224 to a variety of locations around the display 102 via dynamic and / or static pixel patterns positioned in all quadrants of the display 102. For example, a display failure at a specific pixel location and that failure is limited to a single location, while the rest of the display remains undamaged, may present differently from a video jam error that more globally affects the display. That is, a single pixel failure may not be associated with an error in the video stream 156, but rather with a hardware error at the display 102. Therefore, such an error may not trigger a switch to the alternative video source 174. In another example, an error associated with a pop-up error window could be related to pixel mismatch in a predictable area of ​​the display 102.

[0050] Return to Figure 3 In process 200, the error detection logic 176 of the monitoring subsystem 154 can determine (decision box 210) whether the displayed pixel pattern 224 is within a predetermined threshold of the expected dynamically embedded pixel pattern 164 (e.g., pixel mismatch). The error detection logic 176 can compute this determination using algorithms previously discussed (e.g., error thresholding algorithms and pattern thresholding algorithms). The algorithm can analyze data received from the display 102 (e.g., displayed image data and pixel patterns) and compare that data with predetermined thresholds that can be set by the user or operator 110. Thus, error thresholds for pixel positions or emitted colors can be set so that they do not interfere with a seamless viewing experience for customers on the riding device 100. The monitoring subsystem 154 can relay display data 184 and / or error signals 186 to the video stream controller 152. Alternatively, the video stream controller 152 may use an algorithm stored in the memory 158 of the video stream controller 152 to determine whether the relayed display data 184 is within a predetermined threshold of the expected embedded pixel pattern 164.

[0051] When it is determined that the displayed pixel pattern is within the threshold of the expected embedded pixel pattern 164, the video stream controller 152 may continue to send the video stream 156 (process block 212) to the display 102. Thus, the video stream 156 may continue to send the main video stream 168 with the embedded pixel pattern 164.

[0052] On the other hand, if the displayed pixel pattern is not within the threshold of the expected embedded pixel pattern 164, resulting in a video stream error (“B-roll” condition) being determined, the video stream controller 152 can use switch 172 to switch from video stream 156 (process block 214) to alternative video source 174. For illustration, Figure 7 Depict the third frame 232 (frame 3). Figure 7 Frames of alternative video source 174 are displayed. As shown, the third frame 232 of alternative video source 174 may include non-interactive images, such as image data of pre-recorded B-roll video. The pre-recorded video roll may not be interactive or rendered in real time, and therefore may not be embedded with pixel pattern 164 to detect errors that might otherwise be difficult to detect using interactive or real-time rendered video streams. Alternatively, alternatively, alternative video source 174 may be a complete backup of the original video stream 156 that was being displayed before the unexpected display was detected. For example, a complete backup may include a redundant system for generating a video stream 156 that includes the main video stream 168 along with the embedded pixel pattern 164. Alternatively, alternatively, alternative video source 174 may include different interactive or real-time rendered video streams along with the embedded pixel pattern 164. The alternative video source 174 option may be determined based on the source best suited to provide a seamless viewing experience for customers on riding device 100.

[0053] Alternate video source 174 can be switched within a rapid response time frame, resulting in minimal latency associated with stream switching. To provide a rapid switching mechanism, both alternative video source 174 and video stream 156 can be streams sent to display 102 simultaneously. However, display 102 can be instructed to change its image output based on instructions received via video stream controller 152 in the event of an error detection or unexpected display. In this embodiment, the alternative video source can be stored locally at display 102. In this way, the switching may not be observed or noticeable by the customer, thus continuing to provide a seamless viewing experience.

[0054] While only certain features of this disclosure have been described and illustrated herein, many modifications and alterations will occur to those skilled in the art. Therefore, it will be understood that the appended claims are intended to cover all such modifications and alterations that fall within the true spirit of this disclosure. It should be appreciated that any features described or illustrated above with respect to the drawings may be combined in any suitable manner.

[0055] The techniques proposed and claimed herein are referenced and applied to substantial objects and specific examples of practical nature that can arguably improve the technical field and are therefore not abstract, intangible, or purely theoretical. Furthermore, if any claim appended to this specification contains one or more elements designated as “component for [implementing]…[function]” or “step for [implementing]…[function]”, such elements are intended to be interpreted pursuant to 35 USC 112(f). However, for any claim containing elements designated in any other manner, such elements are intended not to be interpreted pursuant to 35 USC 112(f).

Claims

1. A video stream management system comprising: a video controller configured to render video in real-time; and a display communicatively coupled to the video controller and configured to display a primary video feed comprising the real-time rendered video, wherein the video controller, the display, or a combination thereof is configured to embed a pattern of pixels in the primary video feed, and wherein the video stream management system is configured to monitor one or more displayed images on the display to identify an error in the primary video feed, wherein the error is based on a mismatch between an expected pixel location, pixel luminance, and / or pixel color of at least one pixel of the embedded pattern of pixels and a corresponding displayed location, pixel luminance, and / or pixel color of the at least one pixel displayed on the display; wherein the embedded pattern of pixels is dynamic and changes on a frame-by-frame basis to reduce alignment of images of the primary video feed with the embedded pattern of pixels.

2. The system of claim 1, wherein, the display is communicatively coupled to a source of an alternative video feed, and the display is configured to switch to displaying the alternative video feed in response to identifying the error in the primary video feed.

3. The system of claim 2, wherein, the primary video feed comprises an interactive video stream, and the alternative video feed comprises a non-interactive video stream.

4. The system of claim 1, wherein, the error is based on an identified mismatch between the embedded pattern of pixels and an image of the one or more displayed images on the display.

5. The system of claim 1, wherein, the video stream management system does not detect an error when the embedded pattern of pixels is identified in the one or more displayed images on the display.

6. The system of claim 1, wherein, the embedded pattern of pixels is associated with a subset of a total number of pixels of the one or more displayed images on the display.

7. The system of claim 1, wherein, the error is based on a predetermined threshold of expected pixel pattern and pixel mismatch in the one or more displayed images on the display.

8. The system of claim 1, wherein, the embedded pattern of pixels changes between the one or more displayed images forming successive frames of the primary video feed on the display.

9. The system of claim 1, wherein, the embedded pattern of pixels is static between images of the one or more displayed images forming successive frames of the primary video feed on the display.

10. A video stream management system comprising: one or more sensors configured to detect a presence of a customer; a video stream controller configured to: render video in real-time based on the detected presence of the customer to generate a primary video stream; and embed a pattern of pixels in the primary video stream to generate a video stream; and a display communicatively coupled to the video stream controller and configured to: receive the video stream; display the video stream to generate a displayed image; monitor the displayed image on the display to identify an error based on a comparison of the displayed image and the pattern of pixels; and ​ ​ generating an error signal based on identifying the error, wherein the error is based on a mismatch between an expected pixel location, pixel luminosity, and / or pixel color of at least one pixel of the embedded pixel pattern and a corresponding displayed location, pixel luminosity, and / or pixel color of the at least one pixel displayed on the display; wherein the embedded pixel pattern is dynamic and changes frame-by-frame to reduce alignment of images of the primary video stream feed with the embedded pixel pattern.

11. The system of claim 10, wherein, the error includes the displayed image missing one or more pixels of the pixel pattern or having a mismatch with one or more pixels of the pixel pattern, having one or more different color pixels relative to the pixel pattern or one or more pixels of the pixel pattern having a different location.

12. The system of claim 10, wherein, the video stream management system causes display of an alternative video based on the error signal.

13. The system of claim 12, wherein, the alternative video includes a pre-recorded video.

14. A method of managing a video feed, comprising: embedding a dynamic pixel pattern into frames of a live video feed, wherein the dynamic pixel pattern includes a first pixel pattern associated with a first frame of the live video feed and a second pixel pattern associated with a second frame of the live video feed, wherein the first pixel pattern is different than the second pixel pattern; displaying the live video feed including the dynamic pixel pattern on a display; monitoring a displayed image on the display; identifying an error in the live video feed in response to determining that the monitored displayed image includes a displayed pixel pattern that does not match the embedded dynamic pixel pattern; and switching from displaying the live video feed to displaying an alternative video feed in response to identifying a presence of the error in the live video feed; wherein the dynamic pixel pattern changes frame-by-frame to reduce alignment of images of a primary video feed with the embedded pixel pattern.

15. The method of claim 14, wherein, the alternative video feed includes a complete backup of the live video feed and the embedded dynamic pixel pattern, a new video feed different than the live video feed, or a combination thereof.

16. The method of claim 14, wherein, the live video feed is embedded with the dynamic pixel pattern and the alternative video feed is provided to the display concurrently.

17. The method of claim 14, wherein, the first frame and the second frame are embedded in successive frames of the live video feed.

18. The method of claim 14, wherein, monitoring includes evaluating individual frames of the live video feed to identify a pixel pattern of the embedded dynamic pixel pattern associated with the individual frame.

19. The method of claim 14, wherein, information regarding the embedded dynamic pixel pattern is encrypted via a key and accessible.

20. The method of claim 14, wherein identifying the error is based at least in part on a predetermined threshold of pixel mismatch between a displayed pixel pattern and the first pixel pattern during the first frame or the second pixel pattern during the second frame.

21. A video controller configured to: receiving a main video stream; and receive a dynamic pixel pattern, wherein the dynamic pixel pattern includes a changing pixel pattern for each frame of the primary video stream; merge the primary video stream and the dynamic pixel pattern into a single video stream; sending the single video stream for display on a display; and monitoring the display for an error on the display, the error based at least in part on a mismatch associated with the dynamic pixel pattern; wherein the mismatch associated with the dynamic pixel pattern includes a mismatch between an expected pixel location, pixel illumination, and / or pixel color of at least one pixel of the dynamic pixel pattern in the single video stream and a corresponding displayed location, pixel illumination, and / or pixel color of the at least one pixel displayed on the display; wherein the dynamic pixel pattern changes frame-by-frame to reduce an alignment of an image of a primary video feed with the dynamic pixel pattern.

22. The video controller of claim 21, wherein, the primary video stream includes a live video that includes an interactive element that changes based on a presence of a customer.

23. The video controller of claim 21, wherein, the video controller is configured to identify the error based on the mismatch; and automatically switch to sending a secondary video stream to the display.

24. The video controller of claim 23, wherein, the secondary video stream includes a pre-recorded video stream.

25. The video controller of claim 23, wherein, the secondary video stream includes a live rendered video stream embedded with a dynamic pixel pattern as the single video stream.

26. The video controller of claim 21, wherein, the dynamic pixel pattern is encrypted using a corresponding key and is accessible.

27. The video controller of claim 26, wherein, the corresponding key is based at least in part on metadata associated with the video controller or the dynamic pixel pattern, a network connection, a memory location, or any combination thereof.

28. The video controller of claim 21, wherein, the error includes a dialog box on a video frame, a stuck video frame, or a combination thereof.

29. The video controller of claim 21, wherein, a pattern of the dynamic pixel pattern is based at least in part on an element displayed during each frame.

30. The video controller of claim 29, wherein, the pattern includes contiguous pixels or non-contiguous pixels based at least in part on the element.

31. A tangible, non-transitory, machine-readable medium comprising machine-readable instructions to: embedding a pixel pattern into a frame of a video feed, wherein, the pixel pattern includes a first pixel pattern associated with a first frame of the video feed and a second pixel pattern associated with a second frame of the video feed; display the video feed including the pixel pattern using a display; monitor a displayed image on the display; and identify an error in the video feed in response to determining that the monitored displayed image includes a displayed pixel pattern that does not match the embedded pixel pattern; wherein the embedded pixel pattern is dynamic and changes frame-by-frame to reduce an alignment of an image of the video feed with the embedded pixel pattern.

32. The tangible, non-transitory, machine-readable medium of claim 31 comprising machine-readable instructions to: switch from displaying the video feed to displaying an alternative video feed in response to identifying the error in the video feed.

33. The tangible, non-transitory, machine-readable medium of claim 32, wherein, the pixel pattern is associated with a subset of a total number of pixels of the first frame and the second frame.

Images