Systems, methods, and devices for buffer handshake in video streaming

Abstract

To provide systems, methods, and devices for a buffer handshake in video streaming.SOLUTION: Methods include the steps of: receiving video data from a video source, the video data comprising at least one video frame; and determining a plurality of store parameters associated with a store operation and a plurality of fetch parameters associated with a fetch operation for a portion of the video data, wherein the plurality of store parameters and the plurality of fetch parameters identify whether a fetch unit or a store unit should be stalled. The methods further include the steps of: implementing a store operation in a buffer for a designated number of lines of the video data based on the plurality of store parameters; and implementing a fetch operation from the buffer for the designated number of lines of the video data based on the plurality of fetch parameters.SELECTED DRAWING: Figure 4

Description

Technical Field

[0001] The present disclosure generally relates to video and image processing, and more specifically to techniques used to stream video data.

Background Art

[0002] Computing devices and systems can include hardware and software configured to execute one or more software applications and display information related to such software applications on a display device. For example, a computer system can include a host processor and a hard drive used to execute a software application, and can display data related to the software application on a monitor of the computer system. Such data can be video data streamed from a video source. Thus, the components of a computing device can retrieve video data and process such video data for display on a target device. However, in the prior art, there are still limitations in the ability of such prior art to efficiently use resources such as internal memory when streaming such video data.

Brief Description of the Drawings

[0003] [Figure 1] FIG. 1 shows an example of a system for synchronizing a video source and a video destination configured according to some embodiments. [Figure 2] FIG. 2 shows another example of a system for synchronizing a video source and a video destination configured according to some embodiments. [Figure 3] FIG. 3 shows a flowchart of an example of a method for synchronizing a video source and a video destination implemented according to some embodiments. [Figure 4] This figure shows a flowchart of one embodiment of a method for synchronizing a video source and a video destination, implemented according to several embodiments. [Figure 5] This figure shows a flowchart of yet another embodiment of a method for synchronizing a video source and a video destination, implemented according to several embodiments. [Figure 6] This figure shows one embodiment of a diagram of lines in a buffer, implemented according to several embodiments. [Figure 7] This figure shows one embodiment of the components of a processing apparatus configured according to several embodiments. [Modes for carrying out the invention]

[0004] The following explanation includes numerous specific details to ensure a complete understanding of the presented concepts. These concepts may be implemented without using some or all of these specific details. As an alternative, well-known process operations have not been described in detail to avoid unnecessarily obscuring the concepts being discussed. Some concepts will be explained in conjunction with specific embodiments, but it should be understood that these embodiments are not intended to be limiting.

[0005] A computer system can be configured to render and display graphic data on one or more target display devices. Rendering and displaying such graphic data may involve performing one or more transformation operations on the graphic data itself. For example, the rendering process may include resizing or rescaling an image. Furthermore, rendering and displaying such data may include various fetch and store operations, in which an intermediate component between the video data source and sink may trigger the transmission of video data using fetch and store commands. As will be discussed in more detail later, the video data source may be a software application or another source of streaming video, and the video data sink may be a target display device.

[0006] In some computer systems, the video source and video sink are not directly connected by a ready / enable handshake signal. Instead, the source stream has a store unit that writes pixel data to a memory device such as random access memory (RAM), and a display stream fetch unit that reads that data from the same RAM location. Therefore, a full-size frame buffer may be implemented in RAM. Such systems still have limitations, because the amount of memory used can be extremely large, as it needs to store multiple full-size frames in memory that may also be used for other purposes. As a result, a full-size frame buffer uses a considerable amount of system resources and also causes a considerable amount of overhead in terms of the bandwidth used to transmit such frames to and from memory. Moreover, the store and fetch operations from memory are not synchronized. Therefore, in conventional systems, streaming interruptions occur due to the lack of synchronization between the source and sink.

[0007] Embodiments disclosed herein provide a buffer and controller implementation configured to synchronize fetch and store operations related to a portion of a video frame. Thus, as will be described in more detail later, by using a portion of a frame, a relatively small buffer smaller than a single video frame can be used, and therefore the amount of memory and associated bandwidth used for such fetch and store operations can be significantly reduced. Moreover, the controller can selectively stall either the fetch or store operation for a specified amount of time to ensure that both operations remain synchronized. In this way, the controller is configured to perform a "handshake" between the video source and the video sink to ensure a consistent and uninterrupted streaming of video data.

[0008] Figure 1 shows one embodiment of a system for synchronizing a video source and a video destination, configured according to several embodiments. As will be described in more detail later, a system like system 100 can be implemented to synchronize fetch and store operations associated with a video source and a video sink. More specifically, system 100 may include one or more components configured to perform a "handshake" between the video source and the video sink and to ensure that the video data stream is continuous, while also ensuring that the use of system resources is minimized.

[0009] Accordingly, according to various embodiments, the system 100 includes a source 102 configured to supply data that will ultimately be displayed on a display device, as will be described in more detail later. More specifically, the source 102 can be configured to run one or more software applications configured to generate graphic data that should be displayed on a display device, for example, which can be included in a user interface generated for a particular software application. Further details regarding such software applications and graphic data will be described in more detail later with reference to Figure 2.

[0010] According to various embodiments, the source 102 is communicatively coupled to a memory device such as memory 104. According to various embodiments, memory 104 is a memory device such as random access memory (RAM). According to one embodiment, memory 104 is a video random access memory (VRAM) device. Furthermore, memory 104 can be communicatively coupled to source 102, and memory 104 may also include one or more buffers configured to store graphic data from source 102. Thus, buffers can be used to store images or frames contained in the graphic data.

[0011] In addition, system 100 includes a processor 106, which can be a graphics processing unit (GPU) configured to perform one or more rendering operations on graphic data. Thus, the processor 106 can receive graphic data from source 102 and perform one or more graphics rendering operations on this graphic data. For example, the graphic data may contain various video frames, and each frame can be processed by the processor 106. More specifically, one or more pixel mapping or transformation operations can be performed on each frame of the video data. The rendered frames can then be supplied as outputs for transmission to a target display device, such as display 110, which will be described in more detail later.

[0012] As is obvious, while the various embodiments disclosed herein describe systems and devices used with a graphics processing unit, other types of graphics devices can be used in the same way. For example, rendering of video data may be performed by a camera's video capture controller, or by a JPEG decoder used in relation to picture and image data. Therefore, the embodiments disclosed herein are not limited to graphics processing units or video data.

[0013] The system 100 further includes a processor 108, which is configured to synchronize fetch and store operations between the source 102 and the display 110, and can also transmit these operations via the processor 106. More specifically, the processor 108 can be configured to selectively stall the fetch and store units so as to ensure that the different operations are not separated and that the synchronization does not become so disruptive that it interrupts the streaming of video data. For example, the processor 108 can stall store operations related to the source 102 if the buffer is full, and can stall fetch operations related to the display 110 if a specified number of lines are not free in the buffer. Thus, as will be described in more detail later with reference to Figure 2, the processor 108 can include various components such as buffers and controllers configured to synchronize fetch and store operations related to the transmission of video from the source to the sink.

[0014] The system 100 further includes a display 110 configured to display the results of a rendering operation. For this reason, the display 110 can be a display device such as a liquid crystal display (LCD) screen. As will be described in more detail later, the display 110 may include various components configured to receive and display rendered graphic data.

[0015] Figure 2 shows another embodiment of a system for synchronizing a video source and a video destination, configured according to several embodiments. As will be discussed in more detail later, a system like system 200 can be implemented to synchronize fetch and store operations associated with a video source and a video sink. More specifically, system 200 may include a controller and buffers specially configured to ensure that the video data stream is continuous while also ensuring that the use of system resources is minimized.

[0016] Therefore, as previously stated, the system 200 includes a video source, such as a source 202, configured to supply data that will ultimately be displayed on a display device, as will be described in more detail later. More specifically, source 202 can be configured to run one or more software applications configured to generate graphic data to be displayed on a display device, for example, so that it can be included in a user interface generated for a particular software application. Thus, source 202 may include application software, such as application software 204, which is configured to generate one or more commands related to the transmission and display of video data, and similarly to generate graphic data representing the video being transmitted. According to various embodiments, source 202 further includes a graphics driver 206, which is configured to translate such software commands into hardware-specific commands for the system 200.

[0017] According to various embodiments, the source 202 is communicatively coupled to a memory device such as memory 208. According to various embodiments, memory 208 is a memory device such as random access memory (RAM). According to one embodiment, memory 208 is a video random access memory (VRAM) device. Furthermore, memory 208 can be communicatively coupled to the source 202, and memory 208 may also include one or more buffers configured to store graphic data from the source 202. For example, such buffers may be frame buffers specifically configured to store images or frames contained in the graphic data.

[0018] According to various embodiments, the memory 208 includes a buffer 216 configured to store lines of video data being transmitted from system components such as a graphics processing unit 210 and a display 224. According to various embodiments, the buffer 216 is implemented using a portion of the memory 208. Thus, the buffer 216 can be implemented by reserving a portion of the RAM or VRAM contained in the memory 208. According to some embodiments, the buffer 216 can be implemented using a separate memory device. Therefore, the buffer 216 can be implemented in a memory device different from the memory 208, or in a dedicated buffer memory element, which may be included in the memory 208 or implemented separately from the memory 208. Furthermore, the buffer 216 can be configured as a ring buffer that circulates a portion or subset of lines in a video frame when fetch and store operations are performed.

[0019] According to various embodiments, the size of the buffer 216 can be set and determined based on one or more configuration parameters. For example, an entity such as a user or administrator can set the buffer size during a configuration operation. According to some embodiments, the size of the buffer 216 can be determined by a controller 222, which will be described in more detail later, based on one or more parameters of the application software 204, such as the output resolution of the video data generated by the application software 204. As will be described in more detail later, the size of the buffer 216 can be smaller than a single video frame contained in the video data. More specifically, the buffer 216 can have fewer lines than a single frame of video data. In this way, the size of the buffer 216 can be made significantly smaller than the size of the video frame, and system resources can be reduced compared to the frame buffer.

[0020] In addition, the system 200 includes a graphics processing unit 210 configured to perform one or more rendering operations on the graphic data. Therefore, as previously stated, the graphics processing unit 210 can receive graphic data from the source 202 and perform one or more graphics rendering operations on this graphic data. For this purpose, the graphics processing unit 210 may include a processor and dedicated memory specially configured to perform the rendering operations, such as the one or more pixel mapping or transformation operations described above. As also previously stated, the rendered frames can be supplied as output for transmission to a target display device.

[0021] According to various embodiments, the graphics processing unit 210 includes a store unit 218 configured to store graphics data based on store commands generated, for example, based on a signal reception source 202. As previously mentioned, the graphics data to be stored can be stored in a storage location such as a buffer 216 and may include portions of graphics data, such as specific lines of frames of video data. Thus, the store unit 218 can be configured to identify and store such lines of video data in response to the receipt of a command.

[0022] According to various embodiments, system 200 additionally includes a processing device 212, which is configured to synchronize fetch operations and store operations between source 202 and display 224, similar to what was described above. According to various embodiments, the processing device 212 includes a controller 222 configured to control the operations of fetch unit 214 and store unit 218 and manage the interaction therebetween. More specifically, the controller 222 is configured to instruct the fetch unit 214 and the store unit 218 to perform fetch operations and store operations, and is also configured to selectively stall the fetch unit 214 and / or the store unit 218 so that different operations are not desynchronized and video data streaming is not interrupted. More specifically, the controller 222 can be configured to determine parameters that identify conditions when store operations related to the store unit 218 should be stalled, and the controller 222 can also determine parameters that identify conditions when fetch operations related to the fetch unit 214 should be stalled. Further details regarding the calculation and determination of such parameters will be described in more detail later, with reference to FIGS. 3-6. Thus, the controller 222 can include one or more processors configured to determine when fetch operations and store operations should be performed by the fetch unit 214 and the store unit 218, and the controller 222 can be configured to generate signals to perform such operations.

[0023] According to various embodiments, the controller 222 can also be configured to receive enable instructions and generate enable signals for the fetch and store units. Thus, the controller 222 can be configured to selectively enable and disable the synchronization techniques disclosed herein based on one or more detected conditions, such as a set of bits or the reception of an input signal. According to one embodiment, such an input signal can be received from a user or administrator during a setup operation.

[0024] As previously mentioned, the system 200 further includes a display 224 configured to display the results of a fetch operation performed in buffer 216. According to various embodiments, the display 224 includes a fetch unit 214, which can be configured to fetch graphic data based on fetch commands generated based on signals received from components of the display 224, such as a display controller 226, which will be described in more detail later. According to various embodiments, the graphic data to be fetched can be fetched from a storage location such as buffer 216 via a processing unit 212, and the data to be fetched may include a portion of the graphic data. More specifically, the fetch operation can be performed on a particular line of a frame of video data. Thus, the fetch unit 214 can be configured to identify and retrieve such line of video data in response to receiving a command.

[0025] As described above, display 224 can be a display device such as a liquid crystal display (LCD) screen. According to various embodiments, display 224 includes various components configured to receive the rendered graphic data and display such rendered graphic data. For example, display 224 can include a display controller 226 configured to generate a video signal that is ultimately displayed on the display device of display 224. Thus, display controller 226 can manage the operation of the display device based at least in part on the received and rendered graphic data.

[0026] It will be understood that, although the various embodiments disclosed herein have been described with respect to video data and lines of video frames, any suitable unit, portion or division of data may be used. For example, as will be described in more detail later, blocks of video data may be used as a basis for determining buffer parameters and synchronization parameters. Thus, the embodiments disclosed herein are not limited to using lines of video data.

[0027] FIG. 3 shows a flowchart of one example of a method for synchronizing a video source and a video destination implemented according to some embodiments. As will be described in more detail later, a method such as method 300 can be implemented to synchronize fetch operations and store operations related to a video source and a video sink. Thus, one or more operations of method 300 can be used to effect a "handshake" therebetween, ensuring that the video data stream is continuous while also ensuring that the use of system resources is minimized. In this way, according to method 300, continuous video streaming can be provided while also enhancing the efficiency of the resources used for such video streaming.

[0028] Method 300 can proceed to Operation 302, during which video data can be received. According to various embodiments, the video data is a video stream generated by a video source. For example, a software application can generate a video data stream to be displayed on a target display, as previously mentioned. Also as previously mentioned, the video data may include graphic data related to the user interface and / or video media. More specifically, the streamed video data may include various video frames, each frame of which may include a line of pixel data. Thus, during Operation 302, video data can be received from a system component, such as a graphics processor or other system component.

[0029] Method 300 can proceed to Operation 304, during which several synchronization parameters can be determined. As previously mentioned, the synchronization parameters are configured to synchronize fetch and store operations associated with the video source and video sink, and to perform a "handshake" between them. As will be described in more detail later, the synchronization parameters are determined and implemented by a component such as a controller, and are configured to identify the boundaries between such fetch and store operations, and to identify whether either the fetch or store operation should be stalled. Again, as will be described in more detail later, such synchronization parameters can be determined at least in part on the size of the buffer and the number of available lines in the buffer.

[0030] Method 300 can proceed to Operation 306, during which a store operation can be performed on a specified number of lines, at least in part, based on the synchronization parameters. Thus, a specified number of lines can be scanned from the received video data and stored in the buffer lines. As will be described in more detail later, the store operation can be performed according to the synchronization parameters. More specifically, the number of lines, as well as the timing adjustments of the store operation, such as whether or not the store operation should be stalled for a specified period, can be determined by the synchronization parameters.

[0031] Method 300 can proceed to Operation 308, during which a fetch operation can be performed on a specified number of lines, at least in part, based on synchronization parameters. Thus, a specified number of lines can be scanned from the buffer and sent to a video sink that can be used as a target display. As will be described in more detail later, the fetch operation can be performed according to synchronization parameters. More specifically, the number of lines, as well as the timing adjustments of the fetch operation, such as whether or not the fetch operation should be stalled for a specified period, can be determined by the synchronization parameters.

[0032] Figure 4 shows a flowchart of one embodiment of a method for synchronizing a video source and a video destination, implemented according to several embodiments. As previously described, a method like Method 400 can be implemented to synchronize fetch and store operations associated with a video source and a video sink. As will be described in more detail later, system components such as a controller can be configured to manage the execution of fetch and store operations to ensure the continuity of video data streaming. Thus, Method 400 can provide continuous video streaming while also increasing the efficiency of the resources used for such video streaming.

[0033] Method 400 can proceed to Operation 402, during which video data can be received. As previously stated, video data can be a video stream generated by a video source. For example, a software application can generate a video data stream to be displayed on a target display, as previously stated. As previously stated, video data can include graphic data related to the user interface and / or video media. More specifically, streamed video data can include various video frames, each frame can include various pixel data represented as lines of pixels.

[0034] Method 400 can proceed to Operation 404, during which several buffer parameters can be determined. According to various embodiments, the buffer parameters can identify one or more aspects of the buffer, such as the buffer size. More specifically, the buffer parameters can identify the number of lines stored in the buffer and can also maintain a mapping of such buffer lines to frame lines contained in the received video data. For example, one video frame may be received, and this video frame may have data lines corresponding to the lines of pixels in the frame. As mentioned earlier, and as will be described in more detail later with reference to Figure 6, the buffer can be configured as a ring buffer, which can contain fewer lines than the number of lines contained in the frame. Thus, the buffer lines can be mapped to frame lines as fetch and store operations proceed line by line across the entire frame. During Operation 402, system components such as a controller can query the buffer to retrieve information such as the buffer size and the mapping information described above. According to some embodiments, the controller can store and maintain the mapping information itself based on counting performed when a new frame is received, so as to be identifiable in the received video data.

[0035] Method 400 can proceed to Operation 406, during which several fetch parameters can be determined. According to various embodiments, the fetch parameters are configured to identify the currently active line for the fetch operation, as well as the point at which the fetch operation should be stalled. Thus, the fetch parameters can identify the currently active line that will be used for the next fetch operation. Furthermore, the fetch parameters can identify specific lines in the buffer at which the fetch operation should be stalled in order to wait for the store operation to complete and proceed. As will be described in more detail later with reference to Figure 5, a system component such as a controller can determine such fetch parameters by querying the buffer and performing one or more calculations.

[0036] Method 400 can proceed to Operation 408, during which several store parameters can be determined. According to various embodiments, the store parameters are configured to identify the currently active line for a store operation, as well as the point at which the store operation should be stalled. Thus, the store parameters can identify the currently active line that will be used for the next store operation. Furthermore, the store parameters can identify a specific line in the buffer at which the store operation should be stalled in order to wait for the fetch operation to complete and proceed. As will be described in more detail later with reference to Figure 5, a system component such as a controller can determine such store parameters by querying the buffer and performing one or more calculations. Thus, the controller can determine parameters used to synchronize the activity of a fetch operation and a store operation, ensuring that one remains within a specified number of lines from the other.

[0037] Method 400 can proceed to Operation 410, during which it can be determined whether the fetch operation should be stalled. Thus, system components such as a controller can determine whether the fetch operation should be stalled, at least in part, based on the fetch parameters and a comparison of various fetch parameters. More specifically, if the current fetch line is equal to the identified stop line, Method 400 can proceed to Operation 414, which will be described in more detail later. If the current fetch line is smaller than the identified stop line, Method 400 can proceed to Operation 412.

[0038] Therefore, a fetch operation can be performed during operation 412. According to some embodiments, a fetch operation can be performed for a target video display, also referred to herein as a video sink. Thus, one or more lines can be fetched and read from the buffer and supplied to the target video display. In this way, the target video display can be provided with the output of the buffer synchronized according to the implementation of the buffer and controller disclosed herein.

[0039] Method 400 can proceed to Operation 414, during which it can be determined whether the store operation should be stalled. Thus, system components such as a controller can determine whether the store operation should be stalled, at least in part, based on store parameters and a comparison of various store parameters. More specifically, if the current store line is equal to the identified stop line, Method 400 can proceed to Operation 418, which will be described in more detail later. If the current store line is smaller than the identified stop line, Method 400 can proceed to Operation 416.

[0040] Method 400 can proceed to operation 416, during which a store operation can be performed. According to some embodiments, the store operation can be performed for a video source. Thus, one or more lines can be received from the video source and stored in a buffer. In this way, video data can be received from the video source and stored in the buffer in a synchronized manner according to the implementation of the buffer and controller disclosed herein.

[0041] Method 400 can proceed to Operation 418, during which the fetch and store parameters can be updated. Thus, the number or index that identifies the currently active line for the fetch and store operations can be updated based on the activities described above. For example, if a fetch and store operation has been performed, their buffer index number and frame index number can be incremented. Then, as will be described later, once one operation has been performed, the incremented number can be used in subsequent operations.

[0042] Method 400 can proceed to Operation 420, during which it can decide whether to perform further fetch and store operations. Such a decision can be made at least in part based on line numbers and instructions to stop the video stream. For example, if neither the fetch nor the store operation has reached the last line of a frame, it can be decided that further fetch and / or store operations should be performed, and Method 400 can return to Operation 404. If both the fetch and the store operation have reached the last line of a frame and it is determined that there are no further frames, Method 400 can be terminated.

[0043] Figure 5 shows a flowchart of yet another embodiment of a method for synchronizing a video source and a video destination, implemented according to several embodiments. As previously described, a method like Method 500 can be implemented to synchronize fetch and store operations associated with a video source and a video sink. As will be described in more detail later, system components such as a controller can be configured to calculate specific parameters and buffer lines used to manage the execution of fetch and store operations for the purpose of ensuring the continuity of video data streaming. Thus, Method 500 can provide continuous video streaming while also increasing the efficiency of the resources used for such video streaming.

[0044] Method 500 can proceed to Operation 502, during which several buffer parameters can be determined. As previously mentioned, buffer parameters can identify one or more aspects of the buffer, such as the buffer size. More specifically, buffer parameters can identify the number of lines stored in the buffer and can also maintain a mapping of such buffer lines to frame lines contained in the received video data. According to one embodiment, the buffer may currently have eight lines available, and each line can be configured to store a specified number of data values. According to one embodiment, each line may store video data such as a buffer index number, an associated frame index number, and associated pixel data for that line. Thus, during Operation 502, a system component such as a controller can query the buffer to identify the buffer size and its current location within a frame.

[0045] Method 500 can proceed to Operation 504, during which a first fetch parameter can be determined. As previously described, the fetch parameter is configured to identify the currently active line for a fetch operation. Thus, the fetch parameter can identify the currently active line that will be used for the next fetch operation. According to one embodiment, the currently active line for a fetch operation can be determined by a component such as a controller by querying a buffer. According to another embodiment, the currently active line for a fetch operation can be determined by a controller using a counter that can count the fetch operations performed on the lines in the buffer.

[0046] Method 500 can proceed to operation 506, during which a first store parameter can be determined. As previously described, the store parameter is configured to identify the line currently active for a store operation. Thus, the store parameter can identify the line currently in the buffer that will be used for the next store operation. As previously described, the line currently active for a store operation can be determined by a component such as a controller by querying the buffer. According to another embodiment, the line currently active for a store operation can be determined by a controller using a counter that can count the store operations performed on the buffer line.

[0047] Method 500 can proceed to operation 506, during which a second fetch parameter can be determined. As previously mentioned, a system component such as a controller can identify a fetch stop line, which is configured to identify a point at which the fetch operation should be stopped or stalled in order to wait until the store operation is complete and proceeds to the next. According to various embodiments, a system component such as a controller can determine such a fetch stop line based at least in part on the currently active store line. Thus, according to some embodiments, the fetch stop line can be set to the currently active store line, or by specifying an offset to the currently active store line. For example, the fetch stop line can be set to the currently active store line minus 1. Obviously, the offset can be any appropriate number of lines. More specifically, the offset can be three lines to provide further separation between the fetch operation and the store operation.

[0048] Method 500 can proceed to Operation 510, during which a second store parameter can be determined. As previously mentioned, a system component such as a controller can identify a store stop line, which is configured to identify a point where the store operation should be stopped or stalled in order to wait until the fetch operation is complete and proceeds to the next step. According to various embodiments, a system component such as a controller can determine such a store stop line based at least in part on the buffer parameters described above. For example, the store stop line can be set to the last line of the buffer. According to some embodiments, the store stop line can be determined dynamically. For example, the store stop line can be set to a fetch line, and an offset such as the number of lines to be held or reserved can be incorporated. Thus, the store stop line can be set to a fetch line minus an offset value which can be the number of lines to be held or reserved. As is obvious, the fetch and store parameters described above can be stored as index numbers associated with buffer lines. Thus, these parameters can identify a specific line in the buffer in the current fetch and store operation situation.

[0049] Method 500 can proceed to Operation 512, during which a storage parameter can be determined. According to various embodiments, the storage parameter identifies a specified number of lines to be retained or reserved in the buffer after a fetch operation has been performed. For example, a specified number of three lines can be identified and reserved after the current fetch operation. In this way, a specified number of lines can be kept in memory after the fetch operation to enable resampling by the target display device. According to various embodiments, the specified number of lines identified by the storage parameter can be determined based on the resampling technique used by the target display device. According to one embodiment, a system component such as a controller can query the target display device to determine the resampling technique, or the target display device can disclose its technique to the controller. According to some embodiments, a predetermined mapping can be used to identify the number of lines based on the identified technique. Such a mapping can be determined by an entity such as a user or administrator during the initial setup process.

[0050] Figure 6 shows one embodiment of a diagram of lines in a buffer, implemented according to several embodiments. As previously mentioned, fetch and store operations related to a video source and video sink can be synchronized. As shown in Image 600, lines in a buffer that can represent the boundaries of such fetch and store operations can be identified using various fetch and store parameters, and it can be determined whether one or the other should be stalled to ensure synchronization.

[0051] More specifically, image 600 includes data fields 602 and 604 representing line numbers in the frame and buffer. According to various embodiments, the line number is an index number that system components such as the controller and buffer can use to identify such lines. As shown in Figure 6, data field 602 includes an index number for each line of the video frame being rendered and transmitted. Data field 604 includes the corresponding lines in the buffer that are mapped to those lines. As shown in Figure 6, the buffer can be a ring buffer that wraps around sequentially as the operation progresses. Thus, the line numbers in the buffer are smaller than the line numbers in the video frame, and each line in the buffer can be used multiple times in the process of transmitting a single video frame.

[0052] Image 600 further shows a first line 606 which can be a fetch line, as previously described with reference to at least Figure 5. Thus, the first line 606 can identify the currently active line related to the fetch operation. In addition, Image 600 also shows a second line 608 which is a store line, as previously described with reference to at least Figure 5. Thus, the second line 608 can identify the currently active line related to the store operation. According to various embodiments, the second line 608 is also identified as a stop line for the fetch operation, as previously described with reference to at least Figure 5. Therefore, the fetch unit will stop performing the fetch operation when it reaches the second line 608.

[0053] Image 600 further shows a third line 610, which can be a stop line for store operations, as previously described with reference to at least Figure 5. Therefore, the store unit will stop performing store operations once it reaches the third line 610. As previously described, system components such as a controller can determine the first line 606, the second line 608, and the third line 610. In addition to these, Image 600 also shows a line 612, which can be reserved for resampling, as previously described. Therefore, line 612 can be maintained relative to the first line 606 to enable the resampling technique used by the target display device. According to various embodiments, the final line 614 represents the end of a frame.

[0054] As is obvious, the identified lines described above can be updated when a fetch operation and / or store operation is completed. For example, fetch lines and store lines can be updated incrementally, and their associated stop lines can be recalculated as well. As is obvious, store operations are associated with video source activity and fetch operations are associated with video sync activity, so such activities of the video source and video sync may not be synchronized. However, the controller and buffers synchronize their respective associated fetch and store operations by calculating and using stop lines to selectively stall the fetch and store units at the appropriate times, thereby ensuring continuity in data streaming.

[0055] Figure 7 shows one embodiment of components used to implement a processing unit configured according to several embodiments. Therefore, a system such as the system 700 described below can be used to implement the controller and / or processing unit components mentioned above. According to various embodiments, system 700 includes a processor 701, memory 703, storage device 705, interface 711 (e.g., a PCI bus or other interconnect fabric configured to connect to other peripheral devices), and bus 715. According to some embodiments, the processor 701 can be a host processor or a central processing unit (CPU), and the memory 703 can be part of a CPU subsystem. The storage device 705 can be part of a GPU subsystem, and the storage device 705 can include VRAM. Thus, the components of system 700 can operate as various components mentioned above, such as controllers and buffers. Although one specific configuration is described, various alternative configurations are possible. The processor 701 can perform operations as described herein. Instructions for performing such operations can be implemented in memory 703 on one or more non-temporary computer-readable media or any other storage device. Various specially configured devices can also be used instead of, or in addition to, the processor 701. As previously mentioned with reference to Figures 1 and 2, the interface 711 can be configured to send data values ​​to and receive data values ​​from other system components, such as a display.

[0056] While the concepts described above have been explained in some detail to clarify understanding, it is obvious that some modifications and alterations can be made within the scope of the attached claims. It should be noted that there are numerous alternative methods for implementing the processes, systems, and apparatus. Therefore, this embodiment should be considered illustrative and not limiting.

Claims

1. Receiving video data including at least one video frame from a video source; Determining, using one or more processors, a plurality of store parameters related to a store operation and a plurality of fetch parameters related to a fetch operation for a portion of the video data, wherein the plurality of store parameters identify an active store line and a store stop line that defines when a store unit of a source stream should be stalled, and the plurality of fetch parameters identify an active fetch line and a fetch stop line that defines when a fetch unit of a display stream should be stalled, the stop line defines a boundary between the store operation and the fetch operation and ensures synchronization between the source stream and the display stream, and the fetch stop line is determined based on the active store line; Performing the store operation to the buffer for a specified number of lines of the video data based on the plurality of store parameters using the one or more processors; Performing the fetch operation from the buffer for the specified number of lines of the video data based on the plurality of fetch parameters using the one or more processors; A method comprising.

2. The fetch stop line is determined to be the same as the active store line. The method according to claim 1.

3. The fetch stop line is determined to have an offset specified from the active store line. The method according to claim 1.

4. The plurality of store parameters and the plurality of fetch parameters comprise index numbers identifying buffer lines of a ring buffer. The method according to claim 1.

5. The store stop line is determined based on the size of the buffer. The method according to claim 4.

6. The video data is received from a graphics processing unit, and the store operation is associated with the graphics processing unit. The method according to claim 1.

7. The fetch operation is associated with a target display device and is the method according to claim 6.

8. The size of the buffer is smaller than the size of the at least one video frame, and is the method according to claim 1.

9. The number of lines included in the buffer is configurable and determined based on user input, and is the method according to claim 1.

10. A buffer configured to store a specified number of lines of one video frame included in video data, and a controller including one or more processors, and the apparatus is provided with: The one or more processors: determine a plurality of store parameters related to a store operation and a plurality of fetch parameters related to a fetch operation for the specified number of lines, the plurality of store parameters identify an active store line currently and a store stop line that defines when a store unit of a source stream should be stalled, the plurality of fetch parameters identify an active fetch line currently and a fetch stop line that defines when a fetch unit of a display stream should be stalled, the stop line defines a boundary between the store operation and the fetch operation, ensures synchronization between the source stream and the display stream, and the fetch stop line is determined based on the currently active store line; perform the store operation on the buffer for at least a portion of the specified number of lines based on the plurality of store parameters; perform the fetch operation from the buffer for at least a portion of the specified number of lines based on the plurality of fetch parameters; Apparatus.

11. The fetch stop line is determined to be the same as the currently active store line, and is the apparatus according to claim 10.

12. The store stop line is determined based on the size of the buffer, and is the apparatus according to claim 11.

13. ​ ​ ​ ​ ​ ​ ​ ​ The video data is received from a graphics processing unit, the store operation is associated with the graphics processing unit, and the fetch operation is associated with a target display device. The apparatus according to claim 10.

14. The size of the buffer is smaller than the size of the video frame. The apparatus according to claim 10.

15. A host processor and memory configured to execute a software application and a graphics driver, A buffer configured to store a specified number of lines of one video frame included in the video data, A controller, A system including: The controller, Determines a plurality of store parameters related to a store operation and a plurality of fetch parameters related to a fetch operation for the specified number of lines, the plurality of store parameters identify a currently active store line and a store stop line that defines when a store unit of a source stream should be stalled, the plurality of fetch parameters identify a currently active fetch line and a fetch stop line that defines when a fetch unit of a display stream should be stalled, the stop line defines a boundary between the store operation and the fetch operation and ensures synchronization between the source stream and the display stream, and the fetch stop line is determined based on the currently active store line. Performs the store operation to the buffer for at least a portion of the specified number of lines based on the plurality of store parameters. Performs the fetch operation from the buffer for at least a portion of the specified number of lines based on the plurality of fetch parameters. Is configured as follows, The system further includes a display device configured to display the result of the fetch operation. System.

16. The fetch stop line is determined to be the same as the currently active store line. The system according to claim 15.

17. The store stop line is determined based on the size of the buffer, The system according to claim 16.

18. The video data is received from a graphics processing unit, the store operation is associated with the graphics processing unit, and the fetch operation is associated with a target display device. The system according to claim 15.

19. The buffer is coupled between the graphics processing unit and the display device. The system according to claim 18.

20. The size of the buffer is smaller than the size of the video frame. The system according to claim 15.

Images