Frame rate conversion

Abstract

The disclosed method includes accessing video content encoded at a specified frame rate and determining a refresh rate of an electronic display on which the video content is to be presented. Next, the method includes specifying a time interval of the video content during which a frame rate conversion is to be applied to synchronize the frame rate of the video content with the refresh rate of the electronic display. The method also includes presenting the specified time interval of the video content on the electronic display. The interval is played back at this adjusted speed using the original video frames and a plurality of frames. The presentation may also adjust the playback speed of a second portion of the interval. The interval is played back at the adjusted speed using the original frames and a different number of frames. Various other methods, systems, and computer-readable media are also disclosed.

Description

[0001] Cross-references

[0002] This application claims priority to U.S. Non-provisional Application No. 16 / 536,302, filed on August 8, 2019, entitled “FRAME RATE CONVERSION,” the entire contents of which are incorporated herein by reference. Background Art

[0003] Digital video content (including movies, television, online streaming video, and other content) is typically encoded using one of several different encoding formats. Most commonly, video content is encoded using a Moving Picture Experts Group (MPEG) standard, such as MPEG-4. This encoding specifies the frame rate at which the video is presented. For example, film footage is typically encoded at 24 frames per second. This video content is then sent to an electronic display for presentation to a user or group of users. An electronic display (e.g., a liquid crystal display (LCD), an organic light-emitting diode (OLED) display, or some other type of display) typically presents content at a specified refresh rate.

[0004] For example, many displays display content at 50 Hz, 60 Hz, 120 Hz, or even 240 Hz. Thus, the electronic display generates a new image on the display 50, 60, 120, or 240 times per second. Because the video content sent to the display is encoded at a lower frame rate (e.g., 24 frames per second), the display can redraw the same frame multiple times per second. When redrawing these frames multiple times per second, the number of frames per second (e.g., 24) does not typically divide cleanly into the refresh rate of the display, such as 50 Hz, 60 Hz, etc. Therefore, additional video frames or portions of frames are copied during playback to bring the frame rate of the video content into line with the refresh rate of the electronic display. However, current copying and synchronization methods are often unstable and can result in noticeable artifacts that degrade the user's viewing experience. Summary of the Invention

[0005] As will be described in more detail below, the present invention describes a system and method for aligning the frame rate of a video with the refresh rate of an electronic display. The method includes accessing video content encoded at a specified frame rate. The method also includes determining the refresh rate of the electronic display on which the video content is to be presented. The method also includes specifying a time interval of the video content within which a frame rate conversion is to be applied to synchronize the frame rate of the video content with the refresh rate of the electronic display. The method also includes presenting the specified time interval of the video content on the electronic display. The presenting includes: adjusting the playback speed of a first portion of the specified interval of the video content so that the first portion of the specified interval is played back using the original video frames and a first frame copy sequence; and adjusting the playback speed of a subsequent second portion of the specified interval of the video content so that the second portion of the specified interval is played back using the original video frames and a different second frame copy sequence.

[0006] In some examples, the different second frame copy sequences include at least a minimum threshold number of copied frames. In some examples, adjusting the playback speed of the first portion of the specified interval of the video content includes increasing the playback speed of the first portion of the specified interval of the video content. In some examples, adjusting the playback speed of the second portion of the specified interval of the video content includes decreasing the playback speed of the second portion of the specified interval of the video content.

[0007] In some examples, the playback speed is adjusted to within a specified maximum amount that speeds up or slows down the playback speed. In some examples, adjusting the playback speed includes changing a presentation timestamp associated with the video content. In some examples, the specified time interval is greater than 1 second. In some examples, the specified time interval is at least 3 seconds.

[0008] In some examples, the playback speed is adjusted based on the type of video frame being presented. In some examples, intervals of video content that include one or more first-order frame replications are automatically played back at a specified playback speed. In some examples, intervals of video content that include one or more second-order frame replications are automatically played back at a specified playback speed. In some examples, the number of replicated frames is repeated at a specified interval.

[0009] The corresponding system includes at least one physical processor and a physical memory, the physical memory including computer-executable instructions that, when executed by the physical processor, cause the physical processor to access a portion of video content, wherein the video content is encoded at a specified frame rate. The physical processor then determines a refresh rate for an electronic display on which the video content is to be presented. The physical processor also specifies a time interval of the video content within which a frame rate conversion is to be applied to synchronize the frame rate of the video content with the refresh rate of the electronic display. The physical processor also presents the specified time interval of the video content on the electronic display. The presentation includes: adjusting the playback speed of a first portion of the specified interval of the video content so that the first portion of the specified interval is played back using the original video frames and a first frame copy sequence; and adjusting the playback speed of a subsequent second portion of the specified interval of the video content so that the second portion of the specified interval is played back using the original video frames and a different second frame copy sequence.

[0010] In some examples, the playback speed of each portion of a specified interval of video content is stored in a lookup table. In some examples, the specified interval of video content is played back on the electronic display according to the playback information indicated in the lookup table. In some examples, the lookup table includes one or more delta values that are applied during presentation of the specified interval of video content on the electronic display to adjust the presentation timestamp of each frame within the specified interval of video content.

[0011] In some examples, the system further tests the presentation of the video content on the electronic display for a specified time interval to verify a specified quality metric regarding the playback of the video content. In some examples, the frame rate of the video content is 24 frames per second and the refresh rate of the electronic display is 50 Hz. In some examples, the frame rate of the video content is 24 frames per second and the refresh rate of the electronic display is 60 Hz.

[0012] In some examples, the above methods are encoded as computer-readable instructions on a computer-readable medium. For example, the computer-readable medium may include one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to access a portion of video content, the video content being encoded at a specified frame rate; determine a refresh rate of an electronic display on which the video content is to be presented; specify a time interval of the video content during which a frame rate conversion is to be applied to synchronize the frame rate of the video content with the refresh rate of the electronic display; and present the specified time interval of the video content on the electronic display, wherein presenting includes: adjusting a playback speed of a first portion of the specified interval of the video content so that the first portion of the specified interval is played back using the original video frames and a first frame copy sequence; and adjusting a playback speed of a subsequent second portion of the specified interval of the video content so that the second portion of the specified interval is played back using the original video frames and a different second frame copy sequence.

[0013] According to the general principles described herein, the features of any embodiments described herein can be used in combination with each other. These and other embodiments, features and advantages will be more fully understood when the following detailed description is read in conjunction with the accompanying drawings and claims. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The accompanying drawings illustrate several exemplary embodiments and are a part of the specification. Together with the following description, these drawings illustrate and explain various principles of the present invention.

[0015] Figure 1 is a block diagram of an exemplary content distribution ecosystem.

[0016] Figure 2 yes Figure 1 A block diagram of an exemplary distribution infrastructure within a content distribution ecosystem is shown.

[0017] Figure 3 yes Figure 1 A block diagram of an exemplary content player within a content distribution ecosystem is shown.

[0018] Figure 4 A computing architecture is shown in which the embodiments described herein operate.

[0019] Figure 5 A flow chart illustrating an exemplary method for aligning the frame rate of a video with the refresh rate of an electronic display is shown.

[0020] Figure 6 An embodiment is shown with a time interval of two different portions of video content.

[0021] Figure 7An embodiment of converting video frames of different frame rates to different display rates is shown.

[0022] Figure 8 A graph showing different frame rate conversions and different types of duplicated frames as well as a frame rate conversion quality metric is shown.

[0023] Figure 9 An embodiment with original video frames and copied video frames is shown.

[0024] Figure 10A An embodiment is shown in which frame rate conversion from 24 frames per second to 60 frames per second is applied.

[0025] Figure 10B An embodiment is shown in which frame rate conversion from 24 frames per second to 50 frames per second is applied.

[0026] Figure 10C An embodiment is shown where frame rate conversion is applied from 24 frames per second to 50 frames per second over a 3 second window.

[0027] Figure 11A Graphs with different time intervals, number of frame copies, sequences, and playback speeds are shown.

[0028] Figure 11B An embodiment is shown for playing back a plurality of copied video frames at a specified playback speed.

[0029] Figure 11C An embodiment is shown in which presentation timestamps are adjusted in conjunction with frame rate conversion.

[0030] Figure 12 An embodiment is shown in which audio and video synchronization is corrected to produce the desired audio / video output.

[0031] Throughout the drawings, like reference characters and descriptions indicate similar, but not necessarily identical, elements. Although the exemplary embodiments described herein are susceptible to various modifications and alternative forms, specific embodiments have been shown by way of example in the drawings and will be described in detail herein. However, the exemplary embodiments described herein are not limited to the particular forms disclosed. On the contrary, the present invention covers all modifications, equivalents, and alternatives falling within the scope of the appended claims. Specific embodiments

[0032] The present invention generally relates to aligning the frame rate of a video with the refresh rate of an electronic display. As described above, conventional systems attempt to align the frame rate of a video with the refresh rate of a display by simply adding duplicate frames or portions of duplicate frames. For example, when converting from 24 frames per second (fps) to 60 Hz (commonly used in displays in the United States and Japan), conventional systems will duplicate all 24 frames at once, for a total of 48 frames. These conventional systems will then create an additional 12 frames, repeated twice, for a total of 60 frames. These 60 frames are then displayed on the electronic display, with each frame lasting 1 / 60 of a second. Subsequent frames are also duplicated and displayed in this manner for the length of the film.

[0033] If a film shot at 24fps is converted to 50Hz (as is common in displays in Europe and South America), 24 of the original frames are duplicated to create 48 frames, with two of those frames duplicated a second time, for a total of 50 frames. These 50 frames are each displayed on the display for 1 / 50 of a second. In other implementations, the original 24 frames are duplicated once, while one frame is duplicated another time, which is also duplicated a second time. This often results in noticeable jitter and other user-visible display artifacts. It should also be noted that in these traditional systems, every frame plays back at the same rate. Regardless of how the frame duplication occurs, the video content plays back at a steady, unchanging rate throughout the entire movie or TV show.

[0034] Other traditional implementations attempt to generate frames that are not direct duplicates. For example, a motion interpolation system looks at a selected frame and subsequent frames and uses a motion estimation / compensation (MEMC) algorithm to analyze the differences in each frame. Such a system then calculates an intermediate frame that takes the initial positions of pixels in the first frame and transforms the pixels to their final positions in the next frame. This intermediate frame thus represents the "average" pixel values of the selected frame and the subsequent frame. However, these calculations require relatively large amounts of computing power and often leave noticeable artifacts in the picture that users can notice. As a result, users often turn off such motion estimation systems on their televisions or monitors.

[0035] Some video content providers also choose to re-encode their entire library for different markets. For example, videos A, B, and C provided for consumption (e.g., on a streaming platform) may be encoded at 24fps to facilitate conversion to a 60Hz display. However, because 24fps cannot be fully converted to 50Hz, these same videos A, B, and C provided in Europe are re-encoded at 25fps to facilitate conversion to 50Hz. In this case, the frame conversion can simply copy each original frame once, for a total of 50fps, to be displayed on a 50Hz display at a 1:1 ratio. However, for video content providers with very large libraries, re-encoding the entire library for different regions with different display refresh rates is not feasible.

[0036] Thus, the embodiments described herein can provide video content to electronic displays at a rate that provides a smooth and clear picture to the user, without requiring the video stream provider to re-encode its library or introduce motion estimation. In the embodiments described herein, the playback speed of certain frames or groups of frames is sped up or slowed down by a fraction of a second to accommodate more or fewer frames over a variable display interval. These differences in playback speed are imperceptible to the user and allow for smooth playback of video content regardless of its native frame rate and the refresh rate of the display displaying the content.

[0037] In some cases, a system for aligning the frame rate of a video with the refresh rate of an electronic display may access video content encoded at a specified frame rate. The system also determines the refresh rate of the electronic display on which the video content is to be displayed. The system then specifies time intervals for the video content during which frame rate conversion is to be applied to synchronize the frame rate of the video content with the refresh rate of the electronic display. The system further displays the time intervals of the video content on the electronic display. Rendering involves adjusting the playback speed of certain specified intervals of the video content so that portions of the intervals are played back using the original video frames and a frame-replicated sequence. Rendering also involves adjusting the playback speed of different portions of the video content interval so that portions of the intervals are played back using the original video frames and a different frame-replicated sequence.

[0038] Another frame copy sequence includes a minimum number of copied frames. In some cases, adjusting the playback speed of the video content interval includes increasing the playback speed of a first portion of the video content interval. Adjusting the playback speed of another portion of the video content interval includes decreasing the playback speed of that portion of the video content interval. The playback speed is adjusted within a maximum range of speeding up or slowing down the playback speed. In some cases, adjusting the playback speed includes changing a presentation timestamp associated with the video content. The specified time interval is greater than 1 second, and in some cases is at least 3 seconds.

[0039] In some cases, the playback speed is adjusted based on the type of video frame being presented. Intervals of video content that include first-order frame replication will automatically play back at a specified playback speed. Intervals of video content that include second-order frame replication will automatically play back at a specified playback speed. Repeated frames will repeat at a specified interval. In some cases, the playback speed for each portion of the video content interval is stored in a lookup table. The interval of video content is played back on the electronic display based on the playback information indicated in the lookup table. The lookup table includes an incremental value to be applied during the time interval of video content being presented on the electronic display to adjust the presentation timestamp of each frame within the video content interval.

[0040] In some cases, the presentation of time intervals of video content is tested on an electronic display to verify quality metrics associated with the playback of the video content. The frame rate of the video content varies based on the implementation. In some cases, the frame rate of the video content is 24 frames per second and the refresh rate of the electronic display is 50 Hz. In other cases, the frame rate of the video content is 24 frames per second and the refresh rate of the electronic display is 60 Hz. Figure 1-3 FIG. 4 illustrates these embodiments in more detail, which introduces a streaming media player and a streaming media environment. Figure 4 and Figure 5 Describes a computational architecture and method for computing frame transformations, and Figure 6-12 Different implementations and alternative embodiments for performing frame conversion are described.

[0041] The following will refer to Figure 1 A detailed description of an exemplary ecosystem for adaptive streaming of multimedia content is provided. Figure 2 and Figure 3 The corresponding discussions outline an exemplary distribution infrastructure and an exemplary content player, respectively. Figure 4 A detailed description of corresponding computer-implemented methods for adaptive streaming of multimedia content is provided.

[0042] Figure 1 1 is a block diagram of a content distribution ecosystem 100 including a distribution infrastructure 110 in communication with a content player 120. In some embodiments, the distribution infrastructure 110 can be configured to encode data at a particular data rate and transmit the encoded data to the content player 120. The content player 120 can be configured to receive the encoded data via the distribution infrastructure 110 and decode the data for playback to a user. The data provided by the distribution infrastructure 110 can include audio, video, text, images, animations, interactive content, haptic data, virtual or augmented reality data, location data, gaming data, or any other type of data that can be provided via streaming media.

[0043] The distribution infrastructure 110 generally represents any service, hardware, software, or other infrastructure component configured to deliver content to end users. For example, the distribution infrastructure 110 may include a content aggregation system, media transcoding and packaging services, network components, and / or various other types of hardware and software. The distribution infrastructure 110 may be implemented as a highly complex distribution system, a single media server or device, or anything in between. In some examples, regardless of size or complexity, the distribution infrastructure 110 may include at least one physical processor 112 and at least one memory device 114. As described herein, one or more modules 116 may be stored or loaded into the memory 114 to implement adaptive streaming.

[0044] The content player 120 generally represents any type or form factor of device or system capable of playing audio and / or video content provided via the distribution infrastructure 110. Examples of content players 120 include, but are not limited to, mobile phones, tablet computers, laptop computers, desktop computers, televisions, set-top boxes, digital media players, virtual reality headsets, augmented reality glasses, and / or any other type or form factor of device capable of presenting digital content. As with the distribution infrastructure 110, the content player 120 may include a physical processor 122, memory 124, and one or more modules 126. Some or all of the adaptive streaming processing described herein may be performed or enabled by the modules 126, and in some examples, the modules 116 of the distribution infrastructure 110 may coordinate with the modules 126 of the content player 120 to provide adaptive streaming of multimedia content.

[0045] In certain embodiments, Figure 1 One or more of modules 116 and / or 126 in the example embodiment may represent one or more software applications or programs that, when executed by a computing device, may cause the computing device to perform one or more tasks. For example, as will be described in greater detail below, one or more of modules 116 and 126 may represent modules stored and configured to execute on one or more general-purpose computing devices. Figure 1 One or more of the modules 116 and 126 in may also represent all or part of one or more special-purpose computers configured to perform one or more tasks.

[0046] In addition, one or more modules, processes, algorithms, or steps described herein may convert data, a physical device, and / or a representation of a physical device from one form to another. For example, one or more modules described herein may receive audio data to be encoded, transform the audio data by encoding, output the encoded data for use in an adaptive audio bitrate system, send the transformed data to a content player, and present the transformed data to an end user for use. Additionally or alternatively, one or more modules described herein may convert a processor, volatile memory, non-volatile memory, and / or any other portion of a physical computing device from one form to another by executing on a computing device, storing data on a computing device, and / or otherwise interacting with a computing device.

[0047] Physical processors 112 and 122 generally represent any type or form of hardware-implemented processing unit capable of interpreting and / or executing computer-readable instructions. In one example, physical processors 112 and 122 can access and / or modify one or more of modules 116 and 126, respectively. Additionally or alternatively, physical processors 112 and 122 can execute one or more of modules 116 and 126 to facilitate adaptive streaming of multimedia content. Examples of physical processors 112 and 122 include, but are not limited to, microprocessors, microcontrollers, central processing units (CPUs), field programmable gate arrays (FPGAs) implementing soft-core processors, application-specific integrated circuits (ASICs), portions, variations, or combinations of one or more of these, and / or any other suitable physical processors.

[0048] The memories 114 and 124 generally represent any type or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. In one example, the memories 114 and / or 124 can store, load, and / or maintain one or more of the modules 116 and 126. Examples of memories 114 and / or 124 include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, a hard disk drive (HDD), a solid-state drive (SSD), an optical drive, a cache, a variation or combination of one or more memories, and / or any other suitable memory device or system.

[0049] Figure 2 2 is a block diagram of exemplary components of a content delivery infrastructure 110 according to certain embodiments. The delivery infrastructure 110 may include storage 210, services 220, and a network 230. Storage 210 generally represents any device, group of devices, and / or system capable of storing content for delivery to end users. Storage 210 may include a central repository having equipment capable of storing terabytes or petabytes of data and / or may include a distributed storage system (e.g., a device that mirrors or caches content at internet interconnected locations to provide faster access to mirrored content within a particular region). Storage 210 may also be configured in any other suitable manner.

[0050] As shown, storage device 210 can store, among other items, content 212, user data 214, and / or log data 216. Content 212 can include television programs, movies, video games, user-generated content, and / or any other suitable type or form of content. User data 214 can include personally identifiable information (PII), payment information, preferences, language and accessibility settings, and / or any other information associated with a particular user or content player. Log data 216 can include viewing history information, network throughput information, and / or any other metrics associated with a user's connection to or interaction with distribution infrastructure 110.

[0051] Services 220 may include personalization services 222, transcoding services 224, and / or packaging services 226. Personalization services 222 may personalize recommendations, content streams, and / or other aspects of a user's experience with distribution infrastructure 110. Encoding services 224 may compress media at different bit rates, as described in more detail below, which may enable real-time switching between different encodings. Packaging services 226 may package the encoded video for streaming before deploying it to a delivery network (e.g., network 230).

[0052] Network 230 generally represents any medium or architecture capable of facilitating communication or data transmission. Network 230 may facilitate communication or data transmission using wireless and / or wired connections. Examples of network 230 include, but are not limited to, an intranet, a wide area network (WAN), a local area network (LAN), a personal area network (PAN), the Internet, power line communication (PLC), a cellular network (e.g., a Global System for Mobile Communications (GSM) network), a portion of one or more networks, one or more variations or combinations of the same network, and / or any other suitable networks. For example, Figure 2 As shown, network 230 may include an Internet backbone 232, an Internet service provider 234, and / or a local network 236. As discussed in more detail below, bandwidth limitations and bottlenecks within one or more of these network segments may trigger video and / or audio bitrate adjustments.

[0053] Figure 3 yes Figure 1 The block diagram of an exemplary implementation of a content player 120 is shown. The content player 120 generally represents any type or form of computing device capable of reading computer-executable instructions. The content player 120 may include, but is not limited to, a laptop, a tablet, a desktop, a server, a cellular phone, a multimedia player, an embedded system, a wearable device (e.g., a smartwatch, smart glasses, etc.), a smart vehicle, a gaming console, an Internet of Things (IoT) device (e.g., a smart appliance), variations or combinations of one or more thereof, and / or any other suitable computing device.

[0054] like Figure 3 As shown, in addition to the processor 122 and the memory 124, the content player 120 may also include a communication infrastructure 302 and a communication interface 322 coupled to a network connection 324. The content player 120 may also include a graphics interface 326 coupled to a graphics device 328, an input interface 334 coupled to an input device 336, and a storage interface 338 coupled to a storage device 340.

[0055] Communication infrastructure 302 generally represents any type or form of infrastructure capable of facilitating communication between one or more components of a computing device. Examples of communication infrastructure 302 include, but are not limited to, any type or form of communication bus (e.g., a Peripheral Component Interconnect (PCI) bus, a PCI Express (PCIe) bus, a memory bus, a front-side bus, an integrated drive electronics (IDE) bus, a control or register bus, a host bus, etc.).

[0056] As described above, memory 124 generally represents any type or form of volatile or non-volatile storage device or medium capable of storing data and / or other computer-readable instructions. In some examples, memory 124 can store and / or load an operating system 308 for execution by processor 122. In one example, operating system 308 can include and / or represent software that manages computer hardware and software resources and / or provides common services to computer programs and / or applications on content player 120.

[0057] The operating system 308 may perform various system management functions, such as managing hardware components (e.g., graphics interface 326, audio interface 330, input interface 334, and / or storage interface 338). The operating system 308 may also provide a process and memory management model for the playback application 310. Modules of the playback application 310 may include, for example, a content buffer 312, an audio decoder 318, and a video decoder 320.

[0058] The playback application 310 can be configured to retrieve digital content via the communication interface 322 and play the digital content through the graphics interface 326. The graphics interface 326 can be configured to send the rendered video signal to the graphics device 328. In normal operation, the playback application 310 can receive a request from a user to play a specific title or specific content. The playback application 310 can then identify one or more encoded video and audio streams associated with the requested title. After the playback application 310 has located the encoded streams associated with the requested title, the playback application 310 can download the sequence header index associated with each encoded stream associated with the requested title from the distribution infrastructure 110. The sequence header index associated with the encoded content can include information related to the sequence of encoded data included in the encoded content.

[0059] In one embodiment, the playback application 310 can start downloading the content associated with the requested title by downloading sequence data encoded at the lowest audio and / or video playback bit rate to minimize the startup time of playback. The requested digital content file can then be downloaded to a content buffer 312, which can be configured to function as a first-in, first-out queue. In one embodiment, each unit of downloaded data can include a video data unit or an audio data unit. When a video data unit associated with the requested digital content file is downloaded to the content player 120, the video data unit can be pushed into the content buffer 312. Similarly, when an audio data unit associated with the requested digital content file is downloaded to the content player 120, the audio data unit can be pushed into the content buffer 312. In one embodiment, the video data unit can be stored in a video buffer 316 within the content buffer 312, and the audio data unit can be stored in an audio buffer 314 within the content buffer 312.

[0060] The video decoder 320 can read units of video data from the video buffer 316 and can output the units of video data in a sequence of video frames corresponding to the duration of the fixed playback timespan. Reading units of video data from the video buffer 316 can effectively dequeue the units of video data from the video buffer 316. The sequence of video frames can then be rendered by the graphics interface 326 and sent to the device 328 for display to the user.

[0061] The audio decoder 318 can read audio data units from the audio buffer 314 and output the audio data units as a sequence of audio samples, typically synchronized in time with the sequence of decoded video frames. In one embodiment, the sequence of audio samples can be sent to the audio interface 330, which can convert the sequence of audio samples into an electrical audio signal. The electrical audio signal can then be sent to a speaker of the audio device 332, which in response can produce a sound output.

[0062] In situations where the bandwidth of the distribution infrastructure 110 is limited and / or variable, the playback application 310 can download and buffer successive portions of video data and / or audio data from the video codec at different bit rates based on various factors (e.g., scene complexity, audio complexity, network bandwidth, device capabilities, etc.). In some embodiments, video playback quality can be prioritized over audio playback quality. Audio playback and video playback quality can also be balanced with each other, and in some embodiments, audio playback quality can be prioritized over video playback quality.

[0063] Graphics interface 326 may be configured to generate video data frames and send the video data frames to graphics device 328. In one embodiment, graphics interface 326 may be included as part of an integrated circuit along with processor 122. Alternatively, graphics interface 326 may be configured as a hardware accelerator distinct from (i.e., not integrated into) a chipset that includes processor 122.

[0064] Graphics interface 326 generally represents any type or form of device configured to forward images for display on graphics device 328. For example, graphics device 328 can be manufactured using liquid crystal display (LCD) technology, cathode ray tube technology, and light emitting diode (LED) display technology (organic or inorganic). In some embodiments, graphics device 328 can also include a virtual reality display and / or an augmented reality display. Graphics device 328 can include any technically feasible means for generating images for display. In other words, graphics device 328 generally represents any type or form of device capable of visually displaying the information forwarded by graphics interface 326.

[0065] like Figure 3 As shown, the content player 120 may also include at least one input device 336 coupled to the communication infrastructure 302 via an input interface 334. The input device 336 generally represents any type or form of computing device capable of providing computer or human-generated input to the content player 120. Examples of the input device 336 include, but are not limited to, a keyboard, a pointing device, a voice recognition device, a touch screen, a wearable device (e.g., a glove, a watch, etc.), a controller, variations or combinations of one or more thereof, and / or any other type or form of electronic input mechanism.

[0066] The content player 120 may also include a storage device 340 coupled to the communication infrastructure 302 via a storage interface 338. The storage device 340 generally represents any type or form of storage device or medium capable of storing data and / or other computer-readable instructions. For example, the storage device 340 may be a magnetic disk drive, a solid-state drive, an optical disk drive, a flash drive, etc. The storage interface 338 generally represents any type or form of interface or device for transferring data between the storage device 340 and other components of the content player 120.

[0067] Many other devices or subsystems may be included in or connected to the content player 120. Figure 3 One or more of the components and devices shown in the drawings need not be present to practice the embodiments described and / or illustrated herein. Figure 3The content player 120 may also be interconnected in different ways as shown. The content player 120 may also employ any number of software, firmware and / or hardware configurations. For example, one or more of the exemplary embodiments disclosed herein may be encoded as a computer program (also referred to as computer software, software application, computer-readable instructions, or computer control logic) on a computer-readable medium. The term "computer-readable medium" as used herein may refer to any form of device, carrier, or medium capable of storing or carrying computer-readable instructions. Examples of computer-readable media include, but are not limited to, transmission-type media (e.g., carrier waves) and non-transitory media, such as magnetic storage media (e.g., hard drives, tape drives, etc.), optical storage media (e.g., compact disks (CDs), digital video disks (DVDs), and Blu-ray discs), electronic storage media (e.g., solid-state drives and flash memory media), and other digital storage systems.

[0068] A computer-readable medium containing a computer program can be loaded into the content player 120. All or a portion of the computer program stored on the computer-readable medium can then be stored in the memory 124 and / or the storage device 340. When executed by the processor 122, the computer program loaded into the memory 124 can cause the processor 122 to perform the functions of one or more exemplary embodiments described and / or illustrated herein and / or serve as a means for performing such functions. Additionally or alternatively, one or more exemplary embodiments described and / or illustrated herein can be implemented in firmware and / or hardware. For example, the content player 120 can be configured as an application-specific integrated circuit (ASIC) suitable for implementing one or more of the exemplary embodiments disclosed herein.

[0069] Figure 4 A computing environment 400 is shown that includes a computer system 401. Computer system 401 can be substantially any type of computer system, including a local computer system or a distributed (e.g., cloud) computer system. Computer system 401 includes at least one processor 402 and at least some system memory 403. Computer system 401 also includes program modules for performing various functions. The program modules are hardware-based, software-based, or include a combination of hardware and software. Each program module uses computing hardware and / or software to perform a specified function, including the functions described below.

[0070] For example, the communication module 404 communicates with other computer systems. The communication module 404 includes wired or wireless communication devices that receive data from other computer systems and / or send data to other computer systems. These communication devices may include hardware radio devices, for example, including a hardware-based receiver 405, a hardware-based transmitter 406, or a hardware-based combined transceiver capable of receiving and sending data. The radio device may be a WIFI radio device, a cellular radio device, a Bluetooth radio device, a global positioning system (GPS) radio device, or other types of radio devices. The communication module 404 interacts with databases, mobile computing devices (such as mobile phones or tablets), embedded or other types of computing systems.

[0071] Computer system 401 also includes a video content access module 409. Video content access module 409 receives video content 407 from a content source. The video content source can be a producer of a video (e.g., a movie or television program), a distributor of a video (e.g., a streaming video content provider), or other entities that provide video content to an electronic device. As described above, computer system 401 can be any type of computer system, including a set-top box, a mobile phone, a laptop, a PC, or other computer systems running software applications including a video streaming client application. At least one of these applications receives video content 407 and presents the video content on a display (e.g., display 420).

[0072] Video content 407 is encoded at a specified frame rate. This frame rate can be 24fps, 25fps, 30fps, or some other frame rate. Video content access module 409 accesses this video content 407 and prepares it for subsequent operations on computer system 401. Before or after receiving video content 407, refresh rate determination module 410 determines the rate at which electronic display 420 is set to refresh its content. For example, display 420 may refresh content for display at 50Hz, 60Hz, 100Hz, 120Hz, or some other refresh rate. Thus, the display redraws or represents a given video frame 50, 60, 100, or 120 times per second. As mentioned above, display refresh rates vary around the world. For an optimal viewing experience, the refresh rate 421 of display 420 is synchronized with the encoded frame rate 408 of video content 407 so that the video frames displayed on the display are consistent. In other words, the video frames are duplicated in a way that is invisible to the viewer.

[0073] To accomplish this replication, a time interval determination module 412 identifies or specifies a time interval 413 for the video content. In at least some embodiments, the time interval 413 is longer than one second, so that the time interval to which frame rate conversion is applied covers two, three, or more seconds, compared to conventional systems that only view one-second time intervals. A frame rate conversion module 414 performs the frame rate conversion based on the frame rate 408 of the video content and the refresh rate 421 of the display.

[0074] As part of the frame rate conversion process, the frame rate conversion module 414 looks at a relatively long interval of video content (e.g., three seconds) and generates a first frame copy sequence 417 to go along with the original video frames 416A for that time interval, and also generates a second frame copy sequence 418 to go along with the original video frames 416B for that time interval. The time interval 413, along with its original frames and corresponding frame copy sequences, is then played back on the display 420. However, certain portions of the time interval of the video content 407 are played back at different speeds. The playback speed adjustment module 415 increases or decreases the playback speed of the original frames and / or the various frame copy sequences in order to create a video playback that is seamless to the viewer's eye. This will be discussed below with reference to the playback speed adjustment module 415. Figure 5 Method 500 further explains these concepts.

[0075] Figure 5 is a flow chart of an exemplary computer-implemented method 500 for aligning the frame rate of a video with the refresh rate of an electronic display. Figure 5 The steps shown may be performed by any suitable computer executable code and / or computing system, including Figure 1-4 In one example, Figure 5 Each step shown may represent an algorithm whose structure includes and / or is represented by multiple sub-steps, examples of which are provided in more detail below.

[0076] like Figure 5 As shown, at step 510, one or more systems or modules described herein access a portion of video content encoded at a specified frame rate. For example, Figure 44. In step 520, the video content access module 409 of the computer system 401 accesses video content 407 encoded at a frame rate 408. In some examples, the frame rate is 24 fps. In step 520, the refresh rate determination module 410 of the computer system 401 determines the refresh rate of the electronic display 420 on which the video content is to be presented. In some examples, the refresh rate 421 of the display 420 is 50 Hz, and in other examples, the refresh rate is 60 Hz. In step 530, the time interval determination module 412 determines a time interval 413 within which the frame rate conversion is to be applied. In some examples, the time interval is 3 seconds. Therefore, in this example, the frame rate conversion is applied within three seconds of the video content to synchronize the frame rate 408 of the video content 407 with the refresh rate 421 of the electronic display 420.

[0077] At step 540, the frame rate conversion module 414 performs frame rate conversion within the time interval 413 and presents the time interval of the video content on the electronic display 420. The frame rate conversion and presentation on the electronic display 420 includes adjusting the playback speed of a first portion of the designated interval of the video content so that the first portion of the designated interval is played back using the original video frames and the first frame copy sequence. The playback speed of a subsequent second portion of the designated interval of the video content is also adjusted so that the second portion of the designated interval is played back using the original video frames and a different second frame copy sequence.

[0078] For example, Figure 6 As shown, time interval 601 (which can be Figure 4 The time interval 413 is the same or different) is divided into different parts. Although more or less than two parts can be used, Figure 6 The embodiment in includes two parts. Figure 6 The dotted line in indicates that the time interval 601 can be selected from any part of the video content, including the beginning, the middle or the end. In some cases, the time interval 601 spans multiple refreshes of the display 420, and in some cases, the time interval 601 spans a single refresh of the display 420.

[0079] 4. The first portion 602 of the time interval 601 includes an original video frame sequence 603A and a duplicate sequence 604. The second portion 606 of the time interval 601 includes an original video frame sequence 603B and a different duplicate sequence 605. The first portion and the second portion of the time interval can be divided into substantially any ratio, including 50 / 50, 60 / 40, 70 / 30, 80 / 20, 90 / 10, or any other ratio. Therefore, the first portion 602 can occupy more or less time interval than the second portion 606. Each portion includes original frames and / or duplicate frames. The playback speed of these original frames and duplicate frames is adjusted by the playback speed adjustment module 415.

[0080] In at least some embodiments, the playback speed of the original frame or the playback speed of the first copy or the second copy 604 / 605 can be increased or decreased to ensure that a specified number of frames fit within the time interval 601. For example, if Figure 4 For example, if the video content 407 has a frame rate 408 of 24 fps, the time interval 601 may include 24 original frames, 24 replicated frames in the first replicated sequence 604, and 2 replicated frames in the second replicated sequence 605. In other embodiments, the time interval 601 may include 22 original frames, 22 replicated frames in the first replicated sequence 604, and 6 replicated frames in the second replicated sequence 605. The playback speed of these original and replicated frames can be increased or decreased to ensure smooth playback of the video content to the user. The playback speed can be increased or decreased without adjusting the audio speed.

[0081] Outside of specified limits, adjusting the playback speed of video content without adjusting the audio playback speed will result in noticeable A / V synchronization issues (i.e., seeing the user speak before hearing the audio, or hearing the user speak without moving their mouth). However, within certain limits (e.g., within the range of -125ms to +45ms), the audio and video will be considered synchronized, even if they are not completely aligned. Video content is typically played back using a presentation timestamp (PTS). In some embodiments, this presentation timestamp is changed to increase or decrease the playback speed of the video content. In other embodiments, a lookup table is implemented in conjunction with the PTS, wherein the lookup table adds or subtracts a time value from the PTS and uses the modified value when playing back the video content. Therefore, by increasing the time interval for creating duplicate frames to align with the display refresh rate, and by speeding up or slowing down video playback within specified limits, the embodiments herein allow an even number of duplicate frames to be added within this time interval. This greatly improves the smoothness of video playback and reduces or eliminates the jitter caused by other frame rate conversion solutions.

[0082] As mentioned above, and as Figure 7 As shown, the source frame rate 701 of the video content is typically 24 fps (or 24 Hz), 24 Hz, 30 Hz, 50 Hz, or 60 Hz. Figure 4 The frame rate conversion module 414 applies the frame rate conversion 702 to align with the display refresh rate 703. Figure 8As shown, frame rate conversion for some video content frame rates may appear smoother and may have fewer artifacts visible to the viewer. For example, in case #1 (at 801), the frame source rate is 50 Hz and is being converted to a 60 Hz display (at 802). The ratio of original frames to replicated frames is 1.2 (at 803), or 50 original frames to 10 replicated (first-order) frames (at 804). In case #2, the frame source rate is 30 Hz and is converted to 50 Hz at a ratio of 1.67, with 30 original frames, 20 first-order replicated frames, and a total of 50 frames. In case #5, the frame source rate is 24 Hz and is converted to 60 Hz at a ratio of 2.5, with 24 original frames, 24 first-order replicated frames, and 12 second-order replicated frames (805), for a total of 60 frames.

[0083] Figure 8 Most of the cases #1-6 will play back smoothly and the viewer will not notice any severe jitter in the picture. However, in cases #7-9, the frame rate conversion is from 24Hz to 25Hz or from 24Hz to 50Hz with ratios of 1.042 and 2.083 (at 803). This ratio introduces situations where only a single frame with first-order replication is presented (e.g., case #7), or 24 frames with first-order replication and two frames with second-order replication are presented (at 805) (e.g., case #8), or 24 frames with first-order replication frames and one frame with second-order replication frames and one frame with third-order replication frames are presented (at 806) (e.g., case #9). High-order frame replication is generally less than ideal because it often means low frame replication counts and longer frame display intervals, resulting in jitter artifacts.

[0084] For example, a first-order duplicate frame is a frame that repeats itself once. A second-order frame is a frame that replicates itself twice, and a third-order frame is a frame that replicates itself three times. A higher frame order means that the original frame has been replicated more times, and therefore is less smooth and more prone to jitter. Therefore, in the embodiments described herein, the system generates a fixed replication interval to improve playback smoothness. In addition, frame replication is kept below a minimum number (e.g., 2 replications, second order) to avoid jitter and other playback issues.

[0085] Figure 9 An example frame duplication embodiment is shown. At 902, three original frames 901 are duplicated at a pull-down rate of 3:2. Figure 10A This 3:2 pulldown rate is further illustrated. In this example, frame 1000A is converted from 24fps to 60fps. Frames are spaced within a 1 second time window 1001A. Each frame is replicated one or two times. Therefore, when added to the original frame, original + single replicate = 2, original + double replicate = 3. Therefore, Figure 9The 3:2 pattern shown repeats itself over a period of 1 second. Each original plus single copy is displayed for 33 ms (1002A), and each original plus dual copy is displayed for 48 ms. These times coincide with the two or three 16 ms segments 1004A produced by the 60 Hz refresh rate on TV 1005A. Other copy patterns or intervals can also be used, such as 3:2:2 or 3:2:2:2. Some patterns work better with 50 Hz displays, while others work better with 60 Hz displays.

[0086] Figure 10B An alternative example of converting frames 1000B from 24 fps to 50 fps is shown. Frames are spaced within a 1-second time window 1001B, where each frame is replicated once or twice. Each original plus single copy is displayed for 40 ms (1002B), and each original plus double copy is displayed for 60 ms. These times align with the two or three 20 ms segments 1004B produced by a 50 Hz refresh rate on a TV 1005B. As described above, other replication patterns or intervals may also be used.

[0087] Although Figure 10A and Figure 10B The example shown in shows one second of content, but it should be appreciated that in embodiments herein, the time division of the frames may be distributed over different time intervals. Figure 10C As shown, if a three-second time interval is selected for a film encoded at 24 fps, the film will have 72 frames in the three-second time interval. These 72 frames can be displayed on a 50 Hz display, which has 150 display cycles in three seconds. These 72 frames can be displayed by Figure 4 The frame rate conversion module 414 converts the original frame 416A and the first repeat sequence 417 and the other original frames 416B at a specific rate, and these original frames 416B have their own, different set of corresponding duplicate frames 418. The content 1001C at the first second includes the original frame and the duplicate played back in 40ms segments per (frame + duplicate). The content at the second second also includes the original frame and the duplicate played back in 40ms segments per frame. However, the content at the third second in the 3 second window includes multiple 60ms segments, in which (original + two duplicates) are played back. As shown below Figure 11B As shown, these six 60ms segments can be modulated at different speeds to realign the audio and video.

[0088] For example, Figure 11A 1 shows a frame rate conversion (FRC) window 1100, where the window time interval is three seconds. This three-second time window is divided into three different segments, each of which is one second long (0 to 1, 1 to 2, and 2 to 3). In this example, Figure 10ASimilarly, the encoding frame rate of video content is 24fps. Therefore, 72 frames will be displayed in a three-second time window. Compared with traditional playback applications or set-top boxes that play frames at a stable rate, the embodiments of this article can playback the first 24 frames in the first second of the content. Therefore, as shown in chart 1100, 24 original frames are displayed in the first second, from 0 to 1, and 24 first-order frame copies. This 1: 1 matching provides smooth playback experience on a 50Hz display screen, because each frame on the electronic display has corresponding original frame or first-order copy. These frames are displayed on the electronic display with a duration of 0.96s.

[0089] Continuing with the example, the frame rate conversion module (e.g., Figure 4 414) again provides 24 original frames to the display, along with 24 first-order replicated frames. These 48 frames, along with the first 48 frames from second 0 to 1 of the content, are played back at a higher speed, allowing 48 frames to be displayed with a duration of 0.96 seconds, with the video content encoded at 24fps. Figure 4 The playback speed adjustment module 415 adjusts the playback speed upward to allow these 96 frames of 2 seconds of content to be played in 1.92 seconds, vertically aligned with a 50 Hz display. However, speeding up the video playback does not necessarily speed up the audio playback, which may be noticeable to the viewer. Instead, the video speed increase relative to the audio remains within the range mentioned above (e.g., between -125ms and +45ms).

[0090] Figure 11B shows how the audio and video become separated (VA) as the playback speed of the video increases. For example, at time 0, A / V sync 1151 occurs. At this point, the audio and video are perfectly aligned. Two more A / V syncs occur during the first two seconds of the three second time window (e.g., the time period represented by 1153). However, row 1152, which represents the display of the original and copied frames, shows that the display of these frames slowly becomes out of sync with the audio (e.g., 40ms out of sync at 1 second and 80ms out of sync at two seconds). Ideally, A / V sync would occur at 0 for each new frame. However, as Figure 11A As shown, the third second of the three-second window displays only 24 original frames and 24 first-order replicated frames, along with six second-order frames displayed with a duration of 2.08 seconds. The playback speed of these frames is slow, as shown in Figure 11B The slower playback gradually brings the audio and video back into sync, so that at the end of the three-second window, A / V sync again occurs at 0. In this three-second window, 150 frames are displayed (24+24+24+24+24+24+6), which corresponds to 150 cycles on a 50Hz display.

[0091] therefore, Figure 4The playback speed adjustment module 4154 of the frame rate conversion module 414 can adjust and increase the playback speed of some video content over a specified time interval, and can reduce the playback speed of other video content over the same time interval. Thus, at the end of the interval, the audio and video are synchronized. Throughout this process, specified limits can be established and maintained to adjust the playback speed within specified maximum acceleration or deceleration amounts. By staying within these maximum deceleration or overspeeding limits, the viewing user will not be aware of the change in video speed. Instead, the viewer's eye will simply interpret the image as smooth motion.

[0092] In some cases, when the playback speed adjustment module 415 adjusts the playback speed of a portion of the video content, the adjustment module changes the presentation timestamp (PTS) associated with the video content and the copied frame interval in each sequence. In such an embodiment, Figure 11C As shown in FIG1100C , the sequence is 24:24:24 to allow for second-order frame replication with fixed intervals in the last part of the window. The PTS is adjusted over the course of 72 frames. For the first 48 frames, the PTS is adjusted downward, thereby speeding up playback. Then, after the 48th frame, the PTS is adjusted upward, which reduces the playback speed to the alignment point. In some cases, the playback speed adjustment module 415 can create a lookup table that adds or subtracts time from each PTS entry and then uses the lookup table when playing back the video content. In some cases, the lookup table includes incremental values that are applied during the time interval in which the video content is displayed on the electronic display. The incremental values are used to adjust the display timestamp of each frame within the time interval of the video content. Thus, the lookup table can store incremental values that indicate the playback speed of each interval of the video content segment.

[0093] exist Figure 11A and Figure 11B In the example embodiments of FIG, it will be appreciated that essentially any length of time window can be used, and any number of first-order or second-order replications can be used. In some cases, the second-order frame (or the second sequence of frame replications) includes at least a minimum threshold number of replication frames. This minimum threshold number can be essentially any number and can be set by policy. The minimum threshold number can be different, for example, for different encoding frame rates, different display refresh rates, or based on other contextual settings or preferences.

[0094] In some cases, the playback speed of a given video content segment is adjusted based on the type of video frames being presented. For example, some video content may be encoded at a higher rate to obtain fast motion video or other effects. The playback speed of different segments of the time window can be adjusted to accommodate this type of video content. In some cases, intervals of video content that only contain first-order frame replication will automatically play back at a specified playback speed, while intervals of video content that contain second-order frame replication will automatically play back at a different playback speed. Therefore, when certain types of replicated frames are detected, the application or set-top box that plays back the video content to the display can automatically adjust the playback speed.

[0095] Figure 12 An embodiment is shown that tests the time intervals of video content to ensure that they are within the set A / V synchronization boundaries. For example, Figure 12 Graph 1200 shows time on the x-axis and time values of audio / video synchronization on the y-axis. The y-axis shows that in this test embodiment, A / V sync can be +35ms ahead and -165ms behind. At point 1201, the A / V sync approaches or exceeds the +35ms threshold, so synchronization is performed at this point by skipping original video frames to bring the sync closer to 0 (video is advanced by 42ms per frame @ 24 frames per second). At points 1202 and 1203, the A / V sync also drifts towards the respective maximum thresholds. At these points, the video is played back faster or slower to realign the audio and video to the desired synchronization level 1204 (by repeating the original frames (video is slowed down by 42ms per frame @ 24fps). During a portion of the video content (e.g., Figure 4 Throughout the presentation of the video content (407), the computer system 401 can test the presentation of each time interval or certain time intervals to verify that the A / V synchronization is within the specified boundaries. Various quality metrics about the playback of the video content can be measured, including the number of first-order replicated frames used, the number of second-order or third-order replicated frames used, the level of synchronization between the audio and video, or other metrics. These metrics can then be used as feedback to ensure that the viewer has a positive viewing experience without noticeable video freezing or fast motion effects.

[0096] In some embodiments, a corresponding system includes a physical processor and a physical memory. The physical memory includes computer-executable instructions that, when executed by the physical processor, cause the physical processor to: access a portion of video content, wherein the video content is encoded at a specified frame rate, determine a refresh rate of an electronic display on which the video content is to be displayed, specify a time interval of the video content, apply a frame rate conversion within the time interval to synchronize the frame rate of the video content with the refresh rate of the electronic display, and present the specified time interval of the video content on the electronic display, wherein presenting includes adjusting a playback speed of a first portion of the specified time interval of the video content so that the first portion of the specified time interval is played back using the original video frames and a first frame copy sequence, and adjusting a playback speed of a subsequent second portion of the specified time interval of the video content so that the second portion of the specified time interval is played back using the original video frames and a different second frame copy sequence.

[0097] A corresponding non-transitory computer-readable medium includes computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to access a portion of video content, wherein the video content is encoded at a specified frame rate, determine a refresh rate of an electronic display on which the video content is to be displayed, specify a time interval of the video content, apply a frame rate conversion within the time interval to synchronize the frame rate of the video content with the refresh rate of the electronic display, and display the specified time interval of the video content on the electronic display. The presentation includes adjusting a playback speed of a first portion of the specified interval of the video content so that the first portion of the specified interval is played back using the original video frames and a first frame-replicated sequence, and adjusting a playback speed of a subsequent second portion of the specified interval of the video content so that the second portion of the specified interval is played back using the original video frames and a second, different, frame-replicated sequence.

[0098] Thus, in this way, video content can be divided into longer length intervals. These longer intervals allow for different variations of the original and copied frames. Furthermore, the playback speed can be varied within these intervals to ensure that playback on the electronic display is smooth and uniform throughout. The embodiments described herein can be used in virtually any country to convert virtually any source material to any display frame rate. Thus, these implementations can be used by video streaming providers in any market, regardless of which frame rate encoding or display refresh rate is used in that country.

[0099] Example embodiments:

[0100] 1. A computer-implemented method comprising: accessing a portion of video content, the video content being encoded at a specified frame rate; determining a refresh rate of an electronic display on which the video content is to be presented; specifying a time interval of the video content within which a frame rate conversion is to be applied to synchronize the frame rate of the video content with the refresh rate of the electronic display; and presenting the specified time interval of the video content on the electronic display, wherein the presenting comprises: adjusting the playback speed of a first portion of the specified interval of the video content so that the first portion of the specified interval is played back using original video frames and a first frame copy sequence; and adjusting the playback speed of a subsequent second portion of the specified interval of the video content so that the second portion of the specified interval is played back using the original video frames and a different second frame copy sequence.

[0101] 2 . The computer-implemented method of claim 1 , wherein the different second frame replication sequence includes at least a minimum threshold number of replicated frames.

[0102] 3 . The computer-implemented method of claim 1 , wherein adjusting the playback speed of the first portion of the specified interval of the video content comprises increasing the playback speed of the first portion of the specified interval of the video content.

[0103] 4 . The computer-implemented method of claim 1 , wherein adjusting the playback speed of the second portion of the specified interval of the video content comprises decreasing the playback speed of the second portion of the specified interval of the video content.

[0104] 5 . The computer-implemented method of claim 1 , wherein the playback speed is adjusted to within a specified maximum amount of speeding up or slowing down the playback speed.

[0105] The computer-implemented method of claim 1 , wherein adjusting playback speed comprises changing a presentation timestamp associated with the video content.

[0106] The computer-implemented method of claim 1 , wherein the specified time interval is greater than 1 second.

[0107] The computer-implemented method of claim 1 , wherein the specified time interval is at least 3 seconds.

[0108] 9. The computer-implemented method of claim 1, wherein the playback speed is adjusted based on a type of video frame being presented.

[0109] 10. The computer-implemented method of claim 9, wherein an interval of video content including one or more first-order frame replications is automatically played back at a specified playback speed.

[0110] 11. The computer-implemented method of claim 9, wherein an interval of video content including one or more second-order frame copies is automatically played back at a specified playback speed.

[0111] 12. The computer-implemented method of claim 1, wherein the number of replicated frames is repeated at specified intervals.

[0112] 13. A system comprising: at least one physical processor; and a physical memory comprising computer-executable instructions that, when executed by the physical processor, cause the physical processor to: access a portion of video content, the video content being encoded at a specified frame rate; determine a refresh rate of an electronic display on which the video content is to be presented; specify a time interval of the video content within which a frame rate conversion is to be applied to synchronize the frame rate of the video content with the refresh rate of the electronic display; and present the specified time interval of the video content on the electronic display, wherein the presentation comprises: adjusting the playback speed of a first portion of the specified interval of the video content so that the first portion of the specified interval is played back using the original video frame and a first frame copy sequence; and adjusting the playback speed of a subsequent second portion of the specified interval of the video content so that the second portion of the specified interval is played back using the original video frame and a different second frame copy sequence.

[0113] 14. The system of claim 13, wherein the playback speed of each portion of a specified interval of the video content is stored in a lookup table.

[0114] 15. The system of claim 14, wherein a designated interval of the video content is played back on the electronic display according to the playback information indicated in the lookup table.

[0115] 16. The system of claim 15, wherein the lookup table comprises one or more delta values that are applied during a specified time interval of presentation of the video content on the electronic display to adjust a presentation timestamp of each frame within the specified time interval of the video content.

[0116] 17. The system of claim 13, further comprising testing the presentation of specified time intervals of the video content on the electronic display to verify one or more specified quality metrics with respect to playback of the video content.

[0117] 18. The system of claim 13, wherein the frame rate of the video content is 24 frames per second, and wherein the refresh rate of the electronic display is 50 Hz.

[0118] 19. The system of claim 13, wherein the frame rate of the video content is 24 frames per second, and wherein the refresh rate of the electronic display is 60 Hz.

[0119] 20. A non-transitory computer-readable medium comprising one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to: access a portion of video content, the video content being encoded at a specified frame rate; determine a refresh rate of an electronic display on which the video content is to be presented; specify a time interval of the video content within which a frame rate conversion is to be applied to synchronize the frame rate of the video content with the refresh rate of the electronic display; and present the specified time interval of the video content on the electronic display, wherein the presentation comprises: adjusting the playback speed of a first portion of the specified interval of the video content so that the first portion of the specified interval is played back using the original video frames and a first frame copy sequence; and adjusting the playback speed of a subsequent second portion of the specified interval of the video content so that the second portion of the specified interval is played back using the original video frames and a different second frame copy sequence.

[0120] As mentioned above, the computing devices and systems described and / or illustrated herein broadly represent any type or form of computing device or system capable of executing computer-readable instructions, such as those contained in the modules described herein. In their most basic configuration, these computing devices may each include at least one memory device and at least one physical processor.

[0121] In some examples, the term "memory device" generally refers to any type or form of volatile or non-volatile storage device or medium capable of storing data and / or computer-readable instructions. In one example, a memory device can store, load, and / or maintain one or more modules described herein. Examples of memory devices include, but are not limited to, random access memory (RAM), read-only memory (ROM), flash memory, hard disk drive (HDD), solid-state drive (SSD), optical disk drive, cache, variations or combinations of one or more thereof, or any other suitable storage memory.

[0122] In some examples, the term "physical processor" generally refers to any type or form of hardware-implemented processing unit capable of interpreting and / or executing computer-readable instructions. In one example, the physical processor can access and / or modify one or more modules stored in the above-mentioned storage device. Examples of physical processors include, but are not limited to, a microprocessor, a microcontroller, a central processing unit (CPU), a field programmable gate array (FPGA) implementing a soft-core processor, an application-specific integrated circuit (ASIC), parts, variants, or combinations of one or more of them, and / or any other suitable physical processor.

[0123] Although illustrated as separate elements, the modules described and / or illustrated herein may represent portions of a single module or application. Furthermore, in some embodiments, one or more of these modules may represent one or more software applications or programs that, when executed by a computing device, can cause the computing device to perform one or more tasks. For example, one or more modules described and / or illustrated herein may represent modules stored and configured to run on one or more computing devices or systems described and / or illustrated herein. One or more of these modules may also represent all or part of one or more special-purpose computers configured to perform one or more tasks.

[0124] Furthermore, one or more modules described herein can convert data, physical devices, and / or representations of physical devices from one form to another. For example, one or more modules described herein can receive data to be converted, convert the data, output the conversion results to determine how to present video content, use the conversion results to present the video content, and store the conversion results to indicate how the video content is presented. Additionally or alternatively, one or more modules described herein can convert a processor, volatile memory, non-volatile memory, and / or any other portion of a physical computing device from one form to another by executing on a computing device, storing data on a computing device, and / or otherwise interacting with a computing device.

[0125] In some embodiments, the term "computer-readable medium" generally refers to any form of device, carrier, or medium capable of storing or carrying computer-readable instructions. Examples of computer-readable media include, but are not limited to, transmission-type media (e.g., carrier waves) and non-transitory media (e.g., magnetic storage media (e.g., hard drives, tape drives, and floppy disks), optical storage media (e.g., compact disks (CDs), digital video disks (DVDs), and Blu-ray disks), electronic storage media (e.g., solid-state drives and flash memory media), and other distribution systems.

[0126] The process parameters and order of the steps described and / or illustrated herein are provided as examples only and may be modified as needed. For example, although the steps shown and / or described herein may be shown or discussed in a particular order, these steps do not necessarily need to be performed in the order shown or discussed. The various exemplary methods described and / or illustrated herein may also omit one or more steps described or illustrated herein, or include additional steps in addition to the disclosed steps.

[0127] The foregoing description is provided to enable others skilled in the art to best utilize the various aspects of the exemplary embodiments disclosed herein. This exemplary description is not exhaustive or limited to any precise form disclosed. Many modifications and variations may be made without departing from the spirit and scope of the present invention. The embodiments disclosed herein are to be considered in all respects as illustrative and not restrictive. In determining the scope of the present disclosure, reference should be made to the appended claims and their equivalents.

[0128] Unless otherwise indicated, the terms "connected to" and "coupled to" (and their derivatives) used in the specification and claims should be interpreted as allowing both direct and indirect connections (i.e., through other elements or components). Furthermore, the terms "a" or "an" used in the specification and claims should be interpreted as meaning "at least one." Finally, for ease of use, the terms "including" and "having" (and their derivatives) used in the specification and claims are interchangeable with the word "comprising" and have the same meaning as the word "comprising."

Claims

1. A method for frame rate conversion, comprising: accessing a portion of video content, the video content being encoded at a specified frame rate and designed to be played back at a specified playback speed, the encoded frame rate defining a specified frame duration for each frame of the video content; determining a refresh rate of an electronic display on which the video content is to be presented; specifying a time interval of the video content within which frame rate conversion is to be applied to synchronize the frame rate of the video content with a refresh rate of the electronic display; as well as Presenting the video content on the electronic display for a specified time interval, wherein the presenting comprises: adjusting a playback speed of a first portion of a specified interval of the video content to play back the first portion of the specified interval of the video content at an increased playback speed, wherein an increased number of frames per second are played back for at least a portion of the specified time interval using original video frames, wherein frames played back at the increased playback speed exhibit a frame duration that is shorter than a frame duration defined by the encoded frame rate; and Adjusting the playback speed of a subsequent second portion of the specified interval of the video content so that the second portion of the specified interval of the video content is played back at a reduced playback speed, wherein a reduced number of frames per second are played back for at least a portion of the specified time interval using the original video frames and the frame copy sequence, wherein the frames played back at the reduced playback speed display a frame duration that is longer than the frame duration defined by the encoded frame rate.

2. The method according to claim 1, wherein The frame copy sequence includes at least a minimum threshold number of copied frames.

3. The method according to claim 1, wherein Adjusting the playback speed of the first portion of the designated interval of the video content includes increasing the playback speed of the first portion of the designated interval of the video content.

4. The method according to claim 1, wherein Adjusting the playback speed of the second portion of the designated interval of the video content includes decreasing the playback speed of the second portion of the designated interval of the video content.

5. The method according to claim 1, wherein The playback speed is adjusted to within a specified maximum amount that speeds up or slows down the playback speed.

6. The method according to claim 1, wherein Adjusting the playback speed includes changing a presentation timestamp associated with the video content.

7. The method according to claim 1, wherein The specified time interval is greater than 1 second.

8. The method according to claim 1, wherein The specified time interval is at least 3 seconds.

9. The method according to claim 1, wherein The playback speed is adjusted based on the type of video frame being presented.

10. The method according to claim 9, wherein: An interval of video content including one or more first-order frame copies is automatically played back at a specified playback speed.

11. The method according to claim 9, wherein An interval of video content including one or more second-order frame copies is automatically played back at a specified playback speed.

12. The method according to claim 2, wherein: The number of duplicate frames is repeated at specified intervals.

13. A system for frame rate conversion, comprising: At least one physical processor; as well as A physical memory comprising computer-executable instructions, which, when executed by the physical processor, cause the physical processor to perform the following operations: accessing a portion of video content, the video content being encoded at a specified frame rate and designed to be played back at a specified playback speed, the encoded frame rate defining a specified frame duration for each frame of the video content; determining a refresh rate of an electronic display on which the video content is to be presented; specifying a time interval of the video content within which frame rate conversion is to be applied to synchronize the frame rate of the video content with a refresh rate of the electronic display; as well as Presenting the video content on the electronic display for a specified time interval, wherein the presenting comprises: adjusting a playback speed of a first portion of a specified interval of the video content to play back the first portion of the specified interval of the video content at an increased playback speed, wherein an increased number of frames per second are played back for at least a portion of the specified time interval using original video frames, wherein frames played back at the increased playback speed exhibit a frame duration that is shorter than a frame duration defined by the encoded frame rate; and Adjusting the playback speed of a subsequent second portion of the specified interval of the video content so that the second portion of the specified interval of the video content is played back at a reduced playback speed, wherein a reduced number of frames per second are played back for at least a portion of the specified time interval using the original video frames and the frame copy sequence, wherein the frames played back at the reduced playback speed display a frame duration that is longer than the frame duration defined by the encoded frame rate.

14. The system according to claim 13, wherein: The playback speed of each portion of a specified interval of the video content is stored in a lookup table.

15. The system according to claim 14, wherein: A designated interval of the video content is played back on the electronic display according to the playback information indicated in the lookup table.

16. The system according to claim 15, wherein: The lookup table includes one or more delta values that are applied during a specified time interval of presentation of the video content on the electronic display to adjust a presentation timestamp of each frame within the specified time interval of the video content.

17. The system of claim 13, further comprising testing the presentation of specified time intervals of the video content on the electronic display to verify one or more specified quality metrics with respect to playback of the video content.

18. The system of claim 13, wherein: The frame rate of the video content is 24 frames per second, and wherein the refresh rate of the electronic display is 50 Hz.

19. The system of claim 13, wherein: The frame rate of the video content is 24 frames per second, and wherein the refresh rate of the electronic display is 60 Hz.

20. A non-transitory computer-readable medium comprising one or more computer-executable instructions that, when executed by at least one processor of a computing device, cause the computing device to: accessing a portion of video content, the video content being encoded at a specified frame rate and designed to be played back at a specified playback speed, the encoded frame rate defining a specified frame duration for each frame of the video content; determining a refresh rate of an electronic display on which the video content is to be presented; specifying a time interval of the video content within which frame rate conversion is to be applied to synchronize the frame rate of the video content with a refresh rate of the electronic display; as well as Presenting the video content on the electronic display for a specified time interval, wherein the presenting comprises: adjusting a playback speed of a first portion of a specified interval of the video content to play back the first portion of the specified interval of the video content at an increased playback speed, wherein an increased number of frames per second are played back for at least a portion of the specified time interval using original video frames, wherein frames played back at the increased playback speed exhibit a frame duration that is shorter than a frame duration defined by the encoded frame rate; and Adjusting the playback speed of a subsequent second portion of the specified interval of the video content so that the second portion of the specified interval of the video content is played back at a reduced playback speed, wherein a reduced number of frames per second are played back for at least a portion of the specified time interval using the original video frames and the frame copy sequence, wherein the frames played back at the reduced playback speed display a frame duration that is longer than the frame duration defined by the encoded frame rate.

Images