Method for synchronously refreshing multiple display windows of a multimedia broadcasting device

Abstract

The application provides a multi-display window synchronous refreshing control method of a multimedia broadcast device, relates to the technical field of display control, and comprises the following steps: obtaining to-be-displayed frames of multiple display windows, converting time marks of the to-be-displayed frames into synchronous time marks under a unified synchronous time base, and determining an effective display time interval; correcting the effective display time interval according to display link time delays of the display windows, determining candidate to-be-displayed frames and a common overlapping interval from frame buffer queues; determining a synchronous refreshing period and a target display time according to a ready time of the candidate to-be-displayed frames and the common overlapping interval, generating a synchronous refreshing control item, and overall latching when a corresponding vertical synchronous signal arrives, so that the multiple display windows display target display frames corresponding to the same target display time in the same refreshing period, and the time consistency of multi-window display content is improved.

Description

Technical Field

[0001] This invention relates to the field of display control technology, and in particular to a method for synchronously refreshing multiple display windows in a multimedia broadcast control device. Background Technology

[0002] When existing multimedia broadcast control equipment presents multiple display windows on the same display interface, it usually establishes media decoding, image rendering and frame buffer queues for each display window. Each display window independently updates the display content according to the frame rate of the corresponding media data or the ready state of the frame to be displayed. Then, the display synthesis unit synthesizes the image data of multiple display windows according to the position, size and hierarchical relationship of each display window and outputs it to the display device for refresh.

[0003] However, in the central control and broadcasting scenarios of digital exhibition halls, multiple display windows often carry time-related display content. For example, the same central control device may simultaneously set up a main visual window, an exhibit explanation window, an animation demonstration window, and a subtitle information window on a single display page. When the explanation process enters a certain display node, multiple display windows need to synchronously present the screen corresponding to that display node. If each display window still refreshes independently according to the decoding completion time of its respective frame, for example, the video frame corresponding to the main visual window has been decoded before the current vertical refresh cycle, while the video frame corresponding to the same display node in the animation demonstration window has not been ready before the current vertical refresh cycle due to scaling, layer compositing, or frame buffer writing taking a long time, then the current output screen may simultaneously contain the newer screen of the main visual window and the previous screen of the animation demonstration window. Taking a 30 frames per second display video as an example, there is an interval of approximately 33.3 milliseconds between adjacent video frames. When the main visual window has already displayed the screen after the exhibit structure has unfolded, while the explanation window or animation demonstration window is still stuck on the screen before unfolding, a significant cross-window content misalignment will occur within the same display page. At this point, each display window can be displayed continuously without necessarily causing tearing within a single window. However, multiple display windows correspond to different media time ranges within the same refresh cycle, preventing the synchronized visual content, explanatory content, and animation content from being presented together at the same display node, thus disrupting the time correspondence between the content displayed in multiple windows. Summary of the Invention

[0004] The purpose of this invention is to provide a method for synchronously refreshing multiple display windows in a multimedia broadcast control device, aiming to solve the problems mentioned in the background art.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows: A method for synchronously refreshing multiple display windows in a multimedia broadcast control device, the method comprising: Get the frames to be displayed from multiple display windows, determine the synchronization time stamp of each frame to be displayed according to the mapping relationship between the media stream time base to which each frame to be displayed belongs and the unified synchronization time base, and store each frame to be displayed into the corresponding frame buffer queue. The effective display time interval of each frame to be displayed is determined based on the synchronization time stamp of adjacent frames to be displayed in the same frame buffer queue; Obtain the display link delay of each display window, and correct the effective display time interval of each frame to be displayed according to the corresponding display link delay to obtain the corrected effective display time interval; Candidate frames to be displayed are determined from each frame buffer queue, the common overlapping interval of the corrected effective display time interval of multiple candidate frames to be displayed is determined, and at least one candidate frame to be displayed is updated when there is no common overlapping interval, so as to redetermine the common overlapping interval. The readiness times of multiple candidate frames to be displayed under a unified synchronization time base are obtained. The synchronization refresh period is determined based on each readiness time and the common overlapping interval. The display time of the synchronization refresh period under the unified synchronization time base is determined as the target display time, and the multiple candidate frames to be displayed are determined as the target display frames. Generate a synchronization refresh control item containing the buffer addresses of multiple target display frames and the corresponding display window area information. When the vertical synchronization signal corresponding to the synchronization refresh cycle arrives, the synchronization refresh control item is latched as a whole so that multiple display windows display the target display frames corresponding to the same target display time in the same refresh cycle.

[0006] The above-described solution of the present invention has at least the following beneficial effects: First, by converting the timestamps of the frames to be displayed corresponding to multiple display windows to a unified synchronization time base, and determining the effective display time interval based on the synchronization timestamps of adjacent frames to be displayed, the frames to be displayed from different media streams have a unified time comparison basis. Furthermore, the effective display time interval is corrected by the display link delay corresponding to each display window, so that the time range of the candidate display frames can reflect the display time differences corresponding to different display windows. By determining the common overlapping interval of multiple corrected effective display time intervals, the candidate display frames selected by multiple display windows have a common time correspondence range. When there is no common overlapping interval, the time correspondence is re-established by updating the candidate display frames. Based on this, the actual executable synchronous refresh cycle is determined by combining the ready time of each candidate frame to be displayed and the common overlapping interval, and the corresponding display time is determined as the target display time, so that the time correspondence of the media content is combined with the actual refresh sequence of the device. Finally, when the vertical synchronization signal corresponding to the synchronous refresh cycle arrives, the synchronous refresh control item containing the buffer addresses of multiple target display frames and display window area information is latched as a whole, so that multiple display windows display the target display frame corresponding to the same target display time within the same refresh cycle, thereby reducing cross-window content misalignment caused by different ready times of the frames to be displayed and differences in display link latency, and improving the time consistency between multiple related display windows in the digital exhibition hall. Attached Figure Description

[0007] Figure 1 This is a flowchart of the multi-display window synchronous refresh control method provided in the embodiments of the present invention.

[0008] Figure 2 This is a flowchart of the display link latency acquisition and update method provided in an embodiment of the present invention.

[0009] Figure 3 This is a flowchart of a method for matching candidate frames to be displayed in multiple display windows based on a common overlapping region, provided in an embodiment of the present invention.

[0010] Figure 4 This is a flowchart of a multi-display window refresh control method based on a synchronous refresh cycle provided in an embodiment of the present invention. Detailed Implementation

[0011] Exemplary embodiments of the invention will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the invention are shown in the drawings, it should be understood that the invention may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this invention will be thorough and complete, and will fully convey the scope of the invention to those skilled in the art.

[0012] Example 1 This embodiment provides a method for synchronously refreshing multiple display windows in a multimedia broadcast control device. It is applied to a multimedia broadcast control device with multiple display windows to achieve synchronous refresh control of media content corresponding to multiple display windows.

[0013] In this embodiment, the multimedia broadcast control device can be applied to multimedia display scenarios such as digital exhibition halls, themed exhibition halls, and conference display systems. The multimedia broadcast control device can process multiple media contents simultaneously and output different media contents to corresponding display windows. Different display windows can correspond to different video images, animation content, text information, or other forms of media content.

[0014] In this embodiment, the display window refers to a logical display area controlled by the multimedia broadcast control device. Each display window has corresponding display window area information and is used to display corresponding media content. Multiple display windows are set in the display interface of the display device, and each display window can form different logical display areas according to its corresponding position, size, and hierarchical relationship.

[0015] Since the media content corresponding to multiple display windows may originate from different media streams, each media stream typically has its own independent time reference during acquisition, encoding, or transmission. When multiple display windows need to jointly present content with a time relationship, if refresh control is performed separately based on the time information of each media stream, different display windows may experience display time offsets due to differences in time references, media processing, and other factors. This would prevent multiple display windows from presenting content corresponding to the same time point within the same refresh cycle.

[0016] To address the aforementioned issues, the multimedia broadcast control device in this embodiment establishes a unified synchronization time base, performs time conversion on the frames to be displayed corresponding to multiple display windows, and performs synchronous control by combining the display time range of the frames to be displayed with the display refresh process, so that multiple display windows can complete the refresh according to a unified time relationship.

[0017] like Figure 1 As shown, the multi-display window synchronous refresh control method provided in this embodiment includes the following steps.

[0018] S101, obtain the frames to be displayed from multiple display windows, and determine the synchronization time stamp of the frames to be displayed.

[0019] In this embodiment, the multimedia broadcast control device includes multiple display windows, and each display window is provided with a frame buffer queue for caching frames to be displayed.

[0020] Specifically, the multimedia broadcast control equipment receives multiple media streams and parses each media stream to obtain the frames to be displayed in the corresponding display window.

[0021] The frames to be displayed may include video frames, animation frames, image frames, or other media data frames that need to be output in chronological order. Each frame to be displayed corresponds to a timestamp in its media stream, indicating the time position of the frame within the corresponding media content.

[0022] Since different media streams may correspond to different time bases, this embodiment first obtains the time base information of the media stream to which each frame to be displayed belongs, and establishes a mapping relationship between the time base of the media stream to which each frame to be displayed belongs and the unified synchronization time base.

[0023] When establishing the mapping relationship, the numerator and denominator of the media stream time base, the original timestamp of the frame to be displayed, and the playback start time latched by the unified synchronization time base counter when the media stream starts playing are read. First, the original timestamp is converted into the duration calculated from the start of the media stream according to the media stream time base. Then, this duration is added to the playback start time to obtain the synchronization time mark. All times are uniformly converted to the same timing unit before comparison. When the media stream jumps, changes speed, or restarts, a new original timestamp and the unified synchronization time are latched as the mapping anchor point, and the new mapping is only used for frames after the anchor point. When the original timestamp is missing or goes backward and is not a jump event, the corresponding frame is marked as time invalid and temporarily not included in the candidate frame matching.

[0024] Furthermore, based on the mapping relationship, the timestamps corresponding to each frame to be displayed are converted into synchronization timestamps under a unified synchronization time base.

[0025] Specifically, a reference timestamp can be selected from the media stream, and the reference time corresponding to the reference timestamp under a unified synchronization time base can be determined. For any frame to be displayed in the media stream, the time offset of the frame to be displayed relative to the reference timestamp is determined based on the difference between the timestamp of the frame to be displayed and the reference timestamp and the time unit corresponding to the media stream time base. The time offset is then added to the reference time to obtain the synchronization timestamp of the frame to be displayed under the unified synchronization time base.

[0026] For example, let the reference timestamp of the media stream be P0, the reference time corresponding to the reference timestamp P0 under the unified synchronization time base be T0, the timestamp of the frame to be displayed be Pi, and the conversion factor between the time unit corresponding to the media stream time base and the time unit of the unified synchronization time base be K. Then the synchronization timestamp Ti of the frame to be displayed can be expressed as: Ti = T0 + (Pi - P0) × K.

[0027] Thus, different frames to be displayed in the same media stream can be mapped to a unified synchronization time base according to their original time intervals, while frames to be displayed in different media streams can be converted to the same time reference system based on their respective corresponding reference time stamps and reference times.

[0028] The unified synchronization time base is a time reference used within the multimedia broadcast control equipment for multi-window synchronous control. The frames to be displayed corresponding to different display windows are time-correlated through this unified synchronization time base.

[0029] In this way, frames to be displayed in different media streams can be compared under the same time reference system, providing a time basis for subsequently determining the synchronization relationship between multiple display windows.

[0030] The multimedia broadcast control equipment stores each frame to be displayed in the frame buffer queue of the corresponding display window, and associates the determined synchronization time stamp with the corresponding frame to be displayed.

[0031] Each display window corresponds to an independent frame buffer queue. Each frame buffer queue is managed according to the synchronization time stamp of the frame to be displayed, so that the corresponding candidate frame to be displayed can be selected from different frame buffer queues during the subsequent synchronization control process.

[0032] S102, determine the effective display time interval of the frame to be displayed based on the synchronization time stamp.

[0033] In this embodiment, in order to determine whether the frames to be displayed in different display windows can correspond to the same time range, comparing only a single synchronization time stamp will lead to inaccurate judgment of time correlation.

[0034] Therefore, in this embodiment, the effective display time interval corresponding to each frame to be displayed is determined based on the synchronization time stamp of adjacent frames to be displayed in the same frame buffer queue.

[0035] Specifically, for any frame buffer queue corresponding to a display window, multiple frames to be displayed are sorted by time according to their synchronization timestamps. For the current frame to be displayed and the next frame to be displayed that are adjacent in time order, the time indicated by the synchronization timestamp of the current frame to be displayed is determined as the start time of the effective display time interval of the current frame to be displayed, and the time indicated by the synchronization timestamp of the next frame to be displayed is determined as the end time of the effective display time interval of the current frame to be displayed.

[0036] For example, if the synchronization time stamp of the current frame to be displayed is Ti, and the synchronization time stamp of the next frame to be displayed in the time sequence is Ti+1, then the effective display time interval of the current frame to be displayed can be represented as [Ti, Ti+1), where Ti corresponds to the start time of the effective display time interval, and Ti+1 corresponds to the end time of the effective display time interval. Using a time interval that is closed before the frame and open after the frame ensures that adjacent frames to be displayed have a clear affiliation at the time boundaries.

[0037] For a frame to be displayed that does not yet have a subsequent frame to be displayed in the frame buffer queue, since the end time of its effective display time interval cannot be determined based on the adjacent frames to be displayed, the complete effective display time interval of the frame to be displayed can be determined after the corresponding subsequent frame to be displayed enters the frame buffer queue and obtains the synchronization time mark. The frame to be displayed with a complete effective display time interval can then be used to determine the subsequent candidate frames to be displayed.

[0038] By converting the frame to be displayed from a single time stamp into a valid display time interval, the subsequent synchronization control process can make a judgment based on the time coverage relationship between multiple display windows, rather than simply matching based on whether multiple media streams have the same time stamp.

[0039] After determining the effective display time interval for each frame to be displayed, the multimedia broadcast control equipment further corrects the effective display time interval according to the display processing procedure corresponding to the display window, so as to obtain a time range that is more consistent with the actual display output state.

[0040] S103, corrects the effective display time interval based on the display link delay.

[0041] In this embodiment, after determining the effective display time interval of the frame to be displayed for each display window, the processing differences of different display windows in the actual display process are further considered.

[0042] Specifically, the media content and display processing of different display windows may differ, resulting in a time offset between the time indicated by the synchronization time stamp of the frame to be displayed in each display window and the actual start time of display output. Therefore, each display window has its own corresponding display link delay.

[0043] If the effective display time intervals corresponding to each frame to be displayed are used directly for multi-window synchronization, it can only reflect the time relationship of the media content itself, but cannot reflect the offset relationship between the actual presentation times of different display windows.

[0044] Therefore, in this embodiment, the multimedia broadcast control device obtains the display link delay corresponding to each display window and corrects the effective display time interval according to the display link delay of the corresponding display window.

[0045] Specifically, for any display window, the start and end times of the effective display time interval of the frame to be displayed corresponding to the display window are respectively increased by the display link delay corresponding to the display window to obtain the corrected effective display time interval of the frame to be displayed.

[0046] For example, if the effective display time interval of the frame to be displayed is [Ti, Ti+1), and the display link delay of the corresponding display window is Dj, then the corrected effective display time interval of the frame to be displayed can be expressed as [Ti+Dj, Ti+1+Dj). Here, the display link delay Dj represents the delay parameter determined based on multiple displayed reference frames of the corresponding display window, used to characterize the time offset of the current display processing process of the display window. Therefore, by adding the display link delay to the start and end times of the effective display time interval, the corrected time interval can characterize the time range corresponding to when the frame to be displayed actually participates in the display after passing through the corresponding display processing link.

[0047] Different display windows use their respective display link delays to perform the above corrections, so that the frames to be displayed in multiple display windows can be compared according to their display time relationship after passing through their respective display links under a unified synchronous time base.

[0048] Furthermore, when the display processing state corresponding to the display window changes, the display link delay may change. Therefore, the display link delay in this embodiment can be updated according to the current processing state of the display window to ensure that the corrected effective display time interval can reflect the current display processing process.

[0049] Using the above method, the effective display time interval of the frame to be displayed can be corrected by utilizing the display link delay corresponding to each display window, so that the matching of candidate frames to be displayed between multiple subsequent display windows can be based on the corrected effective display time interval.

[0050] It should be noted that the specific method for obtaining the display link latency and the update method under the condition of changes in display processing status will be further explained in subsequent embodiments.

[0051] S104, determine the common overlap interval between candidate frames to be displayed based on the effective display time interval of the correction.

[0052] In this embodiment, after correcting the effective display time intervals corresponding to each display window, the multimedia broadcast control device further determines candidate frames to be displayed from the frame buffer queues corresponding to multiple display windows.

[0053] Specifically, for each display window, the multimedia broadcast control device selects a candidate frame to be displayed from the corresponding frame buffer queue to participate in the judgment of the common overlapping interval.

[0054] Among them, the candidate frames to be displayed correspond to the effective display time interval after the display link delay correction.

[0055] The multimedia broadcast control equipment acquires the corrected effective display time intervals of candidate frames to be displayed corresponding to multiple display windows, and determines whether there are common overlapping intervals among the corrected effective display time intervals of multiple candidate frames to be displayed.

[0056] When multiple valid display time intervals for correction have a common overlapping interval, it indicates that multiple candidate frames to be displayed have a common displayable time range under a unified synchronization time base. At this time, multiple candidate frames to be displayed have a time basis as objects for synchronous refresh.

[0057] Furthermore, the multimedia broadcast control device retains the multiple candidate frames to be displayed, and continues to determine the synchronization refresh cycle based on the ready time and common overlapping interval of each candidate frame to be displayed.

[0058] When there is no common overlapping interval among multiple candidate frames to be displayed, it indicates that the candidate frames to be displayed currently selected from each frame buffer queue cannot be synchronously displayed within the same time range.

[0059] In response to the above situation, this embodiment updates at least one candidate frame to be displayed and re-executes the time range matching process between multiple candidate frames to be displayed.

[0060] Specifically, the multimedia broadcast control equipment re-acquires the corresponding corrected valid display time interval based on the updated candidate display frames, and then determines again whether multiple display windows form a common overlapping interval.

[0061] By performing the process of updating candidate frames to be displayed and re-determining the common overlapping interval, after multiple candidate frames to be displayed form a common overlapping interval, the synchronization refresh period is further determined based on the multiple candidate frames to be displayed.

[0062] In this embodiment, the common overlapping interval is determined based on the effective display time interval of multiple candidate frames to be displayed, thereby making the determination of the subsequent synchronous refresh cycle based on the fact that multiple candidate frames to be displayed have a common overlapping interval.

[0063] It should be noted that the specific method for determining the common overlapping interval and the candidate display frame update strategy when there is no common overlapping interval will be further explained in subsequent embodiments.

[0064] S105, determine the synchronous refresh cycle and target display time based on the candidate frames to be displayed.

[0065] In this embodiment, when the candidate frames to be displayed corresponding to multiple display windows form a common overlapping interval, it indicates that each candidate frame to be displayed has a common displayable time range under a unified synchronization time base.

[0066] However, in actual display processes, display devices refresh the screen according to the vertical synchronization cycle, and the update of the displayed content needs to be coordinated with the display time corresponding to the vertical synchronization signal. Therefore, after forming a common overlapping interval, it is also necessary to combine the ready times of multiple candidate frames to be displayed with the common overlapping interval to determine the synchronization refresh cycle.

[0067] Specifically, the multimedia broadcast control equipment acquires the ready time when multiple candidate frames to be displayed enter the refreshable and readable state under a unified synchronous time base, and determines the synchronous refresh cycle based on each ready time and the common overlapping interval.

[0068] The synchronous refresh cycle is a vertical synchronization cycle determined based on each ready time and the common overlapping interval, used for synchronous refresh of multiple display windows.

[0069] After determining the synchronization refresh cycle, the display time corresponding to the synchronization refresh cycle under the unified synchronization time base is determined as the target display time.

[0070] After determining the target display time, the plurality of candidate frames to be displayed are determined as the target display frames.

[0071] The target display frame is the media content frame that multiple display windows need to display together within the same refresh cycle.

[0072] In this embodiment, the synchronization frame is not determined solely based on the media content time, but rather, in addition to satisfying the media time correlation, the actual refresh conditions of the display device are further considered, so that the determined target display frame can be synchronously output in the real display environment.

[0073] It should be noted that the specific method for determining the synchronization refresh cycle, including the determination of the refresh search starting point, the determination of the first vertical synchronization cycle, and the correction of the display time of the first vertical synchronization cycle, will be further explained in subsequent embodiments.

[0074] S106, Generate synchronous refresh control items and perform overall latch refresh.

[0075] In this embodiment, after determining multiple target display frames, the multimedia broadcast control device further generates a synchronous refresh control item for controlling the synchronous updating of multiple display windows.

[0076] Specifically, the synchronous refresh control items include at least: The buffer address corresponding to the target display frame; Information about the display window area corresponding to the target display frame.

[0077] The buffer address is used to indicate the data storage location of the corresponding target display frame; the display window area information is used to determine the display area of ​​the corresponding target display frame.

[0078] The synchronous refresh control establishes a relationship with the target display frames corresponding to multiple display windows, enabling multiple display windows to output corresponding media content according to the determined display area within the same refresh cycle.

[0079] Furthermore, when the vertical synchronization signal corresponding to the synchronization refresh cycle arrives, the multimedia broadcast control equipment latches the synchronization refresh control item as a whole.

[0080] Among them, overall latch means that when the vertical synchronization signal corresponding to the synchronous refresh cycle arrives, the buffer addresses of multiple target display frames and the corresponding display window area information in the synchronous refresh control item will take effect together.

[0081] By latching the synchronization refresh control item as a whole when the vertical synchronization signal corresponding to the synchronization refresh cycle arrives, multiple display windows can display the target display frame corresponding to the same target display time in the same refresh cycle.

[0082] Compared to the method where each display window independently performs refresh based on the ready time of its own frame, this embodiment determines multiple target display frames and associates the refresh control process of multiple display windows with the same synchronous refresh cycle, which can reduce the content time offset caused by different refresh times of different display windows.

[0083] It should be noted that the specific generation method of the synchronous refresh control item and the refresh cycle selection process corresponding to the vertical synchronization signal can be adjusted according to the specific display device structure, and this embodiment does not limit this.

[0084] Through the above steps, this embodiment first determines the synchronization time marker of each frame to be displayed based on the mapping relationship between the media stream time base to which each frame to be displayed belongs and the unified synchronization time base, and determines the effective display time interval based on the synchronization time markers of adjacent frames to be displayed in the same frame buffer queue; secondly, the effective display time interval is corrected based on the display link delay corresponding to each display window, and candidate frames to be displayed and the common overlap interval of multiple candidate frames to be displayed are determined from each frame buffer queue; subsequently, the synchronization refresh cycle and target display time are determined based on the ready time of multiple candidate frames to be displayed under the unified synchronization time base and the common overlap interval, and multiple candidate frames to be displayed are determined as target display frames; finally, a synchronization refresh control item containing the buffer address of multiple target display frames and the corresponding display window area information is generated, and the synchronization refresh control item is latched as a whole when the vertical synchronization signal corresponding to the synchronization refresh cycle arrives, so that multiple display windows display the target display frame corresponding to the same target display time in the same refresh cycle.

[0085] Through the above control process, the time misalignment of cross-window content caused by multiple display windows refreshing according to their own frame processing status can be reduced, so that media content with time correlation can maintain the corresponding time relationship between multiple display windows.

[0086] Example 2 like Figure 2 As shown, based on Embodiment 1, this embodiment provides a method for obtaining and updating display link latency, which is used to determine the display link latency corresponding to each display window in a multimedia broadcast control device.

[0087] During the operation of multimedia broadcast control equipment, different display windows may have different window sizes, scaling ratios, display levels, or display processing paths, which may cause differences in the time difference between the time indicated by the synchronization time stamp of the reference frame already displayed in each display window and the actual display time.

[0088] Therefore, in this embodiment, the corresponding display link delay is obtained for each display window, and the effective display time interval of the corresponding display window is corrected by the display link delay, so that the frames to be displayed corresponding to different display windows can be subsequently synchronized according to the corrected time relationship.

[0089] Specifically, the process of obtaining the display link latency in this embodiment includes the following steps.

[0090] S201, obtain the synchronization time stamp and actual display time corresponding to the displayed reference frame.

[0091] In this embodiment, the multimedia broadcast control device first selects a displayed reference frame from multiple display frames that have already been displayed and output by each display window.

[0092] Among them, the displayed reference frame is the reference frame that the corresponding display window has completed display output and is used to determine the display delay of a single frame.

[0093] For any given display window, the multimedia broadcast control device selects multiple displayed reference frames and obtains the synchronization time stamp corresponding to each displayed reference frame. The synchronization time stamp is a time stamp determined according to the mapping relationship between the media stream time base and the unified synchronization time base in Embodiment 1, and is used to indicate the time position of the corresponding displayed reference frame under the unified synchronization time base.

[0094] Furthermore, the multimedia broadcast control equipment records the actual display time when each displayed reference frame begins to be displayed and output through its corresponding display window, according to a unified synchronization time base.

[0095] Specifically, when the display device performs refresh output according to the vertical synchronization cycle, when the vertical synchronization signal corresponding to the synchronous refresh cycle arrives and latches the synchronous refresh control item containing the buffer address of the displayed reference frame and the display window area information as a whole, the display time of the synchronous refresh cycle under the unified synchronization time base is recorded, and the recorded display time is used as the actual display time of the corresponding displayed reference frame.

[0096] Since the actual display time and the synchronization time stamp corresponding to the displayed reference frame are both represented according to a unified synchronization time base, the time difference between the two can be directly determined to determine the single-frame display delay of the corresponding displayed reference frame.

[0097] By obtaining the synchronization time stamp and actual display time of each displayed reference frame, the time difference between the time indicated by the synchronization time stamp and the actual display time can be determined, providing a data basis for subsequently determining the display delay of a single frame.

[0098] S202 determines the single-frame display delay based on the synchronization time stamp and the actual display time.

[0099] In this embodiment, for each displayed reference frame, the multimedia broadcast control device determines the corresponding single-frame display delay based on the actual display time and synchronization time stamp of the displayed reference frame.

[0100] Specifically, the display delay of a single frame corresponding to the displayed reference frame is obtained by subtracting the time represented by the synchronization time stamp of the displayed reference frame from the actual display time of the displayed reference frame.

[0101] Using the above method, the display delay of a single frame corresponding to multiple displayed reference frames can be obtained respectively.

[0102] Since the display latency of a single displayed reference frame may vary due to factors such as the current processing task and system scheduling status, this embodiment does not directly use the display latency of a single frame as the display link latency. Instead, it comprehensively determines the display latency based on the display latency of multiple consecutive displayed reference frames.

[0103] S203 determines the display link latency based on the display latency of multiple single frames.

[0104] In this embodiment, the multimedia broadcast control device acquires a preset number of consecutive single-frame display delays, calculates the average value of the preset number of consecutive single-frame display delays, and determines the average value as the display link delay of the corresponding display window.

[0105] By calculating the average value of a preset number of consecutive single-frame display delays, the impact of single-frame display delay variations on the display link delay determination results can be reduced.

[0106] Furthermore, during the continuous operation of the multimedia broadcast control equipment, when a new single-frame display delay is obtained, the earliest single-frame display delay among the currently calculated single-frame display delays is removed in chronological order, and the new single-frame display delay is added to maintain the number of consecutive single-frame display delays used to determine the display link delay at the preset number; the average value of the updated preset number of consecutive single-frame display delays is calculated, and the average value is re-determined as the display link delay of the corresponding display window.

[0107] The above method enables the display link latency to be dynamically updated as the display processing changes.

[0108] S204, redetermine the display link delay based on the change in display processing status.

[0109] In this embodiment, when updating only based on historical single-frame display latency, if the display processing state of the display window changes, the single-frame display latency obtained before the state change may not reflect the current display processing state.

[0110] Therefore, this embodiment further obtains display processing status information that affects the display window processing process.

[0111] Specifically, the processing status information displayed includes: Window size; Scaling ratio; Display hierarchy; Displays the processing path identifier.

[0112] The multimedia broadcast control equipment combines the above display processing status information according to the preset field order to generate a display processing status identifier for the current status of the corresponding display window.

[0113] The display processing status identifier is used to characterize the display processing configuration status corresponding to the current display window.

[0114] Furthermore, the multimedia broadcast control equipment compares the display processing status identifier corresponding to the currently displayed reference frame with the display processing status identifier corresponding to the previous displayed reference frame that is adjacent in time sequence.

[0115] When two display processing status identifiers are consistent, it is determined that the currently displayed reference frame corresponds to the same display processing status as the previous displayed reference frame that is adjacent in time sequence.

[0116] When the display processing status identifier of the currently displayed reference frame is inconsistent with the display processing status identifier of the previous displayed reference frame that is adjacent in time sequence, the multimedia broadcast control device determines the actual display time corresponding to the currently displayed reference frame as the state switching time, and stops using the single-frame display delay corresponding to the displayed reference frame before the state switching time to determine the current display link delay.

[0117] Subsequently, the multimedia broadcast control device obtains a preset number of consecutive single-frame display delays from the displayed reference frames whose display processing status identifier is the same as that of the currently displayed reference frame after the state switching time, and redetermines the display link delay of the corresponding display window based on the preset number of consecutive single-frame display delays.

[0118] By using the above method, the mixed statistics of display delay data before and after the change of display window processing state can be avoided, making the obtained display link delay more consistent with the current display processing state.

[0119] S205, determine the number of displayed link latency statistics based on latency fluctuations.

[0120] In this embodiment, the multimedia broadcast control device determines the preset number of data points for calculating the display link latency based on the single-frame display latency fluctuation under the current display processing status identifier.

[0121] Specifically, the multimedia broadcast control device obtains the display delay of a preset number of consecutive single frames under the current display processing status identifier, and determines the maximum and minimum display delay of the consecutive single frames.

[0122] Calculate the difference between the maximum single-frame display latency and the minimum single-frame display latency, and determine the difference as the latency fluctuation.

[0123] When the delay fluctuation is not greater than a preset fluctuation threshold, the first sampling quantity is determined to be the preset quantity; when the delay fluctuation is greater than the preset fluctuation threshold, the second sampling quantity is determined to be the preset quantity, wherein the second sampling quantity is greater than the first sampling quantity.

[0124] The multimedia broadcast control device acquires and determines a preset number of consecutive single-frame display delays, calculates the average value of the consecutive single-frame display delays, and determines the average value as the display link delay of the corresponding display window.

[0125] Using the above method, the preset quantity used to calculate the display link latency can be determined based on the latency fluctuation under the current display processing status indicator in the display window.

[0126] The display link latency acquisition and update method provided in this embodiment can determine the corresponding display link latency for different display windows, and adjust the latency calculation process by utilizing changes in display processing status and latency fluctuations.

[0127] Compared to setting fixed display delay parameters, this embodiment can dynamically obtain the display link delay based on the actual display processing, so that when subsequent effective display time interval correction is performed based on the display link delay, it can more accurately reflect the actual display time relationship of different display windows.

[0128] Therefore, in the process of synchronously refreshing multiple display windows, the time offset caused by the differences in the processing paths of different display windows can be reduced, and the time consistency of media content associated with multiple windows can be improved.

[0129] Example 3 like Figure 3 As shown, based on Embodiment 1 and Embodiment 2, this embodiment provides a method for matching candidate frames to be displayed in multiple display windows based on a common overlapping interval, which is used to determine candidate frames to be displayed that have a common time range from the frame buffer queues corresponding to multiple display windows.

[0130] In multimedia broadcast control equipment, the synchronization time stamps and display link delays of the frames to be displayed in different display windows may differ. Therefore, the effective display time intervals corresponding to multiple time-related frames to be displayed may not be completely consistent.

[0131] Therefore, this embodiment determines whether there is a common overlapping interval between candidate frames to be displayed corresponding to multiple display windows based on the corrected effective display time interval after display link delay correction, providing a time range basis for determining the subsequent synchronous refresh cycle.

[0132] S301, determine the candidate frames to be displayed from each frame buffer queue.

[0133] In this embodiment, the multimedia broadcast control device maintains frame buffer queues corresponding to multiple display windows.

[0134] Each frame buffer queue stores multiple frames to be displayed, and each frame to be displayed is associated with a corresponding synchronization time stamp, an effective display time interval, and a correction effective display time interval.

[0135] When multi-display window synchronous refresh control is required, the multimedia broadcast control equipment determines the candidate frames to be displayed from the frame buffer queues corresponding to each display window.

[0136] Specifically, for any display window, the multimedia broadcast control device determines a candidate frame to be displayed from the frame buffer queue according to the order of the synchronization time stamps of the frames to be displayed in the corresponding frame buffer queue.

[0137] Furthermore, the multimedia broadcast control equipment acquires the corrected effective display time intervals corresponding to multiple candidate frames to be displayed, in order to determine whether multiple corrected effective display time intervals have a common overlapping interval.

[0138] S302, determine the common overlapping interval of multiple candidate frames to be displayed.

[0139] In this embodiment, the multimedia broadcast control device acquires the corrected effective display time intervals corresponding to multiple candidate frames to be displayed.

[0140] For each valid display time interval of the correction, obtain the corresponding start time and end time respectively.

[0141] The multimedia broadcast control equipment determines the latest start time among multiple valid correction display time intervals as the overlap determination start time and the earliest end time among multiple valid correction display time intervals as the overlap determination end time.

[0142] When the start time of overlap determination is earlier than the end time of overlap determination, the time range between the start time and the end time of overlap determination is determined as the common overlap interval corresponding to multiple candidate frames to be displayed.

[0143] When the start time of overlap determination is not earlier than the end time of overlap determination, it is determined that multiple candidate frames to be displayed do not have a common overlapping interval.

[0144] The common overlap interval is used to characterize the common time range of the corrected effective display time interval of multiple candidate frames to be displayed under a unified synchronization time base, and serves as the basis for subsequently determining the time range of the synchronization refresh cycle.

[0145] In this way, the synchronization relationship between multiple display windows is changed from a single point-in-time comparison to a time range matching, so that the synchronization judgment can adapt to the situation where there is a time offset between different media streams.

[0146] S303, Perform subsequent processing based on the determination result of the common overlapping interval.

[0147] In this embodiment, when it is determined that multiple candidate frames to be displayed have a common overlapping interval, the multimedia broadcast control device retains the current multiple candidate frames to be displayed and continues to perform the subsequent synchronization refresh cycle determination process based on the common overlapping interval.

[0148] When it is determined that multiple candidate frames to be displayed do not have a common overlapping interval, the multimedia broadcast control device updates at least one candidate frame to be displayed and redetermines the common overlapping interval based on the updated candidate frame to be displayed.

[0149] S304, Update candidate frames to be displayed when there are no common overlapping intervals.

[0150] In this embodiment, when multiple candidate frames to be displayed do not have a common overlapping interval, the multimedia broadcast control device determines the latest start time in the corrected effective display time interval of the multiple candidate frames to be displayed.

[0151] Subsequently, the effective display time interval for correction with an end time earlier than or equal to the latest start time is determined as the mismatch time interval.

[0152] Since the end time of the mismatch time interval is earlier than or equal to the latest start time among multiple corrected effective display time intervals, the mismatch time interval cannot form a common overlapping interval with the corrected effective display time interval with the latest start time, and the candidate frames to be displayed corresponding to the mismatch time interval need to be updated.

[0153] For each identified mismatch time interval, the multimedia broadcast control equipment replaces the corresponding candidate frame to be displayed with the next frame to be displayed in the corresponding frame buffer queue along the increasing direction of the synchronization time mark.

[0154] For each identified mismatch time interval, the multimedia broadcast control equipment replaces the corresponding candidate frame to be displayed with the next frame to be displayed in the corresponding frame buffer queue along the increasing direction of the synchronization time mark, while keeping other candidate frames to be displayed that are not identified as mismatch time intervals in the effective display time interval unchanged.

[0155] S305, redetermine the common overlapping interval.

[0156] After replacing the candidate frames to be displayed corresponding to the mismatch time interval, the multimedia broadcast control equipment redetermines the common overlapping interval based on the multiple candidate frames to be displayed after replacement.

[0157] When multiple corrected valid display time intervals have a common overlapping interval, the current multiple candidate frames to be displayed are retained, and the common overlapping interval is used to determine the subsequent synchronous refresh cycle.

[0158] If the multiple corrected effective display time intervals still do not have a common overlapping interval, the mismatch time interval is determined based on the latest start time among the multiple corrected effective display time intervals, and the corresponding candidate display frames are updated to redetermine the common overlapping interval.

[0159] It should be noted that after determining that multiple candidate frames to be displayed have a common overlapping range, it is also necessary to further determine the synchronization refresh cycle by combining the ready time of each candidate frame to be displayed and the vertical synchronization cycle of the display device. The specific determination method will be explained in subsequent embodiments.

[0160] Using the candidate display frame matching method provided in this embodiment, the multimedia broadcast control device determines a common overlapping interval based on the corrected effective display time intervals of multiple candidate display frames. When there is no common overlapping interval, it determines a mismatch time interval based on the latest start time among the multiple corrected effective display time intervals, replaces the candidate display frame corresponding to the mismatch time interval with the next display frame in the corresponding frame buffer queue along the increasing direction of the synchronization time mark, and then redetermines the common overlapping interval based on the replaced candidate display frame.

[0161] Therefore, while retaining candidate frames to be displayed that have not been identified as mismatch time intervals, the candidate frames to be displayed corresponding to the mismatch time intervals can be updated, providing a basis for determining the synchronous refresh cycle based on the common overlapping interval and the ready time of the candidate frames to be displayed.

[0162] Example 4 like Figure 4As shown, based on Embodiments 1 to 3, this embodiment provides a multi-display window refresh control method based on synchronous refresh cycle, which is used to determine the actual refresh timing that can be performed for multi-window synchronous refresh according to the time relationship of candidate frames to be displayed and the refresh characteristics of the display device.

[0163] In multimedia broadcast control equipment, even if multiple candidate frames to be displayed have formed a common overlapping area, it does not mean that the display update can be completed at any time.

[0164] The reason is that display devices typically refresh the screen according to the vertical synchronization cycle, and the switching of displayed content needs to be coordinated with the corresponding vertical synchronization signal.

[0165] Therefore, it is necessary to further determine the synchronous refresh cycle that matches the refresh cycle of the display device within the common overlapping interval corresponding to the candidate frames to be displayed, so that multiple display windows can complete synchronous updates in the actual display environment.

[0166] S401, determine the refresh search starting point based on the candidate frames to be displayed.

[0167] In this embodiment, when multiple candidate frames to be displayed form a common overlapping interval, the multimedia broadcast control device obtains the ready time of each candidate frame to be displayed under a unified synchronization time base.

[0168] The ready time refers to the time corresponding to the candidate frame to be displayed when it has completed the necessary processing for display output and entered the refreshable and readable state, under the unified synchronization time base.

[0169] Since the ready times of multiple candidate frames to be displayed may be different, the multimedia broadcast control device determines the latest ready time among the multiple candidate frames to be displayed, and determines the later of the latest ready time and the start time of the common overlapping interval as the refresh search starting point.

[0170] Therefore, the refresh search start point is no earlier than the latest ready time of multiple candidate frames to be displayed, and no earlier than the start time of the common overlapping interval. Subsequently, the vertical synchronization period that meets the conditions is further determined starting from the refresh search start point.

[0171] S402, ensures the display time is no earlier than the first vertical synchronization cycle of refreshing the search starting point.

[0172] Under a unified synchronization time base, the multimedia broadcast control equipment obtains the reference display time of the reference vertical synchronization period, wherein the reference display time is no later than the refresh search starting point.

[0173] The multimedia broadcast control equipment acquires the display times of two adjacent vertical synchronization cycles under a unified synchronization time base, and determines the difference between the two display times as the duration of the vertical synchronization cycle.

[0174] Furthermore, the time difference between the refresh search start point and the reference display time is calculated, and the ratio of the time difference to the vertical synchronization cycle duration is rounded up to obtain the cycle offset.

[0175] Subsequently, the product of the cycle offset and the vertical synchronization cycle duration is added to the reference display time to obtain the candidate display time, and the vertical synchronization cycle corresponding to the candidate display time is determined as the first vertical synchronization cycle before the display time is earlier than the refresh search starting point.

[0176] S403 corrects the display time of the first vertical synchronization cycle based on the displayed phase deviation.

[0177] In this embodiment, the multimedia broadcast control device obtains the predicted display time determined by the reference display time and the duration of the vertical synchronization cycle in the previous vertical synchronization cycle, and records the actual display time of the previous vertical synchronization cycle according to the unified synchronization time base.

[0178] Calculate the difference between the actual display time and the predicted display time, and determine the difference as the display phase deviation.

[0179] When the absolute value of the displayed phase deviation is not greater than the preset phase deviation threshold, the displayed phase deviation is increased to the reference display time to obtain the correction reference display time.

[0180] When the absolute value of the displayed phase deviation is greater than the preset phase deviation threshold, the actual display time of the previous vertical synchronization cycle is determined as the correction reference display time.

[0181] Further, the time difference between the refresh search start point and the correction reference display time is calculated, the ratio of the time difference to the vertical synchronization cycle duration is rounded up to obtain the correction cycle offset, and the product of the correction cycle offset and the vertical synchronization cycle duration is added to the correction reference display time to obtain the correction candidate display time.

[0182] The vertical synchronization period corresponding to the correction candidate display time is determined as the first vertical synchronization period, and the correction candidate display time is determined as the display time of the first vertical synchronization period under a unified synchronization time base.

[0183] S404 determines the synchronization refresh cycle based on the first vertical synchronization cycle.

[0184] After determining the first vertical synchronization cycle and its display time under a unified synchronization time base, the multimedia broadcast control equipment compares the display time of the first vertical synchronization cycle with the end time of the common overlapping interval.

[0185] When the display time of the first vertical synchronization cycle is earlier than the end time of the common overlap interval, since the display time of the first vertical synchronization cycle is not earlier than the refresh search start point, and the refresh search start point is not earlier than the start time of the common overlap interval and the latest ready time of multiple candidate frames to be displayed, the first vertical synchronization cycle is determined as the synchronous refresh cycle, and the display time of the first vertical synchronization cycle under the unified synchronization time base is determined as the target display time.

[0186] After determining the synchronization refresh cycle and target display time, multiple candidate frames to be displayed that participate in the current synchronization refresh are identified as the target display frames.

[0187] If the display time of the first vertical synchronization cycle is not earlier than the end time of the common overlapping interval, and there is no vertical synchronization cycle with a display time not earlier than the refresh search start point in the common overlapping interval corresponding to the current multiple candidate frames to be displayed, then the corresponding candidate frames to be displayed are updated, and the common overlapping interval is re-determined.

[0188] S405, update the candidate frames to be displayed when the display time of the first vertical synchronization cycle is no earlier than the end time of the common overlap interval.

[0189] When the display time of the first vertical synchronization cycle is not earlier than the end time of the common overlap interval, the multimedia broadcast control equipment determines the candidate frame to be displayed whose end time of the corrected effective display time interval is equal to the end time of the common overlap interval, and determines the corresponding frame buffer queue as the frame buffer queue to be advanced.

[0190] The multimedia broadcast control equipment sequentially acquires the subsequent frames to be displayed in the frame buffer queue along the increasing direction of the synchronization time mark, and determines the corrected effective display time interval of each subsequent frame to be displayed.

[0191] The first subsequent frame to be displayed whose end time of the effective display time interval is later than the display time of the first vertical synchronization cycle is identified as a new candidate frame to be displayed.

[0192] The corresponding candidate frames to be displayed are replaced with the new candidate frames to be displayed, while keeping the candidate frames to be displayed in the remaining frame buffer queue unchanged, and the common overlapping interval of multiple candidate frames to be displayed is re-determined.

[0193] After the common overlap interval is re-established, the synchronization refresh cycle is re-determined based on the ready time of the updated multiple candidate frames to be displayed under the unified synchronization time base and the re-determined common overlap interval.

[0194] S406, generate synchronous refresh control items and perform overall latching.

[0195] After determining the synchronization refresh cycle, target display time, and multiple target display frames, the multimedia broadcast control device generates a synchronization refresh control item, which includes the buffer addresses of multiple target display frames and the corresponding display window area information.

[0196] The buffer address of each target display frame is used to determine the data storage location of the corresponding target display frame, and the display window area information is used to determine the display area corresponding to the target display frame.

[0197] When the vertical synchronization signal corresponding to the synchronous refresh cycle arrives, the multimedia broadcast control device latches the synchronous refresh control item as a whole, so that the buffer addresses of multiple target display frames and the corresponding display window area information in the synchronous refresh control item take effect together under the same vertical synchronization signal.

[0198] Therefore, multiple display windows display the target display frame corresponding to the same target display time in the same refresh cycle.

[0199] Through the synchronous refresh cycle determination and refresh control process provided in this embodiment, the multimedia broadcast control device determines the refresh search starting point based on the ready time of multiple candidate frames to be displayed under a unified synchronization time base, after multiple candidate frames to be displayed have formed a common overlapping interval, and determines the display time not earlier than the first vertical synchronization cycle of the refresh search starting point according to the vertical synchronization cycle of the display device.

[0200] Furthermore, the display time of the first vertical synchronization cycle under the unified synchronization time base is determined based on the reference display time, the duration of the vertical synchronization cycle, and the display phase deviation, and this display time is compared with the end time of the common overlapping interval.

[0201] When the display time of the first vertical synchronization cycle is earlier than the end time of the common overlap interval, the first vertical synchronization cycle is determined as the synchronization refresh cycle, and its display time is determined as the target display time; when the display time of the first vertical synchronization cycle is not earlier than the end time of the common overlap interval, the candidate frames to be displayed whose end time of the corrected effective display time interval is equal to the end time of the common overlap interval are updated, and the common overlap interval and synchronization refresh cycle are re-determined.

[0202] When the vertical synchronization signal corresponding to the synchronous refresh cycle arrives, the synchronous refresh control item containing the buffer address of multiple target display frames and the corresponding display window area information is latched as a whole, so that multiple display windows display the target display frame corresponding to the same target display time in the same refresh cycle.

[0203] Based on the aforementioned embodiments, this embodiment uses a digital exhibition hall central control system as a specific application scenario to illustrate the method for synchronously refreshing multiple display windows of multimedia broadcast control equipment.

[0204] In this embodiment, the digital exhibition hall central control system includes a multimedia broadcast control device and multiple display windows controlled by the multimedia broadcast control device. Each display window is used to display media content with a time correlation.

[0205] In this embodiment, multiple display windows are set in the display interface of the display device controlled by the multimedia broadcast control device. Each display window forms a different logical display area according to its corresponding display window area information, and is used to display media content with time correlation. The multimedia broadcast control device controls the media content and display area corresponding to each display window.

[0206] For example, during an exhibition, the multimedia broadcast control equipment can control: The main visual window is used to display the overall exhibition of the exhibits; An animation display window is used to show animations of structural changes in exhibits; The explanation information window is used to display the corresponding explanation content; The auxiliary information window is used to display text or status information.

[0207] Although the aforementioned multiple display windows each carry different media content, their content all corresponds to the same display process node, and therefore need to be presented synchronously within the same time frame.

[0208] S501: Obtain the frames to be displayed from multiple display windows in the digital exhibition hall.

[0209] In this embodiment, the multimedia broadcast control device acquires multiple media streams and maps each media stream to a corresponding display window according to the content to be displayed.

[0210] The multimedia broadcast control equipment acquires the frames to be displayed for each display window and stores each frame to be displayed in the corresponding frame buffer queue.

[0211] Because the media content and display processing procedures for different display windows may differ, the times when each frame to be displayed enters the refreshable and readable state may also differ. When each display window refreshes according to the ready time of its own frame to be displayed, multiple display windows belonging to the same display process may display media content corresponding to different time ranges in the same refresh cycle.

[0212] S502, determine the synchronization time stamp of the frame to be displayed and the effective display time interval for correction.

[0213] In this embodiment, the multimedia broadcast control device determines the synchronization time marker corresponding to each frame to be displayed based on the mapping relationship between the media stream time base to which each frame to be displayed belongs and the unified synchronization time base.

[0214] Furthermore, the effective display time interval of each frame to be displayed is determined based on the synchronization time stamp of adjacent frames to be displayed in the same frame buffer queue, and the display link delay corresponding to each display window is obtained. The effective display time interval of each frame to be displayed is corrected based on the corresponding display link delay to obtain the corresponding corrected effective display time interval.

[0215] Therefore, the frames to be displayed corresponding to different display windows in the digital exhibition hall can make subsequent time range judgments based on the corrected effective display time interval under a unified synchronous time base.

[0216] S503, determine candidate frames to be displayed that have a common overlapping region.

[0217] In this embodiment, the multimedia broadcast control device determines candidate frames to be displayed from the frame buffer queues corresponding to multiple display windows, and obtains the corrected effective display time intervals corresponding to the multiple candidate frames to be displayed.

[0218] The multimedia broadcast control equipment determines the common overlapping interval based on multiple corrected effective display time intervals.

[0219] If no common overlapping interval exists, update at least one candidate frame to be displayed, and redetermine the common overlapping interval based on the updated candidate frame to be displayed.

[0220] After forming a common overlapping interval, multiple candidate frames to be displayed are retained, and the common overlapping interval is used to determine the subsequent synchronous refresh cycle.

[0221] S504 determines the synchronization refresh cycle and performs synchronous refresh of multiple display windows.

[0222] After forming a common overlapping interval, the multimedia broadcast control equipment obtains the ready time of multiple candidate frames to be displayed under a unified synchronization time base, and determines the synchronization refresh cycle based on each ready time and the common overlapping interval.

[0223] The display time of the synchronous refresh cycle under a unified synchronous time base is determined as the target display time, and multiple candidate frames to be displayed that participate in the current synchronous refresh are determined as the target display frames.

[0224] The multimedia broadcast control equipment generates a synchronous refresh control item containing the buffer addresses of multiple target display frames and the corresponding display window area information.

[0225] When the vertical synchronization signal corresponding to the synchronous refresh cycle arrives, the multimedia broadcast control equipment latches the synchronous refresh control item as a whole, so that multiple display windows display the target display frame corresponding to the same target display time in the same refresh cycle.

[0226] In this digital exhibition hall scenario, the main visual window, the display animation window, the explanatory information window, and the auxiliary information window each display their respective target display frames, and update the corresponding display content in the same refresh cycle.

[0227] For example, during the display of exhibit structural changes, when the main visual window, the display animation window, the explanatory information window, and the auxiliary information window each have their corresponding target display frames, the multimedia broadcast control equipment latches the synchronization refresh control item as a whole when the vertical synchronization signal corresponding to the synchronization refresh cycle arrives, so that each display window displays the target display frame corresponding to the same target display time in the same refresh cycle.

[0228] By using the above-mentioned digital exhibition hall application methods, the situation where different display windows display content of different time ranges in the same refresh cycle due to different ready times of the display frames and different display link delays can be reduced, thereby improving the time consistency between multiple display windows with time correlation.

[0229] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A method for synchronously refreshing multiple display windows in a multimedia broadcast control device, characterized in that, The method includes: Get the frames to be displayed from multiple display windows, determine the synchronization time stamp of each frame to be displayed according to the mapping relationship between the media stream time base to which each frame to be displayed belongs and the unified synchronization time base, and store each frame to be displayed into the corresponding frame buffer queue. The effective display time interval of each frame to be displayed is determined based on the synchronization time stamp of adjacent frames to be displayed in the same frame buffer queue; Obtain the display link delay of each display window, and correct the effective display time interval of each frame to be displayed according to the corresponding display link delay to obtain the corrected effective display time interval; Candidate frames to be displayed are determined from each frame buffer queue, the common overlapping interval of the corrected effective display time interval of multiple candidate frames to be displayed is determined, and at least one candidate frame to be displayed is updated when there is no common overlapping interval, so as to redetermine the common overlapping interval. The readiness times of multiple candidate frames to be displayed under a unified synchronization time base are obtained. The synchronization refresh period is determined based on each readiness time and the common overlapping interval. The display time of the synchronization refresh period under the unified synchronization time base is determined as the target display time, and the multiple candidate frames to be displayed are determined as the target display frames. Generate a synchronization refresh control item containing the buffer addresses of multiple target display frames and the corresponding display window area information. When the vertical synchronization signal corresponding to the synchronization refresh cycle arrives, the synchronization refresh control item is latched as a whole so that multiple display windows display the target display frames corresponding to the same target display time in the same refresh cycle.

2. The method for synchronously refreshing multiple display windows of a multimedia broadcast control device according to claim 1, characterized in that, Obtain the display link latency for each display window, including: Select multiple displayed reference frames in the corresponding display window, obtain the synchronization time stamp of each displayed reference frame, and record the actual display time when each displayed reference frame starts to be displayed and output through the corresponding display window according to the unified synchronization time base; Subtract the time indicated by the corresponding synchronization time stamp from the actual display time of each displayed reference frame to obtain the single-frame display delay of each displayed reference frame; The average value of a preset number of consecutive single-frame display delays is determined as the display link delay of the corresponding display window; when a new single-frame display delay is obtained, the earliest single-frame display delay is removed in chronological order and the new single-frame display delay is added, and the display link delay is re-determined.

3. The method for synchronously refreshing multiple display windows of a multimedia broadcast control device according to claim 2, characterized in that, Obtaining the display link latency for each display window also includes: Obtain the window size, scaling ratio, display level, and display processing path identifier of the display window corresponding to each displayed reference frame, and combine the window size, scaling ratio, display level, and display processing path identifier into a display processing status identifier according to the preset field order; Compare the display processing status flag of the currently displayed reference frame with the display processing status flag of the previous displayed reference frame that is adjacent in time sequence; When the two display processing status identifiers are inconsistent, the actual display time corresponding to the currently displayed reference frame is determined as the state switching time, and the use of the single-frame display delay corresponding to the displayed reference frame before the state switching time to determine the current display link delay is stopped. Obtain a preset number of consecutive single-frame display delays from the displayed reference frames that have the same display processing state identifier after the state switching time, and redetermine the display link delay of the corresponding display window based on the consecutive single-frame display delays.

4. The method for synchronously refreshing multiple display windows of a multimedia broadcast control device according to claim 3, characterized in that, Determining the preset quantity includes: Obtain the continuous single-frame display delay for a preset number of observations under the current display processing status identifier, and determine the maximum and minimum single-frame display delays among the continuous single-frame display delays; The difference between the maximum single-frame display latency and the minimum single-frame display latency is calculated to obtain the latency fluctuation. When the time delay fluctuation is not greater than the preset fluctuation threshold, the first sampling quantity is determined to be the preset quantity; When the time delay fluctuation exceeds a preset fluctuation threshold, the second sampling quantity is determined to be a preset quantity, wherein the second sampling quantity is greater than the first sampling quantity; Obtain the display latency of consecutive single frames that is consistent with the preset number, and determine the average value of the display latency of consecutive single frames as the display link latency of the corresponding display window.

5. The method for synchronously refreshing multiple display windows of a multimedia broadcast control device according to claim 1, characterized in that, Determine the common overlapping interval of the corrected effective display time intervals of multiple candidate frames to be displayed, including: The latest start time among multiple effective display time intervals for correction is determined as the overlap determination start time, and the earliest end time among multiple effective display time intervals for correction is determined as the overlap determination end time. When the start time of overlap determination is earlier than the end time of overlap determination, the time range between the start time of overlap determination and the end time of overlap determination is defined as the common overlap interval. If the start time of the overlap determination is not earlier than the end time of the overlap determination, it is determined that there is no common overlapping interval.

6. The method for synchronously refreshing multiple display windows of a multimedia broadcast control device according to claim 1, characterized in that, When there are no common overlapping intervals, update at least one candidate frame to be displayed, including: Determine the latest start time among the corrected effective display time intervals of multiple candidate frames to be displayed; The effective display time interval for correction with an end time earlier than or equal to the latest start time is defined as the mismatch time interval. Replace the candidate frame to be displayed corresponding to the mismatch time interval with the next frame to be displayed in the corresponding frame buffer queue along the increasing direction of the synchronization time mark, and redetermine the common overlap interval based on the replaced candidate frame to be displayed.

7. The method for synchronously refreshing multiple display windows of a multimedia broadcast control device according to claim 1, characterized in that, The synchronization refresh cycle is determined based on each ready time and the common overlapping interval, including: Determine the latest ready time of multiple candidate frames to be displayed, and determine the later of the latest ready time and the start time of the common overlapping interval as the refresh search starting point; The display time under the unified synchronization time base is determined according to the time sequence and is no earlier than the first vertical synchronization cycle of the refresh search starting point; When the display time of the first vertical synchronization cycle is earlier than the end time of the common overlapping interval, the first vertical synchronization cycle is determined as the synchronization refresh cycle, and its display time is determined as the target display time. If the display time of the first vertical synchronization cycle is not earlier than the end time of the common overlap interval, the candidate frame to be displayed whose end time of the corrected effective display time interval is equal to the end time of the common overlap interval is replaced with the subsequent frame to be displayed in the corresponding frame buffer queue, the common overlap interval is re-determined, and the synchronization refresh cycle is re-determined after the common overlap interval is formed.

8. A method for synchronously refreshing multiple display windows in a multimedia broadcast control device according to claim 7, characterized in that, The display time under the unified synchronization time base is determined according to time sequence, and is no earlier than the first vertical synchronization cycle of the refresh search starting point, including: Under a unified synchronization time base, the reference display time of the reference vertical synchronization cycle is obtained, wherein the reference display time is no later than the refresh search start point, and the duration of the vertical synchronization cycle is determined according to the difference between the display times of two adjacent vertical synchronization cycles in time order. Calculate the time difference between the refresh search start point and the reference display time, and round up the ratio of the time difference to the vertical synchronization cycle duration to obtain the cycle offset. The product of the cycle offset and the vertical synchronization cycle duration is added to the reference display time to obtain the candidate display time, and the vertical synchronization cycle corresponding to the candidate display time is determined as the first vertical synchronization cycle.

9. A method for synchronously refreshing multiple display windows in a multimedia broadcast control device according to claim 8, characterized in that, Determining that the display time under a unified synchronization time base, in chronological order, is no earlier than the first vertical synchronization cycle of the search starting point, also includes: Obtain the predicted display time determined by the reference display time and the duration of the vertical synchronization cycle in the previous vertical synchronization cycle, and record the actual display time of the previous vertical synchronization cycle according to the unified synchronization time base. The difference between the actual display time and the predicted display time is calculated to obtain the display phase deviation; When the absolute value of the displayed phase deviation is not greater than the preset phase deviation threshold, the displayed phase deviation will be increased to the reference display time to obtain the correction reference display time; When the absolute value of the displayed phase deviation is greater than the preset phase deviation threshold, the actual display time of the previous vertical synchronization cycle is determined as the correction reference display time. Calculate the time difference between the refresh search start point and the correction reference display time, round up the ratio of the time difference to the vertical synchronization cycle duration to obtain the correction cycle offset, and add the product of the correction cycle offset and the vertical synchronization cycle duration to the correction reference display time to obtain the correction candidate display time. The vertical synchronization period corresponding to the correction candidate display time is determined as the first vertical synchronization period, and the correction candidate display time is determined as the display time of the first vertical synchronization period under the unified synchronization time base.

10. A method for synchronously refreshing multiple display windows in a multimedia broadcast control device according to claim 7, characterized in that, When the display time of the first vertical synchronization cycle is not earlier than the end time of the common overlap interval, candidate frames to be displayed whose end time of the corrected effective display time interval is equal to the end time of the common overlap interval are replaced with subsequent frames to be displayed in the corresponding frame buffer queue, including: Determine the candidate frames to be displayed whose end time of the effective display time interval is equal to the end time of the common overlapping interval, and determine the corresponding frame buffer queue as the frame buffer queue to be advanced; The subsequent frames to be displayed in the buffer queue of the frames to be advanced are obtained sequentially along the increasing direction of the synchronization time mark, and the corrected effective display time interval of each subsequent frame to be displayed is determined. The first subsequent frame to be displayed whose end time of the effective display time interval is later than the display time of the first vertical synchronization cycle is identified as a new candidate frame to be displayed. Replace the corresponding candidate frame to be displayed with the new candidate frame to be displayed, while keeping the candidate frames to be displayed in the remaining frame buffer queue unchanged, redetermine the common overlapping interval, and redetermine the synchronization refresh period after the common overlapping interval is formed.

Images