Distributed video synchronous display method, device, computer equipment and storage medium

Abstract

The present application relates to a distributed video synchronous display method, device, computer equipment and storage medium, wherein the method comprises: encoding the collected target video frames through each distributed input node, and sending the encoded target video frames to the distributed output nodes; each target video frame has a synchronous display requirement; when the target video frame is in a waiting state, determining the actual minimum display delay time corresponding to the target video frame; determining all the actual minimum display delay times that have a difference with the preset display delay time that exceeds a preset threshold; if each actual minimum display delay time is less than the preset display delay time, controlling each target video frame to be synchronously displayed based on the maximum actual minimum display delay time, thereby solving the problem of being unable to ensure video synchronous display while reducing the display delay from the input node to the output node, realizing video synchronous display, and reducing the display delay from the input node to the output node.

Description

Technical Field

[0001] The present application relates to the field of video display technology, and in particular to a distributed video synchronous display method, device, computer equipment and storage medium. Background Art

[0002] In the existing distributed video display system, the synchronization signal system time and display frame information of the current display frame in the master node and the slave node are usually obtained, the information of each slave node is compared with the master node, and based on the display time offset of each slave node compared to the master node in the same frame, each slave node is adjusted separately to achieve the effect of synchronous display. However, the above method cannot reduce the display delay from the input node to the output node while ensuring the synchronous display of the video.

[0003] There is no effective solution to the problem in the related art that it is impossible to reduce the display delay from the input node to the output node while ensuring the synchronous display of the video. Summary of the invention

[0004] In this embodiment, a distributed video synchronous display method, apparatus, computer equipment and storage medium are provided to solve the problem in the related art that the system cannot reduce the display delay from the input node to the output node while ensuring the synchronous display of the video.

[0005] In a first aspect, a distributed video synchronization display method is provided in this embodiment, which is applicable to a distributed video synchronization display system; wherein the system includes a plurality of distributed input nodes and a plurality of distributed output nodes, and each of the distributed input nodes is connected to each of the distributed output nodes via a network; the method includes:

[0006] Encoding the collected target video frames through each of the distributed input nodes, and sending the encoded target video frames to the corresponding distributed output nodes; wherein each of the target video frames has a synchronous display requirement;

[0007] When it is detected that the target video frame on the distributed output node is in a state to be displayed, determining an actually achievable minimum display delay time corresponding to the target video frame;

[0008] Determine all the actually achievable minimum display delay times whose difference from the preset display delay time exceeds a preset threshold;

[0009] When each of the actually achievable minimum display delay times is less than the preset display delay time, the target video frames are controlled to be displayed synchronously based on the maximum actually achievable minimum display delay time among the actually achievable minimum display delay times.

[0010] In some embodiments, determining the actual achievable minimum display delay time corresponding to the target video frame includes:

[0011] Obtaining a capture completion timestamp of the target video frame from the supplementary enhancement information corresponding to the target video frame;

[0012] Determine the latest vertical synchronization signal timestamp corresponding to the target video frame;

[0013] The actual achievable minimum display delay time corresponding to the target video frame is determined based on the acquisition completion timestamp and the latest vertical synchronization signal timestamp.

[0014] In some embodiments, the determining of all the actually achievable minimum display delay times whose difference from the preset display delay time exceeds a preset threshold value includes:

[0015] Determine the difference between the actual achievable minimum display delay time and the current preset display delay time;

[0016] When the difference exceeds a preset threshold and the reference time corresponding to the target video frame is within a preset range, determining whether the actual achievable minimum display delay time is less than the preset display delay time;

[0017] In a case where the actual achievable minimum display delay time is less than the preset display delay time, all the actual achievable minimum display delay times whose differences from the preset display delay time exceed the preset threshold are determined.

[0018] In some embodiments, when the difference exceeds a preset threshold and the reference time corresponding to the target video frame is within a preset range, determining whether the actual achievable minimum display delay time is less than the preset display delay time includes:

[0019] When the difference between the actual achievable minimum display delay time and the preset display delay time exceeds the preset threshold, corresponding message information is generated; the message information includes the actual achievable minimum display delay time corresponding to the target video frame, the frame number, and the reference time corresponding to the frame number;

[0020] Uploading the message information to a preset window management node to determine whether the reference time is within a preset range; the window management node refers to the distributed output node set to uniformly manage all the distributed output nodes;

[0021] When the reference time is within the preset range, it is determined whether the actual achievable minimum display delay time is less than the preset display delay time.

[0022] In some embodiments, after determining whether the actual achievable minimum display delay time is less than the preset display delay time, the method further includes:

[0023] When the actual achievable minimum display delay time is greater than the preset display delay time, sending the actual achievable minimum display delay time to each of the distributed output nodes;

[0024] Based on the actual achievable minimum display delay time, the target video frames on each of the distributed output nodes are controlled to be displayed synchronously.

[0025] In some of the embodiments, after determining the actual achievable minimum display delay time corresponding to the target video frame, the method further includes:

[0026] In the case where the actual achievable minimum display delay time is greater than the preset display delay time, corresponding message information is generated; the message information includes the actual achievable minimum display delay time corresponding to the target video frame, the frame number, and the reference time corresponding to the frame number;

[0027] Uploading the message information to a preset window management node to determine whether the reference time is within a preset range; the window management node refers to the distributed output node set to uniformly manage all the distributed output nodes;

[0028] When the reference time is not within the preset range, updating the reference time, and sending the updated reference time to each of the distributed output nodes;

[0029] When the reference time is within the preset range, the actual achievable minimum display delay time is sent to each of the distributed output nodes, and the target video frames on each of the distributed output nodes are controlled to be displayed synchronously based on the actual achievable minimum display delay time.

[0030] In some of the embodiments, after sending the encoded target video frame to the corresponding distributed output node, the method further includes:

[0031] When it is detected that the target video frame on the distributed output node is in a state to be displayed, determining the preset display delay time corresponding to the target video frame;

[0032] Determine a first display time corresponding to the target video frame under the preset display delay time;

[0033] When the system time reaches the first display time and the window management node does not send a new display delay time to the distributed output nodes, the target video frames on each of the distributed output nodes are controlled to be displayed synchronously based on the preset display delay time.

[0034] In some of the embodiments, after sending the encoded target video frame to the corresponding distributed output node, the process further includes:

[0035] Determining a predicted display delay time corresponding to the target video frame based on different states currently in which the target video frame is located; the states of the target video frame include a waiting-to-receive state, a waiting-to-decode state, and a waiting-to-scale state;

[0036] When the predicted display delay time is greater than the preset display delay time and the reference time corresponding to the target video frame is within a preset range, updating the preset display delay time based on the predicted display delay time;

[0037] The predicted display delay time on each of the distributed output nodes is cleared.

[0038] In some embodiments, when the current state of the target video frame is a waiting-to-receive state, determining the predicted display delay time corresponding to the target video frame includes:

[0039] Obtaining a capture completion timestamp and a corresponding bitstream length of the target video frame from the supplementary enhancement information corresponding to the target video frame;

[0040] Determine a bitstream receiving start time, a bitstream transmission rate prediction value, a decoding time prediction value, and a scaling time prediction value corresponding to the target video frame;

[0041] The predicted display delay time corresponding to the target video frame is determined based on the acquisition completion timestamp, the bitstream reception start time, the bitstream transmission rate prediction value, the bitstream length, the decoding time prediction value and the scaling time prediction value.

[0042] In some of the embodiments, when the target video frame is currently in a to-be-decoded state, determining the predicted display delay time corresponding to the target video frame includes:

[0043] Obtaining a capture completion timestamp of the target video frame from the supplementary enhancement information corresponding to the target video frame;

[0044] Determine the bitstream reception end time, decoding time prediction value and scaling time prediction value corresponding to the target video frame;

[0045] The predicted display delay time corresponding to the target video frame is determined based on the acquisition completion timestamp, the code stream reception end time, the decoding time prediction value and the scaling time prediction value.

[0046] In some embodiments, when the current state of the target video frame is a to-be-scaled state, determining the predicted display delay time corresponding to the target video frame includes:

[0047] Obtaining a capture completion timestamp of the target video frame from the supplementary enhancement information corresponding to the target video frame;

[0048] Determine the decoding end time and the scaling time prediction value corresponding to the target video frame;

[0049] The predicted display delay time corresponding to the target video frame is determined based on the acquisition completion timestamp, the decoding end time and the scaling time prediction value.

[0050] In some of the embodiments, after determining the predicted display delay time corresponding to the target video frame, the method further includes:

[0051] When the predicted display delay time is greater than the current preset display delay time and the reference time is not within a preset range, updating the reference time and sending the updated reference time to each of the distributed output nodes;

[0052] The predicted display delay time on each of the distributed output nodes is cleared.

[0053] In some of the embodiments, obtaining the supplementary enhancement information corresponding to the target video frame includes:

[0054] After the distributed input node encodes the collected target video frame, generating supplementary enhancement information of the target video frame;

[0055] The supplemental enhancement information is sent to the corresponding distributed output node.

[0056] In some of the embodiments, determining a prediction value corresponding to a target video frame includes:

[0057] When the target video frame is not the first frame received by the distributed output node and it is detected that the preset prediction model has completed training, determine the image group in which the target video frame is located;

[0058] Determining the prediction model that is trained and complete corresponding to the image group; the prediction model corresponds to the category of the prediction value;

[0059] Each video frame in the image group is processed by the fully trained prediction model to obtain the prediction value corresponding to the target video frame.

[0060] In some of the embodiments, the prediction model uses a seasonal autoregressive integrated moving average model.

[0061] In some of the embodiments, after processing each video frame in the image group by the fully trained prediction model to obtain the prediction value corresponding to the target video frame, the method further includes:

[0062] determining an actual value corresponding to the predicted value;

[0063] determining a difference between the actual value and the predicted value for each video frame in the group of pictures;

[0064] When the cumulative sum of the difference values ​​is greater than a preset error threshold, the current prediction model is trained based on the actual value corresponding to each video frame in the image group to obtain the latest trained and complete prediction model.

[0065] In a second aspect, a distributed video synchronization display device is provided in this embodiment, which is applicable to a distributed video synchronization display system; wherein the system includes a plurality of distributed input nodes and a plurality of distributed output nodes, each of the distributed input nodes is connected to each of the distributed output nodes via a network; the device includes: an encoding module, a calculation module, a comparison module and a control module;

[0066] The encoding module is used to encode the collected target video frame through each of the distributed input nodes, and send the encoded target video frame to the corresponding distributed output node; wherein each of the target video frames has a synchronous display requirement;

[0067] The calculation module is used to determine the actual achievable minimum display delay time corresponding to the target video frame when detecting that the target video frame on the distributed output node is in a state to be displayed;

[0068] The comparison module is used to determine all the actual achievable minimum display delay times whose differences from the preset display delay time exceed a preset threshold;

[0069] The control module is used to control the synchronous display of each target video frame based on the maximum actual achievable minimum display delay time among the actual achievable minimum display delay times when each actual achievable minimum display delay time is less than the preset display delay time.

[0070] In a third aspect, a computer device is provided in this embodiment, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the distributed video synchronization display method described in the first aspect is implemented.

[0071] In a fourth aspect, a storage medium is provided in this embodiment, on which a computer program is stored, and when the program is executed by a processor, the distributed video synchronization display method described in the first aspect is implemented.

[0072] Compared with the related art, the distributed video synchronous display method, device, computer equipment and storage medium provided in this embodiment encode the collected target video frame through each distributed input node, and send the encoded target video frame to the corresponding distributed output node; wherein each target video frame has a synchronous display requirement; when it is detected that the target video frame on the distributed output node is in a waiting state, the actual achievable minimum display delay time corresponding to the target video frame is determined; all actual achievable minimum display delay times whose difference with the preset display delay time exceeds a preset threshold are determined; when each actual achievable minimum display delay time is less than the preset display delay time, based on the maximum actual achievable minimum display delay time among each actual achievable minimum display delay time, each target video frame is controlled to be synchronously displayed, thereby solving the problem of not being able to reduce the display delay from the input node to the output node while ensuring the synchronous display of the video, realizing the synchronous display of the video, and reducing the display delay from the input node to the output node.

[0073] Details of one or more embodiments of the present application are set forth in the following drawings and description to make other features, objects, and advantages of the present application more readily apparent. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0075] Figure 1 It is a hardware structure block diagram of a terminal device of a distributed video synchronization display method provided by an embodiment of the present application;

[0076] Figure 2 is a flow chart of a distributed video synchronization display method provided by an embodiment of the present application;

[0077] Figure 3 is a schematic diagram of a distributed video synchronization display system provided by an embodiment of the present application;

[0078] Figure 4is a flow chart of a distributed input node control method provided by an embodiment of the present application;

[0079] Figure 5 This is a schematic diagram of a distributed output node prediction data flow provided by an embodiment of the present application;

[0080] Figure 6 is a flow chart of a distributed output node control method provided by an embodiment of the present application;

[0081] Figure 7 is a flow chart of a distributed video synchronization display method provided by an embodiment of the present application;

[0082] Figure 8 It is a structural block diagram of a distributed video synchronization display device provided in one embodiment of the present application.

[0083] In the figure: 102, processor; 104, memory; 106, transmission device; 108, input and output device; 10, encoding module; 20, calculation module; 30, comparison module; 40, control module. DETAILED DESCRIPTION

[0084] In order to more clearly understand the purpose, technical solutions and advantages of the present application, the present application is described and illustrated below in conjunction with the accompanying drawings and embodiments.

[0085] Unless otherwise defined, the technical terms or scientific terms involved in this application shall have the general meaning understood by people with general skills in the technical field to which this application belongs. The words "one", "a", "the", "these" and the like in this application do not indicate a quantitative limitation, and they may be singular or plural. The terms "include", "comprise", "have" and any variants thereof involved in this application are intended to cover non-exclusive inclusions; for example, a process, method and system, product or device comprising a series of steps or modules (units) is not limited to the listed steps or modules (units), but may include unlisted steps or modules (units), or may include other steps or modules (units) inherent to these processes, methods, products or devices. The words "connect", "connected", "coupled" and the like involved in this application are not limited to physical or mechanical connections, but may include electrical connections, whether direct or indirect. The "multiple" involved in this application refers to two or more. "And / or" describes the association relationship of associated objects, indicating that there may be three relationships, for example, "A and / or B" may mean: A exists alone, A and B exist at the same time, and B exists alone. Generally, the character " / " indicates that the objects associated with each other are in an "or" relationship. The terms "first", "second", "third", etc. involved in this application are only used to distinguish similar objects and do not represent a specific ordering of the objects.

[0086] The method embodiment provided in this embodiment can be executed in a terminal, a computer or a similar computing device. For example, running on a terminal, Figure 1 FIG. 1 is a hardware structure diagram of a terminal of the distributed video synchronous display method of this embodiment. Figure 1 As shown, the terminal may include one or more ( Figure 1 Only one is shown in the figure) a processor 102 and a memory 104 for storing data, wherein the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA. The above terminal may also include a transmission device 106 and an input and output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0087] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the distributed video synchronous display method in this embodiment. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, to implement the above method. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0088] The transmission device 106 is used to receive or send data via a network. The above network includes a wireless network provided by a communication provider of the terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 can be a radio frequency (Radio Frequency, referred to as RF) module, which is used to communicate with the Internet wirelessly.

[0089] In this embodiment, a distributed video synchronization display method is provided. Figure 2 is a flow chart of the distributed video synchronous display method of this embodiment. Figure 2 As shown, the process includes the following steps:

[0090] Step S210, encoding the collected target video frames through each distributed input node, and sending the encoded target video frames to the corresponding distributed output nodes; wherein each target video frame has a synchronous display requirement;

[0091] Step S220, when it is detected that the target video frame on the distributed output node is in a state to be displayed, determining the actual minimum display delay time corresponding to the target video frame;

[0092] Step S230, determining all actual achievable minimum display delay times whose difference from the preset display delay time exceeds a preset threshold;

[0093] Step S240, when all actually achievable minimum display delay times are less than the preset display delay time, based on the maximum actually achievable minimum display delay time among the actually achievable minimum display delay times, control the target video frames to be displayed synchronously.

[0094] In this embodiment, if Figure 3 As shown, the distributed video synchronous display system includes an image signal input terminal, multiple distributed input nodes, multiple distributed output nodes and multiple display screens. Each distributed input node is connected to the image signal input terminal, for example, by using a high-definition multimedia interface (HDMI) and other methods. The image signal input terminal can be an ultra-high-resolution server or a computer, etc., for inputting image signals; each distributed input node is connected to each distributed output node through a network interactive machine, each distributed input node can be connected to multiple distributed output nodes, and each distributed output node can also be connected to multiple distributed input nodes; and each distributed output node is connected to a corresponding display screen, each display screen is usually multiple screens in the same video wall, and can also be a display screen of each part of the same projection fusion screen, and each display screen can synchronously display the same or different video content.

[0095] Specifically, the video synchronization display method in this embodiment is applicable to the above-mentioned distributed video synchronization display system. Figure 4 As shown, each distributed input node in the system collects a target video frame S401, records the acquisition completion timestamp of the target video frame, and the target video frame on each distributed input node has a synchronous display requirement; encodes the target video frame to effectively reduce the network bandwidth of data transmission between the distributed input node and the distributed output node, such as High Efficiency Video Coding (HEVC), Advanced Video Coding (AVC), etc., and records the encoding completion timestamp of the target video frame S402; determines whether the current distributed input node belongs to the acquisition synchronization group S403, and if it belongs to the acquisition synchronization group, calculates the frame sequence number of the distributed input node based on the acquisition synchronization group S404; generates supplemental enhancement information (SEI) of the target video frame, and the supplemental enhancement information includes the acquisition completion timestamp, encoding completion timestamp, frame sequence number, code stream sending timestamp and code stream length of the target video frame S405, adds the supplemental enhancement information to the starting position of the code stream, and then sends the encoded target video frame and its supplemental enhancement information to the corresponding distributed output node S406. Among them, the acquisition completion timestamp, encoding completion timestamp, and code stream sending timestamp are all recorded based on the system time.

[0096] Furthermore, the corresponding target video frame is received by the distributed output node, and operations such as decoding, scaling, and display control are performed on the received target video frame. When it is detected that the target video frame on the distributed output node is in a state to be displayed, the latest vertical synchronization signal timestamp corresponding to the target video frame is obtained, and the latest vertical synchronization signal timestamp is the time when the last vertical synchronization (Vertical Synchronization, Vsync) is triggered, and based on the acquisition completion timestamp and the latest vertical synchronization signal timestamp, the actual minimum display delay time corresponding to the target video frame is calculated, and the actual minimum display delay time refers to the time length required from the acquisition of the target video frame by the distributed input node to the display of the target video frame by the distributed output node.

[0097] Afterwards, for the actual minimum display delay time that can be achieved on each distributed output node, all actual minimum display delay times whose difference with the preset display delay time exceeds the preset threshold are screened out, and when all actual minimum display delay times that can be achieved are less than the preset display delay time, the maximum actual minimum display delay time that can be achieved among the actual minimum display delay times is determined, and based on the maximum actual minimum display delay time, each target video frame is controlled to be displayed synchronously, that is, a unified display time of each target video frame is determined according to the maximum actual minimum display delay time, so that each target video frame can be displayed synchronously according to the same display time, thereby ensuring the synchronous display while minimizing the actual minimum display delay time of each target video frame.

[0098] In the existing distributed video display system, the synchronization signal system time and display frame information of the current display frame in the master node and the slave node are usually obtained, the information of each slave node is compared with the master node, and based on the display time offset of each slave node compared to the master node in the same frame, each slave node is adjusted separately to achieve the effect of synchronous display. However, the above method cannot reduce the display delay from the input node to the output node while ensuring the synchronous display of the video.

[0099] Compared with the prior art, the present application encodes the collected target video frames through each distributed input node, and sends the encoded target video frames to the corresponding distributed output nodes; wherein each target video frame has a synchronous display requirement; when it is detected that the target video frame on the distributed output node is in a waiting-to-be-displayed state, the actual achievable minimum display delay time corresponding to the target video frame is determined; all actual achievable minimum display delay times whose differences with the preset display delay time exceed a preset threshold are determined; when each actual achievable minimum display delay time is less than the preset display delay time, each target video frame is controlled to be displayed synchronously based on the maximum actual achievable minimum display delay time among the actual achievable minimum display delay times. Based on this, when the actual minimum display delay time of the target frame on each output node is less than the preset display delay time, the synchronous display of each target frame is controlled according to the maximum actual minimum display delay time, so as to ensure the synchronous display of the video while reducing the display delay from the input node to the output node based on the real-time situation, solving the problem of not being able to reduce the display delay from the input node to the output node while ensuring the synchronous display of the video, realizing the synchronous display of the video and reducing the display delay from the input node to the output node.

[0100] In some embodiments, determining the actual achievable minimum display delay time corresponding to the target video frame in step S220 includes the following steps:

[0101] Step S221, obtaining a capture completion timestamp of the target video frame from the supplementary enhancement information corresponding to the target video frame;

[0102] Step S222, determining the latest vertical synchronization signal timestamp corresponding to the target video frame;

[0103] Step S223, based on the acquisition completion timestamp and the latest vertical synchronization signal timestamp, determine the actual minimum display delay time corresponding to the target video frame.

[0104] Specifically, the distributed output node receives the corresponding target video frame and the supplementary enhancement information corresponding to the target video frame, and performs operations such as decoding, scaling, and display control on the received target video frame. When the target video frame on the distributed output node is in a state to be displayed, the acquisition completion timestamp of the target video frame is obtained from the supplementary enhancement information corresponding to the target video frame, and the latest vertical synchronization signal timestamp corresponding to the target video frame is obtained at the same time. The latest vertical synchronization signal timestamp is the time of the last vertical synchronization trigger. Based on the acquisition completion timestamp and the latest vertical synchronization signal timestamp, the actual minimum display delay time corresponding to the target video frame is calculated. The specific calculation formula is as follows:

[0105]

[0106] In formula (1), Indicates the actual minimum display delay time that can be achieved corresponding to the target video frame; Indicates the latest vertical synchronization signal timestamp; Indicates the output frame rate of the output node; Indicates the timestamp of collection completion; Indicates the acquisition frame rate of the input node. The actual minimum display delay time refers to the time required from acquiring the target video frame from the distributed input node to displaying the target video frame on the distributed output node.

[0107] Through this embodiment, the acquisition completion timestamp of the target video frame is obtained from the supplementary enhancement information corresponding to the target video frame, and the latest vertical synchronization signal timestamp corresponding to the target video frame is determined. Then, based on the acquisition completion timestamp and the latest vertical synchronization signal timestamp, the actual minimum display delay time corresponding to the target video frame is determined, thereby realizing accurate calculation of the actual minimum display delay time.

[0108] In some embodiments, the step S230 of determining all actual achievable minimum display delay times whose difference with the preset display delay time exceeds the preset threshold value comprises the following steps:

[0109] Step S231, determining the difference between the actual minimum display delay time and the current preset display delay time;

[0110] Step S232, when the difference exceeds a preset threshold and the reference time corresponding to the target video frame is within a preset range, determining whether the actual achievable minimum display delay time is less than a preset display delay time;

[0111] Step S233, when the actual achievable minimum display delay time is less than the preset display delay time, determine all actual achievable minimum display delay times whose differences from the preset display delay time exceed a preset threshold.

[0112] Specifically, the current preset display delay time is obtained, the difference between the actual minimum display delay time and the preset display delay time is calculated, and it is determined whether the calculated difference exceeds the preset threshold. If the difference exceeds the preset threshold, it is determined whether the reference time corresponding to the target video frame is within the preset range. The calculation formula of the reference time is as follows:

[0113]

[0114] In formula (2), t ref Indicates the base time; Indicates the display time of the currently displayed video frame; Indicates the frame number of the video frame being displayed; Indicates the frame number of the benchmark video frame sent for synchronous display; FPS disp Indicates the output frame rate of the output node; FPS cap Indicates the acquisition frame rate of the input node.

[0115] Furthermore, when the difference exceeds a preset threshold and the reference time corresponding to the target video frame is within a preset range, it is determined whether the actual achievable minimum display delay time is less than the preset display delay time. If the actual achievable minimum display delay time is less than the preset display delay time, all actual achievable minimum display delay times whose difference with the preset display delay time exceeds the preset threshold are screened out.

[0116] Through this embodiment, the difference between the actual achievable minimum display delay time and the current preset display delay time is determined. When the difference exceeds the preset threshold and the reference time corresponding to the target video frame is within the preset range, it is determined whether the actual achievable minimum display delay time is less than the preset display delay time. When the actual achievable minimum display delay time is less than the preset display delay time, all the actual achievable minimum display delay times whose differences with the preset display delay time exceed the preset threshold are determined. In this way, when the current actual achievable minimum display delay time is less than the preset display delay time, the actual achievable minimum display delay time with a larger difference with the preset display delay time is screened out, so that it is possible to further judge whether the actual achievable minimum display delay times are all less than the preset value among the screened actual achievable minimum display delay times, thereby ensuring the accuracy and stability of subsequent judgment results.

[0117] In some embodiments, when the difference value exceeds a preset threshold and the reference time corresponding to the target video frame is within a preset range, determining whether the actual achievable minimum display delay time is less than the preset display delay time in step S232 includes the following steps:

[0118] When the difference between the actual minimum display delay time and the preset display delay time exceeds a preset threshold, corresponding message information is generated; the message information includes the actual minimum display delay time corresponding to the target video frame, the frame number, and the reference time corresponding to the frame number;

[0119] Upload the message information to a preset window management node to determine whether the reference time is within a preset range; the window management node refers to a distributed output node that is set to uniformly manage all distributed output nodes;

[0120] When the reference time is within a preset range, it is determined whether the actual achievable minimum display delay time is less than the preset display delay time.

[0121] Specifically, when the difference between the actual minimum display delay time and the preset display delay time exceeds a preset threshold, a corresponding message information is generated, which includes the actual minimum display delay time corresponding to the target video frame, the frame number and the reference time corresponding to the frame number, and the message information is uploaded to the preset window management node.

[0122] Afterwards, it is determined whether the reference time is within the preset range. If the reference time is not within the preset range, the reference time is updated, and the window management node uses multicast to send the updated reference time to each distributed output node; if the reference time is within the preset range, it is determined whether the actual minimum display delay time is less than the preset display delay time, so that if the actual minimum display delay time is less than the preset display delay time, it is further determined whether the display delay time needs to be adjusted based on the current network load.

[0123] It should be noted that the window management node refers to a distributed output node that is set to uniformly manage all distributed output nodes. The window management node has the basic functions of other distributed output nodes and is used to receive video frames for output display, etc. When the difference between the actual minimum display delay time of the target video frame on the window management node and the preset display delay time exceeds the preset threshold, the window management node generates corresponding message information for recording and executes the above-mentioned processing steps for the message information.

[0124] Through this embodiment, when the difference between the actual minimum display delay time and the preset display delay time exceeds the preset threshold, corresponding message information is generated, and the message information includes the actual minimum display delay time corresponding to the target video frame, the frame number and the reference time corresponding to the frame number. The message information is uploaded to the preset window management node to determine whether the reference time is within the preset range. When the reference time is within the preset range, it is determined whether the actual minimum display delay time is less than the preset display delay time. In this way, the preset window management node is used to realize unified processing of the message information, and the complex calculation process is implemented on each distributed output node, so that the window management node will not reduce the real-time control as the scale of the distributed output nodes increases.

[0125] In some of the embodiments, after determining whether the actual achievable minimum display delay time is less than the preset display delay time, the video synchronization display method further includes the following steps:

[0126] When the actual minimum display delay time is greater than the preset display delay time, the actual minimum display delay time is sent to each distributed output node;

[0127] Based on the actual lowest display delay time that can be achieved, the target video frames on each distributed output node are controlled to be displayed synchronously.

[0128] Specifically, when the difference between the actual minimum display delay time and the preset display delay time exceeds the preset threshold, a corresponding message information is generated, the message information includes the actual minimum display delay time corresponding to the target video frame, the frame number and the reference time corresponding to the frame number, and the message information is uploaded to the preset window management node. On the window management node, if it is detected that the actual minimum display delay time is greater than the preset display delay time, the actual minimum display delay time is sent to each distributed output node, and the unified display time of each target video frame is determined based on the actual minimum display delay time, so as to control the target video frames on each distributed output node to be displayed synchronously.

[0129] It should be noted that in the waiting state, if it is detected that the actual minimum display delay time is greater than the preset display delay time, the target video frame on each distributed output node is controlled to be output and displayed according to the actual minimum display delay time to ensure synchronous display of the video.

[0130] Through this embodiment, when the actual minimum achievable display delay time is greater than the preset display delay time, the actual minimum achievable display delay time is sent to each distributed output node, and based on the actual minimum achievable display delay time, the target video frames on each distributed output node are controlled to be displayed synchronously, thereby ensuring the synchronization of video display controlled by each distributed output node.

[0131] In some of the embodiments, after determining the actual achievable minimum display delay time corresponding to the target video frame, the video synchronization display method further includes the following steps:

[0132] When the actual minimum display delay time is greater than the preset display delay time, corresponding message information is generated; the message information includes the actual minimum display delay time corresponding to the target video frame, the frame number, and the reference time corresponding to the frame number;

[0133] Upload the message information to a preset window management node to determine whether the reference time is within a preset range; the window management node refers to a distributed output node that is set to uniformly manage all distributed output nodes;

[0134] When the reference time is not within the preset range, the reference time is updated and the updated reference time is sent to each distributed output node;

[0135] When the reference time is within a preset range, the actual lowest display delay time that can be achieved is sent to each distributed output node, and the target video frames on each distributed output node are controlled to be displayed synchronously based on the actual lowest display delay time that can be achieved.

[0136] Specifically, after calculating the actual minimum display delay time, determine whether the actual minimum display delay time is greater than the preset display delay time. If the actual minimum display delay time is greater than the preset display delay time, generate corresponding message information, which includes the actual minimum display delay time corresponding to the target video frame, the frame number, and the reference time corresponding to the frame number.

[0137] Further, the message information is uploaded to a preset window management node to determine whether the reference time is within a preset range. If the reference time is not within the preset range, the reference time is updated, and the window management node uses a multicast method to send the updated reference time to each distributed output node;

[0138] When the reference time is within the preset range, the window management node uses multicast to send the actual minimum display delay time to each distributed output node, and controls the target video frames on each distributed output node for synchronous display based on the actual minimum display delay time. That is, each distributed output node displays the target video frame on the node according to the sent actual minimum display delay time.

[0139] It should be noted that the window management node refers to a distributed output node that is set to uniformly manage all distributed output nodes. The window management node has the basic functions of other distributed output nodes and is used to receive video frames for output display, etc. When the actual minimum display delay time that can be achieved on the window management node is greater than the preset display delay time, the window management node generates corresponding message information for recording and executes the above-mentioned processing steps for the message information.

[0140] Through this embodiment, if the actual minimum display delay time is greater than the preset display delay time, a corresponding message information is generated, and the message information is uploaded to the preset window management node to determine whether the reference time is within the preset range. When the reference time is not within the preset range, the reference time is updated, and the updated reference time is sent to each distributed output node. When the reference time is within the preset range, the actual minimum display delay time is sent to each distributed output node, and based on the actual minimum display delay time, the target video frame on each distributed output node is controlled to be displayed synchronously, so as to ensure the synchronization of video display when the actual minimum display delay time is greater than the preset display delay time.

[0141] In some of the embodiments, after sending the encoded target video frame to the corresponding distributed output node, the video synchronization display method further includes the following steps:

[0142] When it is detected that the target video frame on the distributed output node is in a state to be displayed, determining a preset display delay time corresponding to the target video frame;

[0143] Determine a first display time corresponding to the target video frame under a preset display delay time;

[0144] When the system time reaches the first display time and the window management node does not send a new display delay time to the distributed output nodes, the target video frames on each distributed output node are controlled to be displayed synchronously based on the preset display delay time.

[0145] Specifically, when the target video frame on the distributed output node is in a state to be displayed, the preset display delay time corresponding to the target video frame is determined, that is, the latest preset display delay time currently recorded by the distributed output node, and the first display time corresponding to the target video frame under the preset display delay time is determined, that is, the specific display time of the target video frame is calculated according to the preset display delay time.

[0146] Furthermore, when the system time reaches the first display time and the window management node does not send a new display delay time to the distributed output nodes, the target video frames on each distributed output node are controlled to be displayed synchronously based on the preset display delay time, that is, the target video frames on each distributed output node are controlled to be displayed synchronously at the first display time.

[0147] It should be noted that, in this embodiment, the situation where the window management node does not send a new display delay time to the distributed output nodes means that the window management node is not triggered to update the preset display delay time. For example, after determining all the actual minimum display delay times whose difference with the preset display delay time exceeds the preset threshold, it is determined whether each actual minimum display delay time is less than the current preset display delay time. If the number of nodes whose actual minimum display delay time is less than the preset display delay time does not meet the number of all output nodes, the window management node will not be triggered to send a new display delay time.

[0148] Through this embodiment, when it is detected that the target video frame on the distributed output node is in a state to be displayed, the preset display delay time corresponding to the target video frame is determined, and the first display time corresponding to the target video frame under the preset display delay time is determined. If the system time reaches the first display time, and the window management node does not send a new display delay time to the distributed output node, the target video frames on each distributed output node are controlled to be displayed synchronously based on the preset display delay time, so that each target video frame can be ensured to be displayed synchronously when only a single or a few target video frames correspond to the actual minimum display delay time that can be achieved is less than the preset value.

[0149] In some of the embodiments, after sending the encoded target video frame to the corresponding distributed output node, the following steps are also included:

[0150] Determine the predicted display delay time corresponding to the target video frame based on different states of the target video frame; the states of the target video frame include a waiting-to-receive state, a waiting-to-decode state, and a waiting-to-scale state;

[0151] When the predicted display delay time is greater than the preset display delay time and the reference time corresponding to the target video frame is within a preset range, updating the preset display delay time based on the predicted display delay time;

[0152] Removes the prediction display delay on each distributed output node.

[0153] It should be noted that the target video frame sent by the corresponding distributed input node is received by the distributed output node, and operations such as decoding, scaling, and display control are performed on the received target video frame at the distributed output node.

[0154] Specifically, based on different states of the target video frame currently, the predicted display delay time corresponding to the target video frame is calculated, wherein the states of the target video frame include a waiting-to-receive state, a waiting-to-decode state, and a waiting-to-scale state.

[0155] Further, it is determined whether the predicted display delay time is greater than the preset display delay time. If the predicted display delay time is greater than the preset display delay time, a corresponding message information is generated, which includes the predicted display delay time corresponding to the target video frame, the frame number, and the reference time corresponding to the frame number. The message information is uploaded to the preset window management node to determine whether the reference time is within the preset range. If the reference time is not within the preset range, the reference time is updated and the updated reference time is sent to each distributed output node; if the reference time is within the preset range, the preset display delay time is updated based on the predicted display delay time, and the predicted display delay time on each distributed output node is cleared.

[0156] Among them, the window management node refers to a distributed output node set to uniformly manage all distributed output nodes. The window management node has the basic functions of other distributed output nodes, and is used to receive video frames for output display, etc. When the predicted display delay time on the window management node is greater than the preset display delay time, the window management node generates corresponding message information for recording, and executes the above-mentioned processing steps for the message information.

[0157] Through this embodiment, based on the different states of the target video frame, the predicted display delay time corresponding to the target video frame is determined. When the predicted display delay time is greater than the preset display delay time and the reference time is within a preset range, the preset display delay time is updated based on the predicted display delay time, and the predicted display delay time on each distributed output node is cleared. In this way, the predicted display delay method is adopted to pre-offset the effects of network fluctuations, system fluctuations, decoding fluctuations or scaling fluctuations, so that early intervention and regulation can be performed before the video is displayed based on the actual situation to ensure the synchronization of the video display.

[0158] In some of the embodiments, when the current state of the target video frame is a waiting-to-receive state, determining the predicted display delay time corresponding to the target video frame includes the following steps:

[0159] Obtaining the acquisition completion timestamp and corresponding bitstream length of the target video frame from the supplementary enhancement information corresponding to the target video frame;

[0160] Determine the bitstream receiving start time, bitstream transmission rate prediction value, decoding time prediction value and scaling time prediction value corresponding to the target video frame;

[0161] Based on the acquisition completion timestamp, the bitstream reception start time, the bitstream length, the bitstream transmission rate prediction value, the decoding time prediction value and the scaling time prediction value, the predicted display delay time corresponding to the target video frame is determined.

[0162] Specifically, the network module in the distributed output node receives the network data and determines whether the network data contains the supplementary enhancement information corresponding to the target video frame. If the received network data contains the supplementary enhancement information, the supplementary enhancement information is parsed and stored in the structure, and the system time when the code stream is started to be received is recorded, that is, the start time of the code stream reception.

[0163] Furthermore, the bitstream reception start time corresponding to the target video frame is obtained, and the bitstream transmission rate prediction value, the decoding time prediction value, and the scaling time prediction value are calculated. Based on the acquisition completion timestamp, bitstream reception start time, bitstream length, bitstream transmission rate prediction value, decoding time prediction value, and scaling time prediction value of the target video frame, the predicted display delay time corresponding to the target video frame is calculated. The target video frame is the k+1th frame, and the specific calculation formula is as follows:

[0164]

[0165] In formulas (3) and (4), t pre_delay1 Indicates the predicted display delay time in the waiting state; t pre_delay_mid1 Indicates the middle value of the predicted display delay time in the waiting state, that is, the output frame rate of the unaligned output node; represents the predicted value of the bitstream transmission rate of the k+1th frame; S k+1 Indicates the length of the code stream of the k+1th frame; represents the predicted decoding time of the k+1th frame; Indicates the predicted value of the scaling time of the k+1th frame; Indicates the start time of receiving the code stream of the k+1th frame; Indicates the timestamp of the completion of the video data acquisition of the k+1th frame; FPS cap Indicates the acquisition frame rate of the input node; FPS disp Indicates the output frame rate of the output node.

[0166] Afterwards, when the distributed output node completes receiving the target video frame, the system time of receiving one frame of code stream is recorded, and the storage structure and the target video frame are transmitted to the decoding module in the distributed output node.

[0167] Through this embodiment, the acquisition completion timestamp and the corresponding bitstream length of the target video frame are obtained from the supplementary enhancement information corresponding to the target video frame, and the bitstream reception start time, bitstream transmission rate prediction value, decoding time prediction value and scaling time prediction value corresponding to the target video frame are determined. Then, based on the acquisition completion timestamp, bitstream reception start time, bitstream transmission rate prediction value, decoding time prediction value and scaling time prediction value, the predicted display delay time corresponding to the target video frame is determined, so as to accurately predict the display delay time of the target video frame in the waiting to be received state.

[0168] In some of the embodiments, when the current state of the target video frame is a to-be-decoded state, determining the predicted display delay time corresponding to the target video frame includes the following steps:

[0169] Obtaining a capture completion timestamp of the target video frame from the supplementary enhancement information corresponding to the target video frame;

[0170] Determine the code stream receiving end time, decoding time prediction value and scaling time prediction value corresponding to the target video frame;

[0171] Based on the acquisition completion timestamp, the code stream reception end time, the decoding time prediction value and the scaling time prediction value, the predicted display delay time corresponding to the target video frame is determined.

[0172] Specifically, when the decoding module receives the target video frame, the acquisition completion timestamp and the code stream receiving end time corresponding to the target video frame are obtained, and the decoding time prediction value and the scaling time prediction value are calculated. Based on the acquisition completion timestamp, the code stream receiving end time, the decoding time prediction value and the scaling time prediction value of the target video frame, the predicted display delay time corresponding to the target video frame is calculated. The target video frame is the k+1th frame. The specific calculation formula is as follows:

[0173]

[0174]

[0175] In formulas (5) and (6), t pre_delay2 Indicates the predicted display delay time in the state to be decoded; t pre_delay_mid2 Indicates the middle value of the predicted display delay time in the state to be decoded, that is, the output frame rate of the unaligned output node; represents the predicted decoding time of the k+1th frame; represents the predicted value of the scaling time of the k+1th frame; Indicates the end time of receiving the code stream of the k+1th frame; Indicates the timestamp of the completion of the video data acquisition of the k+1th frame; FPS cap Indicates the acquisition frame rate of the input node; FPS disp Indicates the output frame rate of the output node.

[0176] Afterwards, the decoding module decodes the code stream. When the distributed output node completes the decoding of the target video frame, it records the system time when the decoding of the target data frame starts and the system time when the decoding of the target data frame is completed, and transmits the storage structure and the target video frame to the scaling module in the distributed output node.

[0177] Through this embodiment, the acquisition completion timestamp of the target video frame is obtained from the supplementary enhancement information corresponding to the target video frame, the code stream reception end time, the decoding time prediction value and the scaling time prediction value corresponding to the target video frame are determined, and based on the acquisition completion timestamp, the code stream reception end time, the decoding time prediction value and the scaling time prediction value, the predicted display delay time corresponding to the target video frame is determined, so as to accurately predict the display delay time of the target video frame in the state to be decoded.

[0178] In some of the embodiments, when the current state of the target video frame is a to-be-scaled state, determining the predicted display delay time corresponding to the target video frame includes the following steps:

[0179] Obtaining a capture completion timestamp of the target video frame from the supplementary enhancement information corresponding to the target video frame;

[0180] Determine the decoding end time and scaling time prediction value corresponding to the target video frame;

[0181] Based on the acquisition completion timestamp, the decoding end time and the predicted value of the scaling time consumption, the predicted display delay time corresponding to the target video frame is determined.

[0182] Specifically, when the scaling module receives the target video frame, it obtains the acquisition completion timestamp and decoding end time corresponding to the target video frame, and calculates the predicted value of the scaling time consumption. Based on the acquisition completion timestamp, decoding end time and scaling time consumption predicted value of the target video frame, the predicted display delay time corresponding to the target video frame is calculated. The target video frame is the k+1th frame. The specific calculation formula is as follows:

[0183]

[0184] In formulas (7) and (8), t pre_delay3 Indicates the prediction display delay time in the state of waiting for zooming; t pre_delay_mid3 Indicates the middle value of the predicted display delay time in the state to be scaled, that is, the output frame rate of the unaligned output node; Represents the predicted value of the scaling time of the k+1th frame Indicates the actual end time of decoding of the k+1th frame; Indicates the timestamp of the completion of the video data acquisition of the k+1th frame; FPS cap Indicates the frame rate of the k+1th frame of video data acquisition; FPS disp Indicates the output frame rate of the output node.

[0185] Afterwards, the scaling module scales the code stream according to the corresponding display area. When the distributed output node completes the scaling process of the target video frame, it records the system time when the scaling of the target data frame starts and the system time when the scaling of the target data frame is completed, and transmits the storage structure and the scaled target video frame to the display module in the distributed output node.

[0186] Through this embodiment, the acquisition completion timestamp of the target video frame is obtained from the supplementary enhancement information corresponding to the target video frame, the decoding end time and the scaling time prediction value corresponding to the target video frame are determined, and based on the acquisition completion timestamp, the decoding end time and the scaling time prediction value, the predicted display delay time corresponding to the target video frame is determined, so as to accurately predict the display delay time of the target video frame in the state to be scaled.

[0187] In some of the embodiments, after determining the predicted display delay time corresponding to the target video frame, the video synchronization display method further includes the following steps:

[0188] When the predicted display delay time is greater than the current preset display delay time and the reference time is not within the preset range, the reference time is updated and the updated reference time is sent to each distributed output node;

[0189] Removes the prediction display delay on each distributed output node.

[0190] Specifically, after determining the predicted display delay time corresponding to the target video frame, determine whether the predicted display delay time is greater than the current preset display delay time. If the predicted display delay time is greater than the current preset display delay time, generate corresponding message information, which includes the predicted display delay time corresponding to the target video frame, the frame number and the reference time corresponding to the frame number.

[0191] Furthermore, the message information is uploaded to the preset window management node to determine whether the reference time corresponding to the current frame number is within the preset range. If the reference time is not within the preset range, the reference time is updated. The window management node uses multicast to send the updated reference time to each distributed output node and clear the predicted display delay time on each distributed output node.

[0192] Through this embodiment, when the predicted display delay time is greater than the current preset display delay time and the reference time is not within the preset range, the reference time is updated and the updated reference time is sent to each distributed output node. At the same time, the predicted display delay time on each distributed output node is cleared to ensure the synchronization of video display.

[0193] In some of the embodiments, obtaining the supplementary enhancement information corresponding to the target video frame includes the following steps:

[0194] After the distributed input node encodes the collected target video frame, the supplementary enhancement information of the target video frame is generated;

[0195] The supplementary enhancement information is sent to the corresponding distributed output nodes.

[0196] Specifically, after the distributed input node encodes the collected target video frame, the supplementary enhancement information of the target video frame is generated, and the supplementary enhancement information includes the collection completion timestamp, encoding completion timestamp, frame sequence number, code stream sending timestamp and code stream length of the target video frame.

[0197] Furthermore, the supplementary enhancement information is added to the starting position of the bitstream, and the encoded target video frame and its supplementary enhancement information are sent to the corresponding distributed output node, so that the acquisition completion timestamp of the target video frame can be obtained from the supplementary enhancement information on the distributed output node. Among them, the acquisition completion timestamp, encoding completion timestamp, and bitstream sending timestamp are all recorded based on the system time.

[0198] Through this embodiment, after the distributed input node encodes the collected target video frame, supplementary enhancement information of the target video frame is generated, and the supplementary enhancement information is sent to the corresponding distributed output node, thereby realizing stable and accurate transmission of information such as the collection completion timestamp and bitstream length.

[0199] See also Figure 5 ,The process of distributed output node prediction data is described in detail below.

[0200] In some of the embodiments, determining a prediction value corresponding to a target video frame comprises the following steps:

[0201] When the target video frame is not the first frame received by the distributed output node and it is detected that the preset prediction model has completed training, determine the image group where the target video frame is located;

[0202] Determining a well-trained prediction model corresponding to the image group; the prediction model corresponds to a category of the prediction value;

[0203] Each video frame in the image group is processed by training a complete prediction model to obtain the prediction value corresponding to the target video frame.

[0204] Specifically, receive the target video frame S501, determine whether the target video frame is the first frame received by the distributed output node S502, if the target video frame is the first frame received by the distributed output node, use the preset default value as the prediction value corresponding to the target video frame S503; if the target video frame is not the first frame received by the distributed output node, determine whether the current prediction model has completed training S504.

[0205] Among them, when the prediction model has not completed training, the actual value corresponding to the previous video frame of the target data frame is calculated as the prediction value of the target data frame. For example, when calculating the predicted value of the bitstream transmission rate corresponding to the target video frame, if the target video frame is not the first frame received by the distributed output node, and it is detected that the preset prediction model has not completed training, the actual value of the bitstream transmission rate corresponding to the previous video frame of the target data frame is calculated, and the calculated actual value of the bitstream transmission rate is used as the predicted value of the bitstream transmission rate of the target data frame S505.

[0206] When the prediction model has been trained, determine the image group where the target video frame is located, and obtain a fully trained prediction model for predicting and processing the image group. The prediction model corresponds to the prediction value category that needs to be calculated currently. For example, when calculating the decoding time prediction value, the decoding time prediction model is correspondingly retrieved, and each video frame in the image group is processed by the fully trained prediction model to obtain the prediction value S506 corresponding to the target video frame. Among them, the pre-built prediction model is trained based on the data frames received by the distributed output nodes. Specifically, the data in each received image group is used as the basic data unit for parameter training, that is, the amount of video frame data used for training is an integer multiple of the number of image groups. At the beginning frame of each image group, it is judged whether the prediction model has completed training. If the prediction model has completed training, the fully trained prediction model is directly called to perform prediction calculations on each data frame in the current image group. If the prediction model has not completed training, it is judged whether the current data amount meets the parameter training requirements of the prediction model. The current data amount refers to the number of all video frames received by the distributed output nodes. For example, it is required to reach k frames of data, and k frames are an integer multiple of the number of image groups. When the data amount meets the parameter training requirements of the prediction model, the prediction model is trained.

[0207] It should be noted that the prediction value in this embodiment refers to the predicted display delay time or the predicted value of the bit stream transmission rate required to actually achieve the minimum display delay time under each state, the predicted decoding time, and the predicted scaling time. The specific type of prediction value referred to is determined by the actual computing requirements.

[0208] Through this embodiment, when the target video frame is not the first frame received by the distributed output node and it is detected that the preset prediction model has completed training, the image group in which the target video frame is located is determined, and the trained prediction model corresponding to the image group is determined. The video frames in the image group are processed by the trained prediction model to obtain the prediction value corresponding to the target video frame, thereby improving the accuracy of the calculated prediction value.

[0209] In some of the embodiments, the forecasting model uses a seasonal autoregressive integrated moving average model.

[0210] In this embodiment, since the bit rate, decoding time and scaling time have a certain periodicity in the same device node, a seasonal autoregressive integrated moving average (SARIMA) model is preferably used as a prediction model to predict the bit rate, decoding time and scaling time of the same window.

[0211] Specifically, the code stream transmission rate prediction model is taken as an example to illustrate that the code stream transmission rate prediction model is constructed by using the SARIMA model, which uses the image group length of image coding as the periodic length S. The mathematical expression of the SARMIA model is as follows:

[0212]

[0213] In formula (9), represents the seasonal autoregressive polynomial; represents a nonseasonal autoregressive polynomial A sliding mean polynomial representing seasonality; represents the non-seasonal sliding mean polynomial; Represents white noise. The parameters to be trained in the SARMIA model include the order p of the non-seasonal autoregressive term, the order d of the non-seasonal difference, the order q of the non-seasonal moving average, the order P of the seasonal autoregressive term, the order D of the seasonal difference, and the order Q of the seasonal moving average.

[0214] Furthermore, for each video frame currently received by the distributed output node, the bitstream transmission rate corresponding to each data frame is calculated to obtain the actual data set of bitstream transmission rate {v t-k , …, v t-1}, through the actual data set of bitstream transmission rate, train the bitstream transmission rate prediction model and obtain the model parameters p v , d v ,q v , P v , D v , Q v , using the well-trained code stream transmission rate prediction model, calculate the transmission rate of each video frame in the next image group, from k+1 frame to k+S frame, respectively: The specific calculation formula for the code stream transmission rate is as follows:

[0215]

[0216] In formula (10), v vedio Indicates the actual bitstream transmission rate; The target video frame’s code stream receiving end timestamp; Indicates the start timestamp of receiving the target video frame’s code stream; size vedio Indicates the code stream size.

[0217] It should be noted that the construction and training process of the decoding time prediction model and the scaling time prediction model are the same as the above-mentioned bitstream transmission rate prediction model, wherein the decoding time is the difference between the decoding completion timestamp and the decoding start timestamp of the target video frame, and the scaling time is the difference between the scaling completion timestamp and the scaling start timestamp of the target video frame. In addition to the seasonal autoregressive integrated moving average algorithm, the autoregressive moving average algorithm, the long short-term memory network and other algorithms can be used for delay prediction. This embodiment does not specifically limit the algorithm used by the prediction model.

[0218] Through this embodiment, since the bit rate, decoding time and scaling time have certain periodicity, a seasonal autoregressive integrated moving average model is used as a prediction model to significantly improve the accuracy of prediction of each parameter.

[0219] In some of the embodiments, after processing each video frame in the image group by using a well-trained prediction model to obtain a prediction value corresponding to the target video frame, the following steps are also included:

[0220] determining actual values ​​corresponding to predicted values;

[0221] determining a difference between an actual value and a predicted value for each video frame in the group of images;

[0222] When the cumulative sum of the differences is greater than a preset error threshold, the current prediction model is trained based on the actual value corresponding to each video frame in the image group to obtain the latest fully trained prediction model.

[0223] Specifically, after using the prediction model to perform prediction processing on each video frame in the image group, the actual value corresponding to the predicted value is calculated. For example, the decoding time prediction value corresponding to each video frame is obtained through model prediction. When the video frame decoding is completed, the actual decoding time value of the video frame is obtained, that is, the difference between the decoding end time and the decoding start time of the video frame is calculated to obtain the actual decoding time value, and the difference between the actual value and the predicted value of each video frame is calculated.

[0224] Furthermore, the cumulative sum of the difference values ​​corresponding to each video frame in the image group is obtained to determine whether the cumulative sum is greater than a preset error threshold. If the cumulative sum of the difference values ​​is greater than the preset error threshold, the current prediction model is trained based on the actual value corresponding to each video frame in the image group to obtain the latest trained and complete prediction model, which is used to predict and calculate the prediction value of each video frame in the next image group; if the cumulative sum of the difference values ​​is less than or equal to the preset error threshold, the actual value corresponding to each video frame in the image group and the prediction model are used to perform prediction processing on each video frame in the next image group.

[0225] For example, the current image group contains video frames from the k+1th frame to the k+Sth frame, and the actual transmission rate v of the bitstream of the k+1th frame is calculated. k+1 and predicted transmission rate Error , calculate the error corresponding to each frame in turn, and then count the total cumulative error of the actual bitstream transmission rate of the current image group. The specific calculation formula is as follows:

[0226]

[0227] In formula (11), Indicates the total cumulative error of the actual bitstream transmission rate of the current picture group; Represents the error corresponding to the k+i-th frame, i∈(1,…,S).

[0228] Through this embodiment, the actual value corresponding to the predicted value is determined, and the difference between the actual value and the predicted value of each video frame in the image group is determined. When the cumulative sum of the differences is greater than the preset error threshold, the current prediction model is trained based on the actual value corresponding to each video frame in the image group to obtain the latest trained and complete prediction model, thereby improving the model performance and enabling the model to adapt to data changes, thereby improving the accuracy of model prediction.

[0229] The present embodiment is described and illustrated by means of specific examples below.

[0230] Figure 6 is a flow chart of the control method of the distributed output node of this embodiment. Figure 6 As shown, the method comprises the following steps:

[0231] When the network module in the distributed output node receives network data S601, it determines whether the data contains supplementary enhancement information corresponding to the target video frame. If the received network data contains supplementary enhancement information, the supplementary enhancement information is parsed and stored in a structure, and the system time when the code stream is received is recorded, that is, the code stream reception start time. The code stream reception start time corresponding to the target video frame is obtained, and the code stream transmission rate prediction value, the decoding time prediction value, and the scaling time prediction value are calculated. Based on the acquisition completion timestamp of the target video frame, the code stream reception start time, the code stream transmission rate prediction value, the decoding time prediction value, and the scaling time prediction value, the predicted display delay time corresponding to the target video frame in the waiting state is calculated S602. It is determined whether the predicted display delay time is greater than the preset display delay time S603. If the predicted display delay time is greater than the preset display delay time, the corresponding message information is generated and the message information is uploaded to the preset window management node S604.

[0232] Further, when the distributed output node completes the reception of the target video frame, the system time of receiving a frame of code stream is recorded, and the storage structure and the target video frame are transmitted to the decoding module in the distributed output node S605. When the decoding module receives the target video frame, the acquisition completion timestamp and code stream reception end time corresponding to the target video frame are obtained, the decoding time prediction value and the scaling time prediction value are calculated, and based on the acquisition completion timestamp, code stream reception end time, decoding time prediction value and scaling time prediction value of the target video frame, the predicted display delay time corresponding to the target video frame in the decoding state is calculated S606. It is determined whether the predicted display delay time is greater than the preset display delay time S607. If the predicted display delay time is greater than the preset display delay time, the corresponding message information is generated and the message information is uploaded to the preset window management node S608.

[0233] Afterwards, the decoding module decodes the code stream. When the distributed output node completes the decoding of the target video frame, it records the system time of starting to decode the target data frame and the system time of completing the decoding of the target data frame S609, and transmits the storage structure and the target video frame to the scaling module in the distributed output node. When the scaling module receives the target video frame, it obtains the acquisition completion timestamp and decoding end time corresponding to the target video frame, and calculates the predicted value of the scaling time consumption. Based on the acquisition completion timestamp, decoding end time and scaling time prediction value of the target video frame, the predicted display delay time corresponding to the target video frame is calculated S610. It is determined whether the predicted display delay time is greater than the preset display delay time S611. If the predicted display delay time is greater than the preset display delay time, the corresponding message information is generated and the message information is uploaded to the preset window management node S612.

[0234] The scaling module scales the code stream according to the corresponding display area. When the distributed output node completes the scaling process of the target video frame, it records the system time when the scaling of the target data frame starts and the system time when the scaling of the target data frame is completed S613, and transmits the storage structure and the scaled target video frame to the display module in the distributed output node.

[0235] In the waiting state, based on the acquisition completion timestamp and the latest vertical synchronization signal timestamp, the actual minimum display delay time corresponding to the target video frame is calculated S614. It is determined whether the actual minimum display delay time is greater than the preset display delay time S615. If the actual minimum display delay time is greater than the preset display delay time, the corresponding message information is generated and uploaded to the preset window management node S616; if the actual minimum display delay time is less than or equal to the preset display delay time, it is determined whether the difference between the actual minimum display delay time and the preset display delay time is greater than the preset threshold value S617. Among them, if the difference between the actual minimum display delay time and the preset display delay time is greater than the preset threshold value, the corresponding message information is generated and uploaded to the preset window management node S616; the display is controlled according to the preset display delay time issued by the window management node to synchronize the display output of each target video frame S618.

[0236] The present embodiment is described and illustrated below through preferred embodiments.

[0237] Figure 7 is a flow chart of the distributed video synchronous display method of the preferred embodiment, such as Figure 7 As shown, the distributed video synchronous display method includes the following steps:

[0238] Step S710, encoding the collected target video frame through each distributed input node, generating supplementary enhancement information of the target video frame, and sending the encoded target video frame and the supplementary enhancement information to the corresponding distributed output node; wherein each target video frame has a synchronous display requirement;

[0239] Step S720, when it is detected that the target video frame on the distributed output node is in a state to be displayed, the acquisition completion timestamp of the target video frame is obtained from the supplementary enhancement information corresponding to the target video frame, and the latest vertical synchronization signal timestamp corresponding to the target video frame is determined;

[0240] Step S730, determining the actual achievable minimum display delay time corresponding to the target video frame based on the acquisition completion timestamp and the latest vertical synchronization signal timestamp;

[0241] Step S740, when the difference between the actual minimum display delay time and the preset display delay time exceeds the preset threshold, generate corresponding message information, upload the message information to the preset window management node, and determine whether the reference time is within the preset range;

[0242] Step S750, when the reference time is within the preset range, determining whether the actual achievable minimum display delay time is less than the preset display delay time, if the actual achievable minimum display delay time is less than the preset display delay time, determining all actual achievable minimum display delay times whose difference with the preset display delay time exceeds a preset threshold;

[0243] Step S760, when all the actually achievable minimum display delay times are less than the preset display delay time, based on the maximum actually achievable minimum display delay time among the actually achievable minimum display delay times, control the target video frames to be displayed synchronously.

[0244] Through this embodiment, each distributed input node encodes the collected target video frame, generates supplementary enhancement information of the target video frame, and sends the encoded target video frame and the supplementary enhancement information to the corresponding distributed output node, and each target video frame has a synchronous display requirement. When it is detected that the target video frame on the distributed output node is in a state to be displayed, the acquisition completion timestamp of the target video frame is obtained from the supplementary enhancement information corresponding to the target video frame, and the latest vertical synchronization signal timestamp corresponding to the target video frame is determined. Based on the acquisition completion timestamp and the latest vertical synchronization signal timestamp, the actual minimum display delay time corresponding to the target video frame is determined.

[0245] Furthermore, when the difference between the actual minimum display delay time and the preset display delay time exceeds the preset threshold, the corresponding message information is generated, the message information is uploaded to the preset window management node, and it is determined whether the reference time is within the preset range. When the reference time is within the preset range, it is determined whether the actual minimum display delay time is less than the preset display delay time. If the actual minimum display delay time is less than the preset display delay time, all the actual minimum display delay times whose difference with the preset display delay time exceeds the preset threshold are determined, and when each actual minimum display delay time is less than the preset display delay time, based on the maximum actual minimum display delay time among the actual minimum display delay times, each target video frame is controlled to be displayed synchronously, which solves the problem of not being able to reduce the display delay from the input node to the output node while ensuring the synchronous display of the video, realizes the synchronous display of the video, and reduces the display delay from the input node to the output node.

[0246] It should be noted that the steps shown in the above process or the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0247] In this embodiment, a distributed video synchronization display device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions that have been made will not be repeated. The terms "module", "unit", "subunit", etc. used below can implement a combination of software and / or hardware of predetermined functions. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0248] Figure 8 is a structural block diagram of the distributed video synchronization display device of this embodiment, such as Figure 8 As shown, the device includes: an encoding module 10, a calculation module 20, a comparison module 30 and a control module 40;

[0249] The encoding module 10 is used to encode the collected target video frames through each distributed input node, and send the encoded target video frames to the corresponding distributed output nodes; wherein each target video frame has a synchronous display requirement;

[0250] The calculation module 20 is used to determine the actual minimum display delay time corresponding to the target video frame when it is detected that the target video frame on the distributed output node is in a state to be displayed;

[0251] A comparison module 30, for determining all actual achievable minimum display delay times whose difference with the preset display delay time exceeds a preset threshold;

[0252] The control module 40 is used to control the synchronous display of each target video frame based on the maximum actual achievable minimum display delay time among the actual achievable minimum display delay times when all actual achievable minimum display delay times are less than the preset display delay time.

[0253] Through the device provided by this embodiment, the collected target video frames are encoded by each distributed input node, and the encoded target video frames are sent to the corresponding distributed output nodes; wherein each target video frame has a synchronous display requirement; when it is detected that the target video frame on the distributed output node is in a waiting state, the actual achievable minimum display delay time corresponding to the target video frame is determined; all actual achievable minimum display delay times whose differences with the preset display delay time exceed a preset threshold are determined; when each actual achievable minimum display delay time is less than the preset display delay time, based on the maximum actual achievable minimum display delay time among the actual achievable minimum display delay times, each target video frame is controlled to be displayed synchronously, thereby solving the problem of not being able to reduce the display delay from the input node to the output node while ensuring the synchronous display of the video, realizing the synchronous display of the video, and reducing the display delay from the input node to the output node.

[0254] In some of the embodiments, the calculation module 20 is further used to obtain the acquisition completion timestamp of the target video frame from the supplementary enhancement information corresponding to the target video frame; determine the latest vertical synchronization signal timestamp corresponding to the target video frame; and determine the actual minimum display delay time corresponding to the target video frame based on the acquisition completion timestamp and the latest vertical synchronization signal timestamp.

[0255] In some of the embodiments, the comparison module 30 is further used to determine the difference between the actual minimum achievable display delay time and the current preset display delay time; when the difference exceeds a preset threshold and the reference time corresponding to the target video frame is within a preset range, determine whether the actual minimum achievable display delay time is less than the preset display delay time; when the actual minimum achievable display delay time is less than the preset display delay time, determine all actual minimum achievable display delay times whose difference with the preset display delay time exceeds the preset threshold.

[0256] In some of the embodiments, the comparison module 30 is also used to generate corresponding message information when the difference between the actual minimum display delay time and the preset display delay time exceeds a preset threshold; the message information includes the actual minimum display delay time corresponding to the target video frame, the frame number and the reference time corresponding to the frame number; the message information is uploaded to a preset window management node to determine whether the reference time is within a preset range; the window management node refers to a distributed output node set to uniformly manage all distributed output nodes; when the reference time is within the preset range, it is determined whether the actual minimum display delay time is less than the preset display delay time.

[0257] In some of the embodiments, the comparison module 30 is also used to send the actual minimum achievable display delay time to each distributed output node when the actual minimum achievable display delay time is greater than the preset display delay time; based on the actual minimum achievable display delay time, control the target video frames on each distributed output node to be displayed synchronously.

[0258] In some of the embodiments, the calculation module 20 is also used to generate corresponding message information when the actual minimum display delay time is greater than the preset display delay time; the message information includes the actual minimum display delay time corresponding to the target video frame, the frame number and the reference time corresponding to the frame number; the message information is uploaded to the preset window management node to determine whether the reference time is within the preset range; the window management node refers to a distributed output node set to uniformly manage all distributed output nodes; when the reference time is not within the preset range, the reference time is updated and the updated reference time is sent to each distributed output node; when the reference time is within the preset range, the actual minimum display delay time is sent to each distributed output node, and the target video frames on each distributed output node are controlled to be displayed synchronously based on the actual minimum display delay time.

[0259] In some of these embodiments, Figure 8 On the basis of, the device also includes a synchronization control module, which is used to determine the preset display delay time corresponding to the target video frame when it is detected that the target video frame on the distributed output node is in a waiting state; determine the first display time corresponding to the target video frame under the preset display delay time; when the system time reaches the first display time and the window management node does not send a new display delay time to the distributed output node, control the target video frame on each distributed output node to be displayed synchronously based on the preset display delay time.

[0260] In some of these embodiments, Figure 8 On the basis of, the device also includes an updating module, which is used to determine the predicted display delay time corresponding to the target video frame based on the different states currently in which the target video frame is located; the states of the target video frame include a waiting to be received state, a waiting to be decoded state and a waiting to be scaled state; when the predicted display delay time is greater than the preset display delay time and the reference time corresponding to the target video frame is within a preset range, the preset display delay time is updated based on the predicted display delay time; and the predicted display delay time on each distributed output node is cleared.

[0261] In some of these embodiments, Figure 8 On the basis of, the device also includes a prediction module, which is used to obtain the acquisition completion timestamp and the corresponding bitstream length of the target video frame from the supplementary enhancement information corresponding to the target video frame; determine the bitstream reception start time, bitstream transmission rate prediction value, decoding time prediction value and scaling time prediction value corresponding to the target video frame; based on the acquisition completion timestamp, bitstream reception start time, bitstream length, bitstream transmission rate prediction value, decoding time prediction value and scaling time prediction value, determine the predicted display delay time corresponding to the target video frame.

[0262] In some of these embodiments, Figure 8 On the basis of, the device also includes a prediction module, which is used to obtain the acquisition completion timestamp of the target video frame from the supplementary enhancement information corresponding to the target video frame; determine the code stream reception end time, decoding time prediction value and scaling time prediction value corresponding to the target video frame; based on the acquisition completion timestamp, the code stream reception end time, the decoding time prediction value and the scaling time prediction value, determine the predicted display delay time corresponding to the target video frame.

[0263] In some of these embodiments, Figure 8 On the basis of, the device also includes a prediction module, which is used to obtain the acquisition completion timestamp of the target video frame from the supplementary enhancement information corresponding to the target video frame; determine the decoding end time and scaling time prediction value corresponding to the target video frame; based on the acquisition completion timestamp, the decoding end time and the scaling time prediction value, determine the predicted display delay time corresponding to the target video frame.

[0264] In some of these embodiments, Figure 8 On the basis of, the device also includes a prediction module, which is used to update the reference time when the predicted display delay time is greater than the current preset display delay time and the reference time is not within the preset range, and send the updated reference time to each distributed output node; clear the predicted display delay time on each distributed output node.

[0265] In some of the embodiments, the encoding module 10 is further used to generate supplementary enhancement information of the target video frame after the collected target video frame is encoded at the distributed input node; and send the supplementary enhancement information to the corresponding distributed output node.

[0266] In some of these embodiments, Figure 8 On the basis of, the device also includes a prediction module, which is used to determine the image group where the target video frame is located when the target video frame is not the first frame received by the distributed output node and it is detected that the preset prediction model has been trained; determine the trained prediction model corresponding to the image group; the prediction model corresponds to the category of the prediction value; and process each video frame in the image group through the trained prediction model to obtain the prediction value corresponding to the target video frame.

[0267] In some of these embodiments, Figure 8 On the basis of, the device also includes a training module for determining an actual value corresponding to a predicted value; determining a difference between an actual value and a predicted value of each video frame in an image group; when a cumulative sum of the differences is greater than a preset error threshold, training the current prediction model based on the actual value corresponding to each video frame in the image group to obtain a newly trained and complete prediction model.

[0268] It should be noted that the above modules can be functional modules or program modules, and can be implemented by software or hardware. For modules implemented by hardware, the above modules can be located in the same processor; or the above modules can be located in different processors in any combination.

[0269] In this embodiment, a computer device is further provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any one of the above method embodiments.

[0270] Optionally, the computer device may further include a transmission device and an input / output device, wherein the transmission device is connected to the processor, and the input / output device is connected to the processor.

[0271] Optionally, in this embodiment, the processor may be configured to perform the following steps through a computer program:

[0272] S1, encoding the collected target video frame through each distributed input node, and sending the encoded target video frame to the corresponding distributed output node; wherein each target video frame has a synchronous display requirement;

[0273] S2, when it is detected that the target video frame on the distributed output node is in a state to be displayed, determining the actual minimum display delay time corresponding to the target video frame;

[0274] S3, determining all actual achievable minimum display delay times whose difference from the preset display delay time exceeds a preset threshold;

[0275] S4, when all actually achievable minimum display delay times are less than the preset display delay time, based on the maximum actually achievable minimum display delay time among the actually achievable minimum display delay times, controlling the target video frames to be displayed synchronously.

[0276] It should be noted that the specific examples in this embodiment can refer to the examples described in the above embodiments and optional implementation modes, and will not be repeated in this embodiment.

[0277] In addition, in combination with the distributed video synchronous display method provided in the above embodiments, a storage medium can also be provided in this embodiment to implement the distributed video synchronous display method. The storage medium stores a computer program; when the computer program is executed by a processor, any one of the distributed video synchronous display methods in the above embodiments is implemented.

[0278] It should be understood that the specific embodiments described herein are only used to explain the application, rather than to limit it. Based on the embodiments provided in this application, all other embodiments obtained by ordinary technicians in this field without creative work are within the protection scope of this application.

[0279] Obviously, the drawings are only some examples or embodiments of the present application. For ordinary technicians in the field, the present application can also be applied to other similar situations based on these drawings without creative work. In addition, it is understandable that although the work done in this development process may be complicated and lengthy, for ordinary technicians in the field, certain changes in design, manufacturing or production based on the technical content disclosed in this application are only conventional technical means and should not be regarded as insufficient content disclosed in this application.

[0280] The term "embodiment" in this application refers to a specific feature, structure or characteristic described in conjunction with the embodiment that can be included in at least one embodiment of the present application. The appearance of this phrase in various locations in the specification does not necessarily mean the same embodiment, nor does it mean that it is mutually exclusive with other embodiments and is independent or optional. It is clearly or implicitly understood by those of ordinary skill in the art that the embodiments described in this application can be combined with other embodiments without conflict.

[0281] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of patent protection. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the scope of protection of the present application. Therefore, the scope of protection of the present application shall be subject to the attached claims.

Claims

1. A distributed video synchronization display method, It is characterized in that Applicable to a distributed video synchronous display system; wherein the system comprises a plurality of distributed input nodes and a plurality of distributed output nodes, each of the distributed input nodes is connected to each of the distributed output nodes via a network; the method comprises: Encoding the collected target video frames through each of the distributed input nodes, and sending the encoded target video frames to the corresponding distributed output nodes; wherein each of the target video frames has a synchronous display requirement; When it is detected that the target video frame on the distributed output node is in a state to be displayed, determining an actually achievable minimum display delay time corresponding to the target video frame; Determine all the actually achievable minimum display delay times whose difference from the preset display delay time exceeds a preset threshold; When the actual achievable minimum display delay times whose differences with the preset display delay time exceed a preset threshold are all less than the preset display delay time, the target video frames are controlled to be displayed synchronously based on the maximum actual achievable minimum display delay time among the actual achievable minimum display delay times.

2. The distributed video synchronous display method according to claim 1, It is characterized in that The determining the actual achievable minimum display delay time corresponding to the target video frame includes: Obtaining a capture completion timestamp of the target video frame from the supplementary enhancement information corresponding to the target video frame; Determine the latest vertical synchronization signal timestamp corresponding to the target video frame; The actual achievable minimum display delay time corresponding to the target video frame is determined based on the acquisition completion timestamp and the latest vertical synchronization signal timestamp.

3. The distributed video synchronous display method according to claim 1, It is characterized in that The determining of all the actually achievable minimum display delay times whose difference with the preset display delay time exceeds a preset threshold value comprises: Determine the difference between the actual achievable minimum display delay time and the current preset display delay time; When the difference exceeds a preset threshold and the reference time corresponding to the target video frame is within a preset range, determining whether the actual achievable minimum display delay time is less than the preset display delay time; In a case where the actual achievable minimum display delay time is less than the preset display delay time, all the actual achievable minimum display delay times whose differences from the preset display delay time exceed the preset threshold are determined.

4. The distributed video synchronous display method according to claim 3, It is characterized in that When the difference exceeds a preset threshold and the reference time corresponding to the target video frame is within a preset range, determining whether the actual achievable minimum display delay time is less than the preset display delay time includes: When the difference between the actual achievable minimum display delay time and the preset display delay time exceeds the preset threshold, corresponding message information is generated; the message information includes the actual achievable minimum display delay time corresponding to the target video frame, the frame number, and the reference time corresponding to the frame number; Uploading the message information to a preset window management node to determine whether the reference time is within a preset range; the window management node refers to the distributed output node set to uniformly manage all the distributed output nodes; When the reference time is within the preset range, it is determined whether the actual achievable minimum display delay time is less than the preset display delay time.

5. The distributed video synchronous display method according to claim 3, It is characterized in that After determining whether the actual achievable minimum display delay time is less than the preset display delay time, the method further includes: When the actual achievable minimum display delay time is greater than the preset display delay time, sending the actual achievable minimum display delay time to each of the distributed output nodes; Based on the actual achievable minimum display delay time, the target video frames on each of the distributed output nodes are controlled to be displayed synchronously.

6. The distributed video synchronous display method according to claim 1, It is characterized in that After determining the actual achievable minimum display delay time corresponding to the target video frame, the method further includes: In the case where the actual achievable minimum display delay time is greater than the preset display delay time, corresponding message information is generated; the message information includes the actual achievable minimum display delay time corresponding to the target video frame, the frame number, and the reference time corresponding to the frame number; Uploading the message information to a preset window management node to determine whether the reference time is within a preset range; the window management node refers to the distributed output node set to uniformly manage all the distributed output nodes; When the reference time is not within the preset range, updating the reference time, and sending the updated reference time to each of the distributed output nodes; When the reference time is within the preset range, the actual achievable minimum display delay time is sent to each of the distributed output nodes, and the target video frames on each of the distributed output nodes are controlled to be displayed synchronously based on the actual achievable minimum display delay time.

7. The distributed video synchronous display method according to claim 4 or claim 6, It is characterized in that After sending the encoded target video frame to the corresponding distributed output node, the method further includes: When it is detected that the target video frame on the distributed output node is in a state to be displayed, determining the preset display delay time corresponding to the target video frame; Determine a first display time corresponding to the target video frame under the preset display delay time; When the system time reaches the first display time and the window management node does not send a new display delay time to the distributed output nodes, the target video frames on each of the distributed output nodes are controlled to be displayed synchronously based on the preset display delay time.

8. The distributed video synchronous display method according to claim 1, It is characterized in that After sending the encoded target video frame to the corresponding distributed output node, the method further includes: Determining a predicted display delay time corresponding to the target video frame based on different states currently in which the target video frame is located; the states of the target video frame include a waiting-to-receive state, a waiting-to-decode state, and a waiting-to-scale state; When the predicted display delay time is greater than the preset display delay time and the reference time corresponding to the target video frame is within a preset range, updating the preset display delay time based on the predicted display delay time; The predicted display delay time on each of the distributed output nodes is cleared.

9. The distributed video synchronous display method according to claim 8, It is characterized in that When the current state of the target video frame is a waiting-to-receive state, determining the predicted display delay time corresponding to the target video frame includes: Obtaining a capture completion timestamp and a corresponding bitstream length of the target video frame from the supplementary enhancement information corresponding to the target video frame; Determine a bitstream receiving start time, a bitstream transmission rate prediction value, a decoding time prediction value, and a scaling time prediction value corresponding to the target video frame; The predicted display delay time corresponding to the target video frame is determined based on the acquisition completion timestamp, the bitstream reception start time, the bitstream transmission rate prediction value, the bitstream length, the decoding time prediction value and the scaling time prediction value.

10. The distributed video synchronous display method according to claim 8, It is characterized in that When the target video frame is currently in a to-be-decoded state, determining the predicted display delay time corresponding to the target video frame includes: Obtaining a capture completion timestamp of the target video frame from the supplementary enhancement information corresponding to the target video frame; Determine the bitstream reception end time, decoding time prediction value and scaling time prediction value corresponding to the target video frame; The predicted display delay time corresponding to the target video frame is determined based on the acquisition completion timestamp, the code stream reception end time, the decoding time prediction value and the scaling time prediction value.

11. The distributed video synchronous display method according to claim 8, It is characterized in that When the current state of the target video frame is a to-be-scaled state, determining the predicted display delay time corresponding to the target video frame includes: Obtaining a capture completion timestamp of the target video frame from the supplementary enhancement information corresponding to the target video frame; Determine the decoding end time and the scaling time prediction value corresponding to the target video frame; The predicted display delay time corresponding to the target video frame is determined based on the acquisition completion timestamp, the decoding end time and the scaling time prediction value.

12. The distributed video synchronous display method according to claim 8, It is characterized in that After determining the predicted display delay time corresponding to the target video frame, the method further includes: When the predicted display delay time is greater than the current preset display delay time and the reference time is not within a preset range, updating the reference time and sending the updated reference time to each of the distributed output nodes; The predicted display delay time on each of the distributed output nodes is cleared.

13. The distributed video synchronous display method according to claim 2 or any one of claims 9 to 11, It is characterized in that Obtaining supplementary enhancement information corresponding to the target video frame, including: After the distributed input node encodes the collected target video frame, generating supplementary enhancement information of the target video frame; The supplemental enhancement information is sent to the corresponding distributed output node.

14. The distributed video synchronous display method according to any one of claims 9 to 11, It is characterized in that Determine the prediction value corresponding to the target video frame, including: When the target video frame is not the first frame received by the distributed output node and it is detected that the preset prediction model has completed training, determine the image group in which the target video frame is located; Determining the prediction model that is trained and complete corresponding to the image group; the prediction model corresponds to the category of the prediction value; Each video frame in the image group is processed by the fully trained prediction model to obtain the prediction value corresponding to the target video frame.

15. The distributed video synchronous display method according to claim 14, It is characterized in that The forecasting model adopts a seasonal autoregressive integrated moving average model.

16. The distributed video synchronous display method according to claim 14, It is characterized in that After processing each video frame in the image group by the fully trained prediction model to obtain the prediction value corresponding to the target video frame, the method further includes: determining an actual value corresponding to the predicted value; determining a difference between the actual value and the predicted value for each video frame in the group of pictures; When the cumulative sum of the difference values ​​is greater than a preset error threshold, the current prediction model is trained based on the actual value corresponding to each video frame in the image group to obtain the latest trained and complete prediction model.

17. A distributed video synchronization display device, It is characterized in that Applicable to a distributed video synchronous display system; wherein the system comprises a plurality of distributed input nodes and a plurality of distributed output nodes, each of the distributed input nodes is connected to each of the distributed output nodes via a network; the device comprises: an encoding module, a calculation module, a comparison module and a control module; The encoding module is used to encode the collected target video frame through each of the distributed input nodes, and send the encoded target video frame to the corresponding distributed output node; wherein each of the target video frames has a synchronous display requirement; The calculation module is used to determine the actual achievable minimum display delay time corresponding to the target video frame when detecting that the target video frame on the distributed output node is in a state to be displayed; The comparison module is used to determine all the actual achievable minimum display delay times whose differences from the preset display delay time exceed a preset threshold; The control module is used to control the synchronous display of each target video frame based on the maximum actual achievable minimum display delay time among the actual achievable minimum display delay times, when the actual achievable minimum display delay times whose differences with the preset display delay time exceed a preset threshold are all less than the preset display delay time.

18. A computer device comprising a memory and a processor, It is characterized in that A computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps of the distributed video synchronous display method according to any one of claims 1 to 16.

19. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the distributed video synchronous display method described in any one of claims 1 to 16 are implemented.

Images