Control Method and Device for Synchronous Playback

Abstract

The present application provides a control method and device for synchronous playback. The method includes: receiving first feature data of multiple playback devices, where the first feature data is used to characterize the playback screens of the synchronous playback content displayed in the frames corresponding to the multiple playback devices at a preset time; determining synchronous adjustment data of the multiple playback devices according to the first feature data of the multiple playback devices, where the synchronous adjustment data is used to adjust the playback delay of the multiple playback devices for the synchronous playback content; and sending the synchronous adjustment data to the multiple playback devices. By this means, the synchronous adjustment data between the playback devices is determined by using the first feature data of the playback screens at the frame level, thereby improving the accuracy of synchronous playback and the synchronous playback effect.

Description

Technical Field

[0001] The present invention relates to the technical field of intelligent terminals, and in particular, to a control method and device for synchronous playback. Background Art

[0002] With the popularization of playback devices, the use of multiple playback devices can achieve the alternating playback of large-size content and multiple different contents. At the same time, the synchronous playback of multiple playback devices can also achieve a better information dissemination effect, realizing the unified transmission of information, so as to ensure that users receive unified information at the same time point.

[0003] In the related art, the time difference can be continuously corrected by sending time information to the playback devices for synchronous playback through a Network Time Protocol (NTP) server, so as to correct the playback time of the playback devices and achieve the effect of synchronous playback.

[0004] However, through the NTP server, only the time difference within 100 milliseconds between playback devices can be corrected, resulting in low accuracy of synchronous playback among multiple playback devices and unable to fully achieve the effect of synchronous playback. Summary of the Invention

[0005] Embodiments of the present application provide a control method and device for synchronous playback to solve the problem of low synchronous accuracy when multiple playback devices perform synchronous playback in the prior art.

[0006] In a first aspect, embodiments of the present application provide a control method for synchronous playback, the method including:

[0007] Receiving first feature data of multiple playback devices, where the first feature data is used to characterize the playback pictures of the synchronous playback content displayed in frames corresponding to the multiple playback devices at a preset time;

[0008] Determining synchronous adjustment data of the multiple playback devices respectively according to the first feature data of the multiple playback devices, where the synchronous adjustment data is used to adjust the playback delay of the multiple playback devices for the synchronous playback content;

[0009] Sending the synchronous adjustment data to the multiple playback devices respectively.

[0010] In an optional implementation manner, the first feature data includes: the feature value of the playback picture, the identifier of the playback device, the time stamp corresponding to the preset time, and the network delay of the playback device.

[0011] In an optional implementation manner, the feature value of the playback picture is determined by performing graphic analysis on the playback picture.

[0012] In an alternative embodiment, determining the synchronization adjustment data of the plurality of playback devices respectively according to the first feature data of the plurality of playback devices includes:

[0013] Determining the difference data between the first feature data and the second feature data of the plurality of playback devices respectively, where the second feature data is used to represent the playback picture of the synchronized playback content predicted by the first server to be displayed in the frame corresponding to the preset time, and the difference data is used to represent the number of frames by which the playback picture corresponding to the first feature data differs from the playback picture corresponding to the second feature data;

[0014] Determining the synchronization adjustment data of the plurality of playback devices respectively according to the difference data between the first feature data and the second feature data of the plurality of playback devices.

[0015] In an alternative embodiment, before determining the difference data between the first feature data and the second feature data of the plurality of playback devices respectively, the method further includes:

[0016] Determining the second feature data in the feature dataset of the synchronized playback content according to the preset time;

[0017] Wherein, the feature dataset of the synchronized playback content contains the feature values of each frame of the playback picture arranged according to the playback order of the synchronized playback content.

[0018] In an alternative embodiment, before determining the difference data between the first feature data and the second feature data of the plurality of playback devices, the method further includes:

[0019] Aligning the first feature data of the plurality of playback devices according to the timestamp.

[0020] In an alternative embodiment, determining the synchronization adjustment data of the plurality of playback devices respectively according to the difference data between the first feature data and the second feature data of the plurality of playback devices includes:

[0021] Determining the minimum total adjustment number of frames of the plurality of playback devices according to the difference data between the first feature data and the second feature data of the plurality of playback devices;

[0022] Determining the synchronization adjustment data of the plurality of playback devices respectively according to the playback picture of the target frame corresponding to the minimum total adjustment number of frames and the network delays corresponding to the plurality of playback devices.

[0023] In an alternative embodiment, determining the minimum total adjustment frame number of the plurality of playback devices according to the difference data between the first feature data and the second feature data of the plurality of playback devices includes:

[0024] Filtering the difference data according to a resolution threshold;

[0025] Calculating the minimum total adjustment frame number of the plurality of playback devices by weighted calculation according to the number of playback devices corresponding to the filtered difference data respectively.

[0026] In a second aspect, an embodiment of the present application provides a control method for synchronous playback. The method includes:

[0027] Sending first feature data to a first server, where the first feature data is used to represent a playback screen of synchronous playback content displayed in a frame corresponding to a preset time by a playback device;

[0028] Receiving synchronous adjustment data sent by the first server;

[0029] Adjusting the playback delay of the synchronous playback content according to the synchronous adjustment data.

[0030] In an alternative embodiment, the first feature data includes: a feature value of the playback screen, an identifier of the playback device, a time stamp corresponding to the preset time, and a network delay of the playback device.

[0031] In an alternative embodiment, before sending the first feature data to the first server, the method further includes:

[0032] Performing graphic analysis on a playback screen of synchronous playback content displayed in a frame corresponding to the preset time to determine a feature value of the playback screen. <�

[0033] In an alternative embodiment, before sending the first feature data to the first server, the method further includes:

[0034] Sending a synchronous time acquisition request to a second server, where the synchronous time acquisition request is used to acquire a time stamp corresponding to the preset time.

[0035] In a third aspect, an embodiment of the present application provides a control device for synchronous playback. The device includes: \

[0036] A receiving module, configured to receive first feature data of a plurality of playback devices, where the first feature data is used to represent a playback screen of synchronous playback content displayed in a frame corresponding to a preset time by the plurality of playback devices;

[0037] A processing module, configured to determine synchronization adjustment data for the multiple playback devices according to first feature data of the multiple playback devices, where the synchronization adjustment data is used to adjust the playback delay of the multiple playback devices for the synchronized playback content;

[0038] A sending module, configured to send the synchronization adjustment data to the multiple playback devices.

[0039] In an optional implementation, the first feature data includes: a feature value of the playback screen, an identifier of the playback device, a timestamp corresponding to the preset time, and a network delay of the playback device.

[0040] In an optional implementation, the feature value of the playback screen is determined by performing graphic analysis on the playback screen.

[0041] In an optional implementation, the processing module is specifically configured to respectively determine difference data between first feature data and second feature data of the multiple playback devices, where the second feature data is used to characterize a playback screen of the synchronized playback content predicted by a first server to be displayed in a frame corresponding to the preset time, and the difference data is used to characterize the number of frames by which the playback screen corresponding to the first feature data differs from the playback screen corresponding to the second feature data; and determine the synchronization adjustment data for the multiple playback devices according to the difference data between the first feature data and the second feature data of the multiple playback devices. [[ID=*]]

[0042] In an optional implementation, the processing module is further configured to determine the second feature data in a feature dataset of the synchronized playback content according to the preset time;

[0043] Wherein, the feature dataset of the synchronized playback content contains feature values of each frame of the playback screen arranged according to the playback order of the synchronized playback content.

[0044] In an optional implementation, the processing module is further configured to align the first feature data of the multiple playback devices according to the timestamp.

[0045] In an optional implementation, the processing module is specifically configured to determine a minimum total adjustment number of frames for the multiple playback devices according to the difference data between the first feature data and the second feature data of the multiple playback devices; and determine the synchronization adjustment data for the multiple playback devices respectively according to the playback screen of a target frame corresponding to the minimum total adjustment number of frames and the network delays corresponding to the multiple playback devices.

[0046] In an alternative embodiment, the processing module is specifically configured to filter the difference data according to a resolution threshold, and weightedly calculate the minimum total adjustment frames of the multiple playback devices according to the number of playback devices corresponding to the filtered difference data respectively.

[0047] In a fourth aspect, an embodiment of the present application provides a control device for synchronous playback. The device includes:

[0048] A sending module, configured to send first feature data to a first server, where the first feature data is used to represent a playback screen of synchronous playback content displayed in a frame corresponding to a preset time by a playback device;

[0049] A receiving module, configured to receive synchronous adjustment data sent by the first server;

[0050] A processing module, configured to adjust a playback delay of the synchronous playback content according to the synchronous adjustment data.

[0051] In an alternative embodiment, the first feature data includes: a feature value of the playback screen, an identifier of the playback device, a timestamp corresponding to the preset time, and a network delay of the playback device.

[0052] In an alternative embodiment, the processing module is further configured to perform graphic analysis on a playback screen of synchronous playback content displayed in a frame corresponding to the preset time, and determine a feature value of the playback screen.

[0053] In an alternative embodiment, the sending module is further configured to send a synchronous time acquisition request to a second server, where the synchronous time acquisition request is used to acquire a timestamp corresponding to the preset time.

[0054] In a fifth aspect, the present application further provides a server, including: a processor and a memory; the memory is used to store a computer program of the processor; the processor is configured to implement any possible method in the first aspect by executing the computer program.

[0055] In a sixth aspect, the present application further provides a playback device, including: a processor and a memory; the memory is used to store a computer program of the processor; the processor is configured to implement any possible method in the first aspect by executing the computer program.

[0056] In a seventh aspect, the present invention further provides a computer storage medium, where the computer storage medium stores multiple instructions, and the instructions are suitable for being loaded and executed by a processor to implement any possible method in the first aspect.

[0057] In an eighth aspect, the present invention further provides a computer storage medium storing a plurality of instructions adapted to be loaded and executed by a processor to perform any of the possible methods in the first aspect.

[0058] A control method and device for synchronous playback provided by an embodiment of the present application first receive first feature data of a plurality of playback devices, where the first feature data is used to characterize the playback pictures of the synchronous playback content displayed in frames corresponding to the plurality of playback devices at a preset time. Subsequently, according to the first feature data of the plurality of playback devices, synchronous adjustment data for the plurality of playback devices is determined, and the synchronous adjustment data is used to adjust the playback delay of the plurality of playback devices for the synchronous playback content. Finally, the synchronous adjustment data is sent to the plurality of playback devices. In this way, the synchronous adjustment data between the playback devices is determined by using the first feature data of the playback pictures at the frame level, thereby improving the accuracy of synchronous playback and the synchronous playback effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] To more clearly illustrate the technical solutions in the present invention or the prior art, the following will briefly introduce the drawings required for the description of the embodiments or the prior art. Obviously, the following drawings are some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0060] Figure 1 Schematic diagram of an existing control method for synchronous playback;

[0061] Figure 2 Schematic diagram of another existing control method for synchronous playback;

[0062] Figure 3 Schematic diagram of yet another existing control method for synchronous playback;

[0063] Figure 4 Schematic diagram of an application scenario of a control method for synchronous playback provided by an embodiment of the present application;

[0064] Figure 5 Schematic diagram of the flow of a control method for synchronous playback provided by an embodiment of the present application;

[0065] Figure 6 Schematic diagram of the flow of another control method for synchronous playback provided by an embodiment of the present application;

[0066] Figure 7 Schematic diagram of the flow of yet another control method for synchronous playback provided by an embodiment of the present application;

[0067] Figure 8Schematic structural diagram of a control device for synchronous playback provided by an embodiment of the present application;

[0068] Figure 9 Schematic structural diagram of another control device for synchronous playback provided by an embodiment of the present application;

[0069] Figure 10 Schematic structural diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0070] To make the objectives, technical solutions, and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present application. Apparently, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments of the present application without making creative efforts shall fall within the protection scope of the present application.

[0071] With the popularization of playback devices, using multiple playback devices can achieve the alternating playback of large-size content and multiple different contents. At the same time, the synchronous playback of multiple playback devices can also achieve a better information dissemination effect, realizing the unified transmission of information, so as to ensure that users receive unified information at the same time point.

[0072] Figure 1 Schematic diagram of an existing control method for synchronous playback. As Figure 1 shown, device 11, device 12, and device 13 are respectively connected to the NTP server 14 in the cloud. The NTP server 14 sends time information to device 11, device 12, and device 13 at preset time intervals to correct the time difference between the playback devices, thereby calibrating the playback times of device 11, device 12, and device 13 to achieve the effect of synchronous playback. However, through the NTP server, only the time difference within 100 milliseconds between the playback devices can be corrected, and the time difference caused by the tiny differences in the network links of each playback device cannot be calibrated, resulting in low accuracy of synchronous playback among multiple playback devices and unable to fully achieve the effect of synchronous playback.

[0073] In addition, in related technologies, a hardware synchronization scheme can also be used to achieve the synchronous playback of multiple playback devices.

[0074] Figure 2 Schematic diagram of another existing control method for synchronous playback. As Figure 2As shown in the figure, on the device side, a screen mirroring device 15 can be added outside or inside the playback device. The screen mirroring device 15 is used to shield the clock fluctuations caused by the network. Devices 11, 12, and 13 are respectively connected to the screen mirroring device 15 and then access the cloud control device 16 through the network. Through the screen mirroring device, the playback content of multiple playback devices can be completely synchronized. However, due to the limited number of sockets of the screen mirroring device, only a limited number of screens can be supported for synchronization, the screen expansion cannot be achieved, the distance between playback devices is limited, and the device cost is increased.

[0075] Figure 3 It is a schematic diagram of another existing synchronous playback control method. As Figure 3 shown in the figure, on the device side, a screen cascading control circuit can be added to devices 11, 12, and 13, and the screen cascading control circuits of devices 11, 12, and 13 are connected in series through a High Definition Multimedia Interface-in (HDMI-in). Subsequently, the cascaded playback devices are connected to the playback control device 17. Finally, the playback control device 17 accesses the cloud control device 16 through the network. Through this method, the expansion of playback devices can be supported, and the playback content of multiple playback devices can be completely synchronized. However, due to the addition of the screen cascading control circuit and the playback control device, the device cost is increased.

[0076] To solve the above problems, the embodiments of the present application provide a synchronous playback control method and device. The playback device sends first feature data representing the playback screen of the synchronous playback content corresponding to the frame display at a preset time to the first server to adjust the playback delay of each playback device for the synchronous playback content, so as to achieve frame-level synchronous playback, improve the accuracy of synchronous playback, and thus improve the synchronous playback effect.

[0077] Next, the application scenario of the synchronous playback control method involved in the present application will be described.

[0078] Figure 4 It is a schematic diagram of the application scenario of a synchronous playback control method provided by the embodiments of the present application. As Figure 4As shown in the figure, when there are multiple playback devices 101 playing synchronized playback content, the multiple playback devices 101 can capture a playback screen at a preset time (for example, every 10 seconds), and respectively determine the eigenvalue corresponding to the playback screen through graphic analysis. At the same time, the multiple playback devices 101 can respectively obtain the timestamp corresponding to the preset time from the second server 102 (for example, NTP server). Subsequently, the multiple playback devices 101 can report the eigenvalue corresponding to the playback screen and the timestamp corresponding to the preset time to the first server 103. Finally, the first server 103 determines the playback delay of each playback device for the synchronized playback content, and sends synchronization adjustment data to the multiple playback devices 101.

[0079] Among them, the playback device 101 can be a tablet computer (pad), a virtual reality (VR) terminal device, an augmented reality (AR) terminal device, a wireless terminal in self-driving, a wireless terminal in remote medical surgery, a wireless terminal in a smart grid, a wireless terminal in a smart home, etc.

[0080] The first server 103 and the second server 102 can be, but are not limited to, a single network server, a server group composed of multiple network servers, or a cloud composed of a large number of computers or network servers based on cloud computing. Among them, cloud computing is a type of distributed computing, which is a super virtual computer composed of a group of loosely coupled computers.

[0081] It can be understood that the above method for controlling synchronized playback can be implemented by the control device for synchronized playback provided in the embodiments of the present application. The control device for synchronized playback can be part or all of a certain device, such as the above-mentioned playback device and server.

[0082] Taking the playback device and server integrated or installed with relevant execution codes as an example below, the technical solutions of the embodiments of the present application will be described in detail with specific embodiments. These specific embodiments below can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments.

[0083] Figure 5 It is a schematic flowchart of a method for controlling synchronized playback provided by an embodiment of the present application. This embodiment relates to how to control multiple playback devices to perform synchronized playback. As Figure 5 shown, the method includes:

[0084] S201. Multiple playback devices respectively send first feature data to the first server, where the first feature data is used to characterize the playback screen of the synchronized playback content displayed in the frame corresponding to the preset time by the playback device.

[0085] In this application, when multiple playback devices play synchronized playback content, they can send first feature data to the first server to reflect the playback screen of the synchronized playback content displayed in the frame corresponding to the preset time by the playback device.

[0086] It should be understood that the synchronized playback content in the embodiments of this application is not limited and can be videos, pictures, texts, etc. In some embodiments, the cloud control device can receive the synchronized playback content uploaded by the user and send the synchronized playback content to multiple playback devices according to the requirements of offline playback. Among them, the cloud control device can be the above-mentioned first server, or it can also be other cloud devices that control the playback devices.

[0087] It should be understood that the preset time in the embodiments of this application is not limited and can be a fixed time point, such as the 10th second after the synchronized playback content starts playing, or it can also be a time point set at time intervals. For example, every 10 seconds after the synchronized playback content starts playing is a preset time.

[0088] The following describes the first feature data involved in the embodiments of this application.

[0089] In some embodiments, the first feature data may include: the feature value of the playback screen, the identifier of the playback device, the time stamp corresponding to the preset time, and the network latency of the playback device.

[0090] Exemplarily, the first feature data can form the first feature data in the form of a two-dimensional matrix with the feature value of the playback screen, the identifier of the playback device, the time stamp corresponding to the preset time, and the network latency of the playback device. For example, Device 1 (Time Stamp A, Network Latency 1A, Feature Value 1A), Device 1 (Time Stamp B, Network Latency 1B, Feature Value 1B), Device 2 (Time Stamp A, Network Latency 2A, Feature Value 2A), Device 2 (Time Stamp B, Network Latency 2B, Feature Value 2B), etc.

[0091] Among them, the identifier of the playback device and the network latency of the playback device can be directly obtained from the playback device.

[0092] It should be understood that the embodiments of this application do not limit how the playback device determines the feature value of the playback screen. In some embodiments, the playback device can take a screenshot of the playback screen of the synchronized playback content displayed in the frame corresponding to the preset time, and then perform graphic analysis on the screenshot of the playback screen to obtain the feature value of the playback screen.

[0093] It should be understood that the embodiments of the present application do not limit how the playback device determines the time stamp corresponding to the preset time. In some embodiments, the playback device may send a synchronization time acquisition request to the second server, and the synchronization time acquisition request is used to acquire the time stamp corresponding to the preset time. Subsequently, the second server may send the time stamp corresponding to the preset time to the playback device. Exemplarily, the second server may be an NTP server, and the NTP server contains the latest system time. The first server and the playback device may use the same NTP server.

[0094] S202. The first server respectively determines synchronization adjustment data of multiple playback devices according to the first feature data of the multiple playback devices, and the synchronization adjustment data is used to adjust the playback delay of the multiple playback devices for the synchronized playback content.

[0095] In this step, after the first server receives the first feature data respectively sent by multiple playback devices, it may respectively determine the synchronization adjustment data of the multiple playback devices according to the first feature data of the multiple playback devices, so as to control the synchronized playback of the multiple playback devices.

[0096] In some embodiments, before determining the synchronization adjustment data of multiple playback devices, if the first server receives the first feature data of multiple preset times sent by the playback device at the same time, the first server may align the first feature data of the multiple playback devices according to the time stamp.

[0097] Exemplarily, if there is first feature data with a time stamp of A and first feature data with a time stamp of B, the first feature data corresponding to the time stamp A of different playback devices may be aligned, and at the same time, the first feature data with a time stamp of B of different playback devices may be aligned.

[0098] It should be understood that the embodiments of the present application do not limit how to respectively determine the synchronization adjustment data of multiple playback devices according to the first feature data of the multiple playback devices. In some embodiments, the first server first respectively determines the difference data between the first feature data and the second feature data of the multiple playback devices. Subsequently, the first server may respectively determine the synchronization adjustment data of the multiple playback devices according to the difference data between the first feature data and the second feature data of the multiple playback devices.

[0099] Wherein, the second feature data is used to represent the playback picture of the synchronized playback content predicted by the first server to be displayed in the frame corresponding to the preset time, and the difference data is used to represent the number of frames by which the playback picture corresponding to the first feature data differs from the playback picture corresponding to the second feature data.

[0100] It should be understood that the embodiments of the present application do not limit how to determine the second feature data. In some embodiments, the first server may determine the second feature data from the feature dataset of the synchronized playback content according to a preset time. The feature dataset of the synchronized playback content contains the feature values of each frame of the playback screen arranged according to the playback order of the synchronized playback content.

[0101] Exemplarily, if the duration of the synchronized playback content is 10 seconds and each second contains 10 frames of playback screens (each frame is 1 / 24 second), then a total of 240 frames of playback screens can be parsed from the synchronized playback content. The first server may perform graphic analysis on the 240 frames of graphics in advance to determine the feature value of each frame of the playback screen. Subsequently, the first server may combine the feature values of the 240 frames of playback screens into the feature dataset of the synchronized playback content according to the playback order, for example, (feature value 1, feature value 2, feature value 3... feature value 240).

[0102] Correspondingly, if the preset time is 5 seconds after the start of the broadcast, then the corresponding feature value in the feature dataset of the synchronized playback content is the feature value 120 of the 120th frame of the playback screen. Subsequently, the first server may compare the feature value 120 of the 120th frame of the playback screen with the feature value in the first feature data (for example, the feature value 125 of the 125th frame of the playback screen), and combine the feature dataset of the synchronized playback content to determine that the difference data is 5 frames.

[0103] It should be noted that the feature values in the second feature data determined by the first server and the feature values in the first feature data determined by the playback device adopt the same image analysis algorithm.

[0104] It should be understood that the embodiments of the present application do not limit how to determine the synchronization adjustment data of multiple playback devices according to the difference data between the first feature data and the second feature data of multiple playback devices. In some embodiments, the first server may first determine the minimum total adjustment frames of multiple playback devices according to the difference data between the first feature data and the second feature data of multiple playback devices. Subsequently, the first server may determine the synchronization adjustment data of multiple playback devices respectively according to the playback screen of the target frame corresponding to the minimum total adjustment frames and the network delays corresponding to multiple playback devices.

[0105] It should be understood that the embodiments of the present application do not limit how to determine the minimum total adjustment frames of multiple playback devices either. The first server may first filter the difference data according to the resolution threshold, and then weighted calculate the minimum total adjustment frames of multiple playback devices according to the number of playback devices corresponding to the filtered difference data.

[0106] Among them, the resolution threshold can be specifically set according to the actual situation. Since the minimum resolution of the human eye is 50 ms, the average is 250 ms, and the maximum is 1000 ms, correspondingly, the resolution threshold can be set to 0 - 1 frame.

[0107] Exemplarily, the first feature data corresponding to 5 seconds after the start of the broadcast includes one device 1 (timestamp 5 s, network delay 1 frame, feature value 121), two device 2s (timestamp 5 s, network delay 1 frame, feature value 125), and two device 3s (timestamp 5 s, network delay 1 frame, feature value 127). If the feature value in the second feature data corresponding to 5 seconds after the start of the broadcast is feature value 120, then it can be determined that the difference data of device 1 is 1 frame, the difference data of device 2 is 5 frames, and the difference data of device 3 is 7 frames. Subsequently, through the resolution threshold of 1 frame, the difference data of device 1 can be filtered out, that is, device 1 is equivalent to playing the 120th frame of the playback screen 5 seconds after the start of the broadcast. Finally, by weighting the number of devices, when the playback screen of the adjusted target frame is the 125th frame, the minimum total adjustment frames can be 9 frames. At this time, the synchronization adjustment data of each device 1 is adjusted backward by 5 frames, the synchronization adjustment data of each device 2 is not adjusted, and the synchronization adjustment data of each device 3 is adjusted forward by 2 frames.

[0108] In this way, by calculating the minimum total adjustment frames and then determining the synchronization adjustment data of each playback device based on the minimum total adjustment frames, the adjustment amount can be reduced, the communication data between the cloud and the devices can be saved, and the impact of synchronization adjustment on playback can be reduced.

[0109] S203. The first server sends synchronization adjustment data to multiple playback devices respectively.

[0110] S204. Multiple playback devices adjust the playback delay of the synchronized playback content according to the synchronization adjustment data respectively.

[0111] Exemplarily, if the synchronization adjustment data of device 1 is adjusted backward by 5 frames, then device 1 adjusts the playback delay backward by 5 frames; if the synchronization adjustment data of device 2 is not adjusted, then device 2 does not adjust the playback delay; if the synchronization adjustment data of device 3 is adjusted forward by 2 frames, then device 3 adjusts the playback delay forward by 2 frames.

[0112] The control method for synchronous playback provided by the embodiments of the present application first receives the first feature data of multiple playback devices, where the first feature data is used to characterize the playback images of the synchronous playback content corresponding to the frames of the multiple playback devices at a preset time. Subsequently, according to the first feature data of the multiple playback devices, the synchronous adjustment data of the multiple playback devices is determined, and the synchronous adjustment data is used to adjust the playback delay of the multiple playback devices for the synchronous playback content. Finally, the synchronous adjustment data is sent to the multiple playback devices. In this way, the first feature data of the playback images at the frame level is used to determine the synchronous adjustment data between the playback devices, thereby improving the accuracy of synchronous playback and the synchronous playback effect.

[0113] Based on the above embodiments, the following describes how the first server determines the synchronous adjustment data of multiple playback devices. Figure 6 It is a schematic flowchart of another control method for synchronous playback provided by the embodiments of the present application. As Figure 6 shown, the method includes:

[0114] S301. Multiple playback devices respectively send the first feature data to the first server, where the first feature data is used to characterize the playback images of the synchronous playback content corresponding to the frames of the playback devices at a preset time.

[0115] S302. The first server aligns the first feature data of the multiple playback devices according to the timestamps.

[0116] S303. The first server determines the second feature data in the feature dataset of the synchronous playback content according to the preset time.

[0117] Among them, the feature dataset of the synchronous playback content contains the feature values of each frame playback image arranged according to the playback order of the synchronous playback content.

[0118] S304. Respectively determine the difference data between the first feature data and the second feature data of the multiple playback devices.

[0119] Among them, the second feature data is used to characterize the playback image of the synchronous playback content predicted by the first server to be displayed at the frame corresponding to the preset time, and the difference data is used to characterize the number of frames by which the playback image corresponding to the first feature data differs from the playback image corresponding to the second feature data.

[0120] In some embodiments, the first server may first determine the minimum total adjustment frames of the multiple playback devices according to the difference data between the first feature data and the second feature data of the multiple playback devices. Subsequently, the first server respectively determines the synchronous adjustment data of the multiple playback devices according to the playback image of the target frame corresponding to the minimum total adjustment frames and the network delays corresponding to the multiple playback devices.

[0121] In some embodiments, the first server may first filter the differential data according to the resolution threshold. Subsequently, the minimum total adjustment frame numbers of multiple playback devices are calculated by weighting according to the number of playback devices corresponding to the filtered differential data respectively.

[0122] S305. The first server determines the synchronization adjustment data of multiple playback devices respectively according to the differential data between the first feature data and the second feature data of the multiple playback devices.

[0123] S306. The first server sends the synchronization adjustment data to multiple playback devices respectively.

[0124] S307. Multiple playback devices adjust the playback delay of the synchronized playback content respectively according to the synchronization adjustment data.

[0125] For the technical terms, technical effects, technical features, and optional embodiments of steps S301 to S307, reference may be made to Figure 5 Steps S201 to S204 shown. For repeated information, it will not be elaborated here.

[0126] Based on the above embodiments, the following describes how the playback device side sends the first feature data to the first server. Figure 7 It is a schematic flowchart of another synchronization playback control method provided by an embodiment of the present application. As Figure 7 shown, the method includes:

[0127] S401. The playback device performs graphic analysis on the playback picture of the synchronized playback content displayed in the frame corresponding to the preset time to determine the feature value of the playback picture.

[0128] S402. The playback device sends a synchronization time acquisition request to the second server, and the synchronization time acquisition request is used to acquire the time stamp corresponding to the preset time.

[0129] Among them, the second server may be an NTP server.

[0130] It should be noted that the embodiment of the present application does not limit the execution order of steps S401 and S402. Step S401 may be executed before step S402, may be executed after step S402, or may be executed simultaneously with step S402.

[0131] S403. The second server sends the time stamp corresponding to the preset time to the playback device.

[0132] S404. The playback device sends the first feature data to the first server.

[0133] Among them, the first feature data includes: the feature value of the playback screen, the identifier of the playback device, the time stamp corresponding to the preset time, and the network latency of the playback device.

[0134] S405. The first server determines the synchronization adjustment data of multiple playback devices respectively according to the first feature data of the multiple playback devices, and the synchronization adjustment data is used to adjust the playback latency of the multiple playback devices for the synchronized playback content.

[0135] S406. The first server sends the synchronization adjustment data to the multiple playback devices respectively.

[0136] S407. The multiple playback devices adjust the playback latency of the synchronized playback content respectively according to the synchronization adjustment data.

[0137] For the technical terms, technical effects, technical features, and optional implementation manners of steps S401 to S407, reference can be made to Figure 5 Steps S201 to S204 shown. For repeated information, it will not be elaborated here.

[0138] The synchronization playback control method provided by the embodiments of the present application first receives the first feature data of multiple playback devices, and the first feature data is used to represent the playback screens of the synchronized playback content displayed in frames corresponding to the multiple playback devices at the preset time. Subsequently, according to the first feature data of the multiple playback devices, the synchronization adjustment data of the multiple playback devices is determined, and the synchronization adjustment data is used to adjust the playback latency of the multiple playback devices for the synchronized playback content. Finally, the synchronization adjustment data is sent to the multiple playback devices. In this way, the first feature data of the playback screen at the frame level is used to determine the synchronization adjustment data between the playback devices, thereby improving the accuracy of synchronized playback and the synchronized playback effect.

[0139] Those of ordinary skill in the art can understand that all or part of the steps to implement the above method embodiments can be completed by hardware related to program instructions. The foregoing program can be stored in a computer-readable storage medium. When the program is executed, it executes the steps including the above method embodiments; and the foregoing storage medium includes: various media such as ROM, RAM, magnetic disk, or optical disk that can store program codes.

[0140] Figure 8 It is a schematic structural diagram of a synchronization playback control device provided by the embodiments of the present application. The synchronization playback control device can be implemented through software, hardware, or a combination of both to execute the synchronization playback control method on the side of the first server in the above embodiments. As Figure 8 shown, the synchronization playback control device 500 includes: a receiving module 501, a processing module 502, and a sending module 503.

[0141] A receiving module 501, configured to receive first feature data of multiple playback devices, where the first feature data is used to characterize the playback images of the synchronized playback content displayed in the frames corresponding to a preset time by the multiple playback devices;

[0142] A processing module 502, configured to determine synchronization adjustment data of the multiple playback devices according to the first feature data of the multiple playback devices, where the synchronization adjustment data is used to adjust the playback delay of the multiple playback devices for the synchronized playback content;

[0143] A sending module 503, configured to send the synchronization adjustment data to the multiple playback devices.

[0144] In an optional implementation manner, the first feature data includes: the feature value of the playback image, the identifier of the playback device, the time stamp corresponding to the preset time, and the network delay of the playback device.

[0145] In an optional implementation manner, the feature value of the playback image is determined by performing graphic analysis on the playback image.

[0146] In an optional implementation manner, the processing module 502 is specifically configured to respectively determine the difference data between the first feature data and the second feature data of the multiple playback devices, where the second feature data is used to characterize the playback image of the synchronized playback content predicted by the first server to be displayed in the frame corresponding to the preset time, and the difference data is used to characterize the number of frames by which the playback image corresponding to the first feature data differs from the playback image corresponding to the second feature data; and determine the synchronization adjustment data of the multiple playback devices according to the difference data between the first feature data and the second feature data of the multiple playback devices.

[0147] In an optional implementation manner, the processing module 502 is further configured to determine the second feature data in the feature dataset of the synchronized playback content according to the preset time;

[0148] Wherein, the feature dataset of the synchronized playback content contains the feature values of each frame of the playback image arranged according to the playback order of the synchronized playback content.

[0149] In an optional implementation manner, the processing module 502 is further configured to align the first feature data of the multiple playback devices according to the time stamp.

[0150] In an optional implementation manner, the processing module 502 is specifically configured to determine the minimum total adjustment number of frames of the multiple playback devices according to the difference data between the first feature data and the second feature data of the multiple playback devices; and determine the synchronization adjustment data of the multiple playback devices respectively according to the playback image of the target frame corresponding to the minimum total adjustment number of frames and the network delays corresponding to the multiple playback devices.

[0151] In an alternative embodiment, the processing module 502 is specifically configured to filter the difference data according to a resolution threshold, and weighted calculate the minimum total adjustment frames of multiple playback devices according to the number of playback devices corresponding to the filtered difference data respectively.

[0152] It should be noted that Figure 8 The synchronization playback control device provided in the foregoing embodiment can be used to execute the method on the first server side provided in any of the foregoing embodiments. The specific implementation manners and technical effects are similar and will not be elaborated herein.

[0153] Figure 9 FIG. is a schematic structural diagram of another synchronization playback control device provided in an embodiment of the present application. The synchronization playback control device can be implemented by software, hardware, or a combination of both to execute the synchronization playback control method on the terminal device side in the foregoing embodiment. As Figure 9 shown, the synchronization playback control device 600 includes: a sending and receiving module 601, a receiving module 602, and a processing module 603.

[0154] The sending module 601 is configured to send first feature data to the first server, where the first feature data is used to characterize the playback screen of the synchronization playback content corresponding to the frame displayed by the playback device at a preset time;

[0155] The receiving module 602 is configured to receive synchronization adjustment data sent by the first server;

[0156] The processing module 603 is configured to adjust the playback delay of the synchronization playback content according to the synchronization adjustment data.

[0157] In an alternative embodiment, the first feature data includes: a feature value of the playback screen, an identifier of the playback device, a time stamp corresponding to the preset time, and a network delay of the playback device.

[0158] In an alternative embodiment, the processing module 603 is further configured to perform graphic analysis on the playback screen of the synchronization playback content corresponding to the frame displayed at the preset time to determine the feature value of the playback screen.

[0159] In an alternative embodiment, the sending module 601 is further configured to send a synchronization time acquisition request to the second server, where the synchronization time acquisition request is used to acquire the time stamp corresponding to the preset time.

[0160] It should be noted that Figure 9 The synchronization playback control device provided in the foregoing embodiment can be used to execute the method on the terminal device side provided in any of the foregoing embodiments. The specific implementation manners and technical effects are similar and will not be elaborated herein.

[0161] Figure 10The structural schematic diagram of an electronic device provided by an embodiment of the present application. As Figure 10 shown, the electronic device may include: a plurality of processors 701 and a memory 702. Figure 10 The electronic device taking one processor as an example is shown.

[0162] The memory 702 is used to store programs. Specifically, the program may include program codes, and the program codes include computer operation instructions.

[0163] The memory 702 may include a high-speed RAM memory, and may also include a non-volatile memory, such as a plurality of disk memories.

[0164] The processor 701 is used to execute the computer execution instructions stored in the memory 702 to implement the control method for synchronous playback of the above-mentioned playback device, or to implement the control method for synchronous playback on the side of the above-mentioned playback device;

[0165] Among them, the processor 701 may be a processor (Central Processing Unit, simply referred to as CPU), or a specific integrated circuit (Application Specific Integrated Circuit, simply referred to as ASIC), or one or more integrated circuits configured to implement the embodiments of the present application.

[0166] Optionally, in specific implementation, if the communication interface, the memory 702, and the processor 701 are implemented independently, the communication interface, the memory 702, and the processor 701 may be interconnected through a bus and complete communication with each other. The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, or an Extended Industry Standard Architecture (EISA) bus, etc. The bus may be divided into an address bus, a data bus, a control bus, etc., but it does not mean that there is only one bus or one type of bus.

[0167] Optionally, in specific implementation, if the communication interface, the memory 702, and the processor 701 are integrated on a chip, the communication interface, the memory 702, and the processor 701 may complete communication through an internal interface.

[0168] An embodiment of the present application also provides a chip, which includes a processor and an interface. The interface is used to input and output the data or instructions processed by the processor. The processor is used to execute the method provided in the above method embodiments. This chip can be applied to a control device for synchronous playback.

[0169] The present application also provides a computer-readable storage medium, which may include: various media that can store program codes such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc. Specifically, program information is stored in this computer-readable storage medium, and the program information is used for the above-mentioned control method for synchronous playback.

[0170] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions according to the embodiments of the present invention are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from a website, a computer, a server, or a data center to another website, a computer, a server, or a data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or a wireless manner (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or a data center that includes one or more integrated available media. The available medium can be a magnetic medium (such as a floppy disk, a hard disk, a magnetic tape), an optical medium (such as a DVD), or a semiconductor medium (such as a solid state disk (SSD)).

[0171] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, and are not intended to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A control method for synchronous playback, characterized in that, Applied to a first server, the method includes: Receiving first feature data of a plurality of playback devices, where the first feature data is used to characterize the playback pictures of synchronized playback content displayed in frames corresponding to a preset time by the plurality of playback devices; Respectively determining difference data between the first feature data and second feature data of the plurality of playback devices, where the second feature data is used to characterize the playback pictures of synchronized playback content predicted by the first server to be displayed in frames corresponding to the preset time, and the difference data is used to characterize the number of frames by which the playback pictures corresponding to the first feature data differ from the playback pictures corresponding to the second feature data; Respectively determining synchronization adjustment data for the plurality of playback devices according to the difference data between the first feature data and the second feature data of the plurality of playback devices, where the synchronization adjustment data is used to adjust the playback time delay of the plurality of playback devices for the synchronized playback content; Respectively sending the synchronization adjustment data to the plurality of playback devices.

2. The method according to claim 1, characterized in that, The first feature data includes: the feature value of the playback picture, the identifier of the playback device, the time stamp corresponding to the preset time, and the network delay of the playback device.

3. The method according to claim 2, characterized in that The feature value of the playback picture is determined by performing graphic analysis on the playback picture.

4. The method according to claim 1, characterized in that, Before respectively determining the difference data between the first feature data and the second feature data of the plurality of playback devices, the method further includes: Determining the second feature data in the feature dataset of the synchronized playback content according to the preset time; Wherein, the feature dataset of the synchronized playback content contains the feature values of each frame of playback picture arranged according to the playback order of the synchronized playback content.

5. The method according to claim 1, wherein Before determining the difference data between the first feature data and the second feature data of the plurality of playback devices, the method further includes: Aligning the first feature data of the plurality of playback devices according to the time stamp corresponding to the preset time.

6. The method according to any one of claims 1-5, characterized in that, The respectively determining the synchronization adjustment data for the plurality of playback devices according to the difference data between the first feature data and the second feature data of the plurality of playback devices includes: Determining the minimum total adjustment frames of the plurality of playback devices according to the difference data between the first feature data and the second feature data of the plurality of playback devices; Respectively determining the synchronization adjustment data for the plurality of playback devices according to the playback picture of the target frame corresponding to the minimum total adjustment frames and the network delays corresponding to the plurality of playback devices.

7. The method according to claim 6, characterized in that, The determining the minimum total adjustment frames of the plurality of playback devices according to the difference data between the first feature data and the second feature data of the plurality of playback devices includes: Filtering the difference data according to a resolution threshold; Weightedly calculating the minimum total adjustment frames of the plurality of playback devices according to the number of playback devices corresponding to the filtered difference data respectively.

8. A control method for synchronous playback, characterized in that, Applied to a playback device, the method includes: Send the first feature data to the first server, so that the first server respectively determines the difference data between the first feature data and the second feature data of multiple playback devices, and respectively determines the synchronization adjustment data of the multiple playback devices according to the difference data between the first feature data and the second feature data of the multiple playback devices; wherein, the first feature data is used to characterize the playback screen of the synchronized playback content displayed in the frame corresponding to the preset time by the playback device; the second feature data is used to characterize the playback screen of the synchronized playback content predicted by the first server to be displayed in the frame corresponding to the preset time, and the difference data is used to characterize the number of frames by which the playback screen corresponding to the first feature data differs from the playback screen corresponding to the second feature data. Receive the synchronization adjustment data sent by the first server. Adjust the playback delay of the synchronized playback content according to the synchronization adjustment data.

9. The method according to claim 8, characterized in that, The first feature data includes: the feature value of the playback screen, the identifier of the playback device, the time stamp corresponding to the preset time, and the network delay of the playback device.

10. The method according to claim 8, wherein Before sending the first feature data to the first server, the method further includes: Perform graphic analysis on the playback screen of the synchronized playback content displayed in the frame corresponding to the preset time to determine the feature value of the playback screen.

11. The method according to claim 9, wherein Before sending the first feature data to the first server, the method further includes: Send a synchronization time acquisition request to the second server, and the synchronization time acquisition request is used to acquire the time stamp corresponding to the preset time.

12. A control device for synchronous playback, characterized in that, The device includes: A receiving module, configured to receive the first feature data of multiple playback devices, where the first feature data is used to characterize the playback screen of the synchronized playback content displayed in the frame corresponding to the preset time by the multiple playback devices. A processing module, configured to respectively determine the synchronization adjustment data of the multiple playback devices according to the first feature data of the multiple playback devices, where the synchronization adjustment data is used to adjust the playback delay of the multiple playback devices for the synchronized playback content. A sending module, configured to respectively send the synchronization adjustment data to the multiple playback devices. The processing module is specifically configured to respectively determine the difference data between the first feature data and the second feature data of the multiple playback devices, where the second feature data is used to characterize the playback screen of the synchronized playback content predicted by the first server to be displayed in the frame corresponding to the preset time, and the difference data is used to characterize the number of frames by which the playback screen corresponding to the first feature data differs from the playback screen corresponding to the second feature data; and respectively determine the synchronization adjustment data of the multiple playback devices according to the difference data between the first feature data and the second feature data of the multiple playback devices.

13. A control device for synchronous playback, characterized in that, The device includes: A sending module, configured to send first feature data to a first server, so that the first server respectively determines difference data between the first feature data and second feature data of multiple playback devices, and respectively determines synchronization adjustment data of the multiple playback devices according to the difference data between the first feature data and the second feature data of the multiple playback devices; wherein, the first feature data is used to characterize a playback picture of synchronized playback content displayed in a frame corresponding to a preset time by the playback device; the second feature data is used to characterize a playback picture of synchronized playback content predicted by the first server to be displayed in the frame corresponding to the preset time, and the difference data is used to characterize the number of frames by which the playback picture corresponding to the first feature data differs from the playback picture corresponding to the second feature data. A receiving module, configured to receive the synchronization adjustment data sent by the first server. A processing module, configured to adjust a playback delay of the synchronized playback content according to the synchronization adjustment data.

Images