Multi-device audio and video synchronization method, device, electronic device and storage medium

Abstract

The embodiments of the present application provide a multi-device audio and video synchronization method, apparatus, electronic device and storage medium, and relate to the field of computer technology. The method includes: dividing the audio and video into multiple time periods; obtaining the number of played frames of different devices at the sampling moment in the time period; determining the device with the least number of played frames at the last sampling moment among the devices as the first device, and determining the second device based on the number of played frames of each device at each sampling moment; in the second time period, adjusting the playback progress of the second device to be synchronized with the playback progress of the first device. By dividing the audio and video into multiple time periods, the playback progress of the device is adjusted at the end of each time period, so that the user cannot physiologically perceive the playback difference of the audio and video of different devices, so as to achieve the effect of audio and video synchronization. The solution of the present application is adjusted according to the playback progress of each device in each time period, and is not affected by the performance differences of hardware and software between different devices and interference from external factors.

Description

Technical Field

[0001] The present application relates to the field of computer technology. Specifically, the present application relates to a method, apparatus, electronic device and storage medium for implementing audio and video synchronization of multiple devices. Background Art

[0002] Audio and video applications are becoming increasingly widespread, and playing them on electronic devices is very common. However, an increasing number of business scenarios require multiple devices to simultaneously play the same audio or video, which is called multi-device audio and video synchronization.

[0003] In the prior art, frame synchronization is used, that is, multiple different devices are required to play the same video frame and the same audio frame at the same time to achieve the effect of audio and video synchronization.

[0004] However, in real-world scenarios, achieving complete synchronization of audio and video frames across different devices is extremely difficult. Differences in hardware and software performance between devices lead to different processing speeds for each frame of the same audio or video clip. This results in different devices being unable to play the same audio and video frames at the same time, effectively preventing them from achieving complete frame synchronization. Furthermore, during device operation, many external and internal factors can interfere with the operation. For example, if a device temporarily processes a higher-priority task, its resources for playing audio or video are squeezed out, further disrupting frame synchronization. Summary of the Invention

[0005] The purpose of this application is to solve at least one of the above technical deficiencies. The technical solutions provided by the embodiments of this application are as follows:

[0006] In a first aspect, an embodiment of the present application provides a method for implementing audio and video synchronization of multiple devices, including:

[0007] Divide the audio and video into at least two preset time periods in sequence;

[0008] Obtain the number of played frames of different devices at at least one preset sampling moment within a first preset time period;

[0009] Determine the device with the least number of played frames at the last preset sampling moment among the devices as the first device, and determine the second device whose playback status is early playback based on the number of played frames of each device at each preset sampling moment;

[0010] For each second device, the playback progress of the second device is adjusted within a second preset time period.

[0011] In an optional embodiment of the present application, determining the second device whose playback status is early playback based on the number of played frames of each device at each preset sampling moment includes:

[0012] For each preset sampling moment, obtain the median of the number of played frames of each device at the preset sampling moment, and obtain the relationship between the number of played frames of each device and the median;

[0013] The second device is determined based on the relationship between the number of played frames of each device at each preset sampling moment and the corresponding median.

[0014] In an optional embodiment of the present application, determining the second device based on the relationship between the number of played frames of each device at each preset sampling moment and the corresponding median includes:

[0015] For each preset sampling moment, use the first mark to mark the device whose number of played frames is less than the median corresponding to the preset sampling moment, and use the second mark to mark the device whose number of played frames is greater than the median corresponding to the preset sampling moment;

[0016] For each device, performing a signed rank sum test on the number of first marks and the second number of marks marked on the device within a first preset time period to obtain a test result corresponding to the device;

[0017] The second device is determined as a device whose inspection result shows that the number of the second mark exceeds the preset multiple of the number of the first mark.

[0018] In an optional embodiment of the present application, for each second device, adjusting the playback progress of the second device to be synchronized with the playback progress of the first device within the second preset time period includes:

[0019] Obtaining the playback speed of the first device based on the number of frames played by the first device at the last preset sampling moment within the first preset time period, and obtaining the playback speed of the second device based on the number of frames played by the second device at the last preset sampling moment within the first preset time period;

[0020] Obtaining a first preset number of frames and a second preset number of frames based on a playback speed of the first device, a playback speed of the second device, and a preset number of repeated frames; wherein a ratio of the first preset number of frames to the playback speed of the first device is equal to a ratio of the second preset number of frames to the playback speed of the second device, and the second preset number of frames is equal to the sum of the number of repeated frames and the first preset number of frames;

[0021] The second device is configured to repeatedly play the audio and video with a preset number of repeated frames every time it plays the audio and video with a second preset number of frames after the start of the second preset time period, wherein the audio and video with a preset number of repeated frames is the audio and video with a preset number of repeated frames located at the end of the audio and video with the second preset number of frames.

[0022] In an optional embodiment of the present application, the method may further include:

[0023] For each first preset time period after the second preset time period, obtaining a first variance of the corresponding number of played frames based on the number of played frames of each device at each preset sampling moment in a second preset time period that is earlier than the first of the two preset time periods before the first preset time period, and obtaining a second variance of the corresponding number of played frames based on the number of played frames of each device at each preset sampling moment in a third preset time period that is later than the first preset time period.

[0024] For each preset sampling moment, compare the first variance and the second variance corresponding to the preset sampling moment, and obtain the proportion of the preset sampling moments in which the first variance is greater than the second variance;

[0025] If the ratio is less than the preset ratio value, obtaining a third device whose playback status is delayed in the third preset time period and whose playback progress has been adjusted in the third preset time period, and canceling the adjustment of the playback progress of the third device after the first preset time period starts;

[0026] If the ratio is not less than the preset ratio value, then obtain the fourth device with the least number of played frames at the last preset sampling moment in the third preset time period and the fifth device whose playback status is early playback in the third preset time period, and adjust the playback progress of the fifth device after the start of the first preset time period.

[0027] In an optional embodiment of the present application, obtaining a third device whose playback status in a third preset time period is playback lag and whose playback progress has been adjusted in the third preset time period includes:

[0028] For each preset sampling moment, obtain the median of the number of played frames of each device at the preset sampling moment, and obtain the relationship between the number of played frames of each device and the median;

[0029] Based on the relationship between the number of played frames of each device at each preset sampling moment and the corresponding median, a third device is determined from the devices whose play progress has been adjusted in the third preset time period.

[0030] In an optional embodiment of the present application, determining the third device from the devices whose playback progress has been adjusted in the third preset time period based on the relationship between the number of played frames of each device at each preset sampling moment and the corresponding median includes:

[0031] For each preset sampling moment, use the first mark to mark the device whose number of played frames is less than the median corresponding to the preset sampling moment, and use the second mark to mark the device whose number of played frames is greater than the median corresponding to the preset sampling moment;

[0032] For each device, performing a signed rank sum test on the number of first marks and the second number of marks marked on the device within a third preset time period to obtain a test result corresponding to the device;

[0033] The third device is determined from the devices whose playback progress has been adjusted in the third preset time period by a device whose inspection result shows that the number of first marks exceeds the number of second marks which is a preset multiple.

[0034] In a second aspect, an embodiment of the present application provides an apparatus for synchronizing audio and video of multiple devices, including:

[0035] A preset time period setting module, used to divide the audio and video into at least two preset time periods in sequence;

[0036] A playback frame number sampling module is used to obtain the number of played frames of different devices at at least one preset sampling moment within a first preset time period;

[0037] The device determination module that needs to be adjusted is used to determine the device with the least number of played frames at the last preset sampling time among the devices as the first device, and determine the second device whose playback situation is ahead of schedule based on the number of played frames of each device at each preset sampling time;

[0038] The play progress adjustment module is used to adjust the play progress of each second device within a second preset time period.

[0039] In an optional embodiment of the present application, the module for determining the device requiring adjustment is specifically configured to:

[0040] For each preset sampling moment, obtain the median of the number of played frames of each device at the preset sampling moment, and obtain the relationship between the number of played frames of each device and the median;

[0041] The second device is determined based on the relationship between the number of played frames of each device at each preset sampling moment and the corresponding median.

[0042] In an optional embodiment of the present application, the module for determining the device requiring adjustment is further configured to:

[0043] For each preset sampling moment, use the first mark to mark the device whose number of played frames is less than the median corresponding to the preset sampling moment, and use the second mark to mark the device whose number of played frames is greater than the median corresponding to the preset sampling moment;

[0044] For each device, performing a signed rank sum test on the number of first marks and the second number of marks marked on the device within a first preset time period to obtain a test result corresponding to the device;

[0045] The second device is determined as a device whose inspection result shows that the number of the second mark exceeds the preset multiple of the number of the first mark.

[0046] In an optional embodiment of the present application, the playback progress adjustment module is specifically configured to:

[0047] Obtaining the playback speed of the first device based on the number of frames played by the first device at the last preset sampling moment within the first preset time period, and obtaining the playback speed of the second device based on the number of frames played by the second device at the last preset sampling moment within the first preset time period;

[0048] Obtaining a first preset number of frames and a second preset number of frames based on a playback speed of the first device, a playback speed of the second device, and a preset number of repeated frames; wherein a ratio of the first preset number of frames to the playback speed of the first device is equal to a ratio of the second preset number of frames to the playback speed of the second device, and the second preset number of frames is equal to the sum of the number of repeated frames and the first preset number of frames;

[0049] The second device is configured to repeatedly play the audio and video with a preset number of repeated frames every time it plays the audio and video with a second preset number of frames after the start of the second preset time period, wherein the audio and video with a preset number of repeated frames is the audio and video with a preset number of repeated frames located at the end of the audio and video with the second preset number of frames.

[0050] In an optional embodiment of the present application, the playback progress adjustment module is further configured to:

[0051] For each first preset time period after the second preset time period, obtaining a first variance of the corresponding number of played frames based on the number of played frames of each device at each preset sampling moment in a second preset time period that is earlier than the first of the two preset time periods before the first preset time period, and obtaining a second variance of the corresponding number of played frames based on the number of played frames of each device at each preset sampling moment in a third preset time period that is later than the first preset time period.

[0052] For each preset sampling moment, compare the first variance and the second variance corresponding to the preset sampling moment, and obtain the proportion of the preset sampling moments in which the first variance is greater than the second variance;

[0053] If the ratio is less than the preset ratio value, obtaining a third device whose playback status is delayed in the third preset time period and whose playback progress has been adjusted in the third preset time period, and canceling the adjustment of the playback progress of the third device after the first preset time period starts;

[0054] If the ratio is not less than the preset ratio value, then obtain the fourth device with the least number of played frames at the last preset sampling moment in the third preset time period and the fifth device whose playback status is early playback in the third preset time period, and adjust the playback progress of the fifth device after the start of the first preset time period.

[0055] In an optional embodiment of the present application, the module for determining the device requiring adjustment may also be used to:

[0056] For each preset sampling moment, obtain the median of the number of played frames of each device at the preset sampling moment, and obtain the relationship between the number of played frames of each device and the median;

[0057] Based on the relationship between the number of played frames of each device at each preset sampling moment and the corresponding median, a third device is determined from the devices whose play progress has been adjusted in the third preset time period.

[0058] In an optional embodiment of the present application, the module for determining the device requiring adjustment may also be used to:

[0059] For each preset sampling moment, use the first mark to mark the device whose number of played frames is less than the median corresponding to the preset sampling moment, and use the second mark to mark the device whose number of played frames is greater than the median corresponding to the preset sampling moment;

[0060] For each device, performing a signed rank sum test on the number of first marks and the second number of marks marked on the device within a third preset time period to obtain a test result corresponding to the device;

[0061] The third device is determined from the devices whose playback progress has been adjusted in the third preset time period by a device whose inspection result shows that the number of first marks exceeds the number of second marks which is a preset multiple.

[0062] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory;

[0063] The processor executes the computer program to implement the method provided in the embodiment of the first aspect or any optional embodiment of the first aspect.

[0064] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the method provided in the embodiment of the first aspect or any optional embodiment of the first aspect is implemented.

[0065] The beneficial effects of the technical solution provided by the embodiments of the present application are:

[0066] By dividing the audio and video into multiple preset time periods, the playback progress of devices with excessively fast playback speeds is adjusted at the end of each preset time period, so that the playback speed of each device is as close to the same as possible at the beginning of the next preset time period. This makes it impossible for the user to physiologically perceive the difference in audio and video playback between different devices, thus achieving audio and video synchronization. Furthermore, the solution of this application adjusts the playback progress of each device in each preset time period and is not affected by hardware and software performance differences between different devices or interference from external factors. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for describing the embodiments of the present application.

[0068] Figure 1 A flowchart of a method for implementing multi-device audio and video synchronization provided in an embodiment of the present application;

[0069] Figure 2 This is a flowchart of a method for determining a second device in an example of an embodiment of the present application;

[0070] Figure 3 This is a flowchart of a method for adjusting playback progress in an example of an embodiment of the present application;

[0071] Figure 4 A structural block diagram of a multi-device audio and video synchronization apparatus provided in an embodiment of the present application;

[0072] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] The following describes the embodiments of the present application in conjunction with the accompanying drawings. It should be understood that the embodiments described below in conjunction with the accompanying drawings are exemplary descriptions for explaining the technical solutions of the embodiments of the present application and do not constitute a limitation on the technical solutions of the embodiments of the present application.

[0074] Those skilled in the art will understand that, unless otherwise stated, the singular forms "a", "an", "said", and "the" used herein may also include plural forms. It should be further understood that the terms "including" and "comprising" used in the embodiments of the present application mean that the corresponding features can be implemented as the presented features, information, data, steps, operations, elements, and / or components, but do not exclude implementation as other features, information, data, steps, operations, elements, components, and / or combinations thereof supported by the present technical field. It should be understood that when we say that an element is "connected" or "coupled" to another element, the element can be directly connected or coupled to the other element, or it can refer to the element and the other element establishing a connection relationship through an intermediate element. In addition, the "connection" or "coupling" used here can include wireless connection or wireless coupling. The term "and / or" used here indicates at least one of the items defined by the term, for example, "A and / or B" can be implemented as "A", or as "B", or as "A and B".

[0075] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0076] The following describes several exemplary embodiments to illustrate the technical solutions of the embodiments of the present application and the technical effects produced by the technical solutions of the present application. It should be noted that the following embodiments can refer to, draw on, or combine with each other, and the same terms, similar features, and similar implementation steps in different embodiments will not be repeated.

[0077] Figure 1 A flow chart of a method for implementing multi-device audio and video synchronization is provided for an embodiment of the present application. The execution subject of the method can be an audio and video playback control terminal or an audio and video playback device. Generally, the control relationship between the audio and video playback control terminal and the audio and video playback device is relatively weak. The audio and video playback control terminal usually only has the functions of providing the audio and video materials to be played (referred to as audio and video when there is no ambiguity) and related data to the audio and video playback device, issuing audio and video playback instructions, collecting or receiving audio and video playback status feedback of the audio and video playback device, and has no right to interfere with the audio and video playback device in performing other tasks. Different audio and video playback devices have no control relationship between each other, and cannot even transmit data directly to each other (of course, data can be transmitted indirectly through the audio and video playback control terminal). For example, the audio and video playback control terminal can be a server, and the audio and video playback device can be a terminal device such as a mobile phone or a smart TV. For example Figure 1 As shown, the method may include:

[0078] Step S101: divide the audio and video into at least two preset time periods in sequence.

[0079] The preset time period is a partial segment of the audio or video, and is used to divide a target audio or video into at least two segments.

[0080] Specifically, after receiving the audio and video that needs to be synchronized, the audio and video playback control terminal will divide the audio and video into at least two preset time periods according to the length of the audio and video, that is, divide the audio and video into at least two segments.

[0081] It should be noted that, except for the last preset time period, the durations of the other preset time periods are the same.

[0082] Step S102: Obtain the number of played frames of different devices at at least one preset sampling moment within a first preset time period.

[0083] Among them, the first preset time period is the first preset time period in the above steps that includes the moment when the audio and video start playing; the preset sampling moment is the sampling time point pre-set in each preset time period. Whenever the audio and video are played to the sampling time point, the audio and video playback control terminal or the audio and video playback device (hereinafter referred to as the device) will perform a sampling count on the number of played frames; the number of played frames refers to the total number of frames of audio and video played from the start of this preset time period to this preset sampling moment.

[0084] Specifically, when the audio and video starts playing, the number of played frames of the audio and video in each device is counted from the start of the first preset time period (that is, the time when the audio and video starts playing). At each preset sampling time, the number of played frames of each device from the start of the first preset time period to the preset sampling time is recorded until the audio and video plays to the end of the first preset time period, and the counting of the played frames of each device is stopped.

[0085] It should be noted that the number and corresponding positions of the preset sampling moments set for each device in each preset time period (i.e., the time difference relative to the start time of the preset time period) are the same, and a preset sampling moment must be set at the end time of the preset time period to record the total number of frames played by each device in the preset time period; and in the last preset time period, no preset sampling moment is set.

[0086] For example, it is assumed that the first preset time period is from 00:00 to 02:00, and the preset sampling times are set to 00:30, 01:00, 01:30, and 02:00 respectively. When the audio and video starts playing (i.e., 00:00), start counting the number of frames played on each device. When the audio and video plays to 00:30, record the number of frames played on each device from 00:00 to 00:30 (e.g., 2000 frames). When playing to 01:00, record the number of frames played on each device from 00:00 to 01:00 (e.g., 4000 frames). When playing to 01:30, record the number of frames played on each device from 00:00 to 01:30 (e.g., 6000 frames). When playing to 02:00, record the number of frames played on each device from 00:00 to 02:00 (e.g., 8000 frames, i.e., the total number of frames played by the device in the first preset time period), and stop counting.

[0087] Step S103 : determining the device with the least played frames at the last preset sampling moment as the first device, and determining the second device with the advanced playback status based on the played frames of each device at each preset sampling moment.

[0088] The last preset sampling moment is the end moment of the preset time period. The first device is the device that has played the least number of frames in the first preset time period. The second device is the device that plays in advance in the first preset time period.

[0089] Specifically, after obtaining the number of frames played by all devices in a first preset time period, the device with the least number of frames played in the first preset time period is designated as the first device. The playback status of each device is then determined based on the number of frames played by all devices at all preset sampling moments in the first preset time period, and all devices with an early playback status are designated as the second device. The playback status may include early playback, delayed playback, and normal playback.

[0090] It should be noted that, if there are multiple devices with the same minimum number of played frames, these devices may all be used as the first device.

[0091] Step S104 : For each second device, adjust the playback progress of the second device within a second preset time period.

[0092] The second preset time period is the first preset time period after the first preset time period from the start of the audio and video playback, and the playback progress can be represented by the total number of frames of the audio and video played within the preset time period.

[0093] Specifically, after the first preset time period ends, the first device and the second device in the first preset time period are obtained, and at the start of the second preset time period, the playback progress of the second device is adjusted according to the playback progress of the first device, so that the playback progress of the second device and the first device are synchronized as much as possible after the start of the second preset time period.

[0094] The solution provided by this application achieves audio and video synchronization by dividing the audio and video into multiple preset time periods. At the end of each preset time period, the playback progress of devices with excessively fast playback speeds is adjusted, so that the playback speed of each device is as close to the same as possible at the beginning of the next preset time period. This makes it impossible for the user to physiologically perceive the difference in audio and video playback between different devices, thereby achieving the effect of audio and video synchronization. Furthermore, the solution of this application adjusts the playback progress of each device in each preset time period and is not affected by hardware and software performance differences between different devices or interference from external factors.

[0095] In an optional embodiment of the present application, determining the second device whose playback status is early playback based on the number of played frames of each device at each preset sampling moment includes:

[0096] For each preset sampling moment, obtain the median of the number of played frames of each device at the preset sampling moment, and obtain the relationship between the number of played frames of each device and the median;

[0097] The second device is determined based on the relationship between the number of played frames of each device at each preset sampling moment and the corresponding median.

[0098] The median is calculated by sorting the number of played frames of each device at each preset sampling moment from small to large, and each preset sampling moment has a corresponding median.

[0099] Specifically, if Figure 2 As shown, for each preset sampling moment, the number of frames played by each device at that preset sampling moment is sorted from smallest to largest. After sorting, the median corresponding to that preset sampling moment is calculated. The number of frames played by each device at that preset sampling moment is then compared with the median. Finally, based on the comparison results for each device at all preset sampling moments, the second device is determined. The median of N numbers is typically defined as follows: sort the N numbers from largest to smallest or smallest to largest. If N = 2k + 1, the k + 1th number is the median. If N = 2k, the median is the average of the k and k + 1th numbers.

[0100] In an optional embodiment of the present application, determining the second device based on the relationship between the number of played frames of each device at each preset sampling moment and the corresponding median includes:

[0101] For each preset sampling moment, use the first mark to mark the device whose number of played frames is less than the median corresponding to the preset sampling moment, and use the second mark to mark the device whose number of played frames is greater than the median corresponding to the preset sampling moment;

[0102] For each device, performing a signed rank sum test on the number of first marks and the second number of marks marked on the device within a first preset time period to obtain a test result corresponding to the device;

[0103] The second device is determined as a device whose inspection result shows that the number of the second mark exceeds the preset multiple of the number of the first mark.

[0104] The first mark indicates that the number of frames played by the device at the preset sampling time is less than the median corresponding to the preset sampling time, and the second mark indicates that the number of frames played by the device at the preset sampling time is greater than the median corresponding to the preset sampling time. In actual operation, the first mark can be "-" and the second mark can be "+". A common signed rank test method is to substitute the number of first marks m and the number of second marks n of each device in the preset time period into the formula χ 2 =(|nm|-1) 2 / (n+m), calculate χ 2 , then χ 2 Compare with a preset control value.

[0105] Specifically, if Figure 2 As shown, for each preset sampling moment, the device whose number of played frames at the preset sampling moment is less than the median corresponding to the preset sampling moment is marked with a first mark, and the device whose number of played frames at the preset sampling moment is greater than the median corresponding to the preset sampling moment is marked with a second mark. Then, the number m of the first mark and the number n of the second mark of each device at the preset sampling moment are counted, and m and n are substituted into the formula χ 2 =(|nm|-1) 2 / (n+m) to obtain χ 2 Then, 2 The value of is compared with the corresponding preset control value in the table of critical values ​​of the signed rank test. 2 The value of is greater than the preset control value and n is greater than m, then it can be considered that the playback status of the device in the preset time period is playback advance; if χ 2 The value of is greater than the preset control value and m is greater than n, then it can be considered that the playback condition of the device in the preset time period is playback lag; if 2 If the value of is less than or equal to the preset control value, it can be considered that the playback status of the device in the preset time period is normal.

[0106] The device with the playback status of "playing ahead" is regarded as the second device. For example, in the first preset time period, 10 preset sampling moments are set. The median corresponding to each of the 10 preset sampling moments is calculated based on the number of frames played by each device at each preset sampling moment. Then, the number of frames played by device A at each preset sampling moment in the preset time period is compared with the corresponding median, and two first marks and eight second marks are marked. After calculation, χ 2 The value of = 4 is greater than the corresponding preset control value in the signed rank test critical value table, so device A can be regarded as a device whose playback condition in the first preset time period is playback advance, and device A can be regarded as the second device.

[0107] In an optional embodiment of the present application, for each second device, adjusting the playback progress of the second device within the second preset time period includes:

[0108] Obtaining the playback speed of the first device based on the number of frames played by the first device at the last preset sampling moment within the first preset time period, and obtaining the playback speed of the second device based on the number of frames played by the second device at the last preset sampling moment within the first preset time period;

[0109] Obtaining a first preset number of frames and a second preset number of frames based on a playback speed of the first device, a playback speed of the second device, and a preset number of repeated frames; wherein a ratio of the first preset number of frames to the playback speed of the first device is equal to a ratio of the second preset number of frames to the playback speed of the second device, and the second preset number of frames is equal to the sum of the number of repeated frames and the first preset number of frames;

[0110] The second device is configured to repeatedly play the audio and video with a preset number of repeated frames every time it plays the audio and video with a second preset number of frames after the start of the second preset time period, wherein the audio and video with a preset number of repeated frames is the audio and video with a preset number of repeated frames located at the end of the audio and video with the second preset number of frames.

[0111] The playback speed is calculated by dividing the number of frames played by the device at the last preset sampling moment in the preset time period by the total duration of the preset time period. The preset number of repeated frames is the preset number of frames that the second device, after adjusting, needs to repeat after playing a certain number of frames in the next preset time period, compared to the first device. The first preset number of frames and the second preset number of frames indicate that, at the same time in the first preset time period, the first device played the first preset number of frames of audio or video, and the second device played the second preset number of frames of audio or video.

[0112] Specifically, if Figure 3As shown, the playback speed of the first device during the first preset time period is calculated by dividing the number of frames played by the first device at the last preset sampling moment of the first preset time period by the total duration of the preset time period. The playback speed of the second device is calculated using the same method. Then, based on the condition that the ratio of the first preset number of frames to the first device's playback speed is equal to the ratio of the second preset number of frames to the second device's playback speed, equations are established to calculate the first and second preset numbers of frames. After the start of the second preset time period, each time the second device plays audio or video of the second preset number of frames, the preset number of repeated frames of the audio or video within the second preset number of frames is repeatedly played to wait for the playback progress of the first device. The preset number of repeated frames within the second preset number of frames is generally located at the end of the second preset number of frames and can be the last frame or the last few frames of the second preset number of frames. To ensure synchronization, the preset number of repeated frames should not be too large. This ensures that the number of frames played by the first and second devices during the same time period in the second time period is as close as possible to the same.

[0113] It should be noted that at the preset sampling moment of the second preset time period, for audio and video with a preset number of repeated frames, the count of any frame included in the number of played frames is 1, and the count of the number of played frames is not increased because it is a preset number of repeated frames.

[0114] For example, assume that the first preset time period is 180 seconds long and the preset number of repeated frames is 1 frame. At the end of the preset time period, the number of frames played by the second device A is 5400, while the number of frames played by the first device B is 5220. By calculation, the playback speed of the second device A is 30 frames per second, and the playback speed of the first device B is 29 frames per second. Based on the above data, the equation is established: The calculated M (i.e., the first preset number of frames) is 29, and M+1 (i.e., the second preset number of frames) is 30. This means that in the first preset time period, for every 30 frames of audio and video played by the second device A, the first device B only played 29 frames of audio and video. Then after the start of the second preset time period, for every 30 frames of audio and video played by the second device A, it will repeatedly play the last frame of audio and video in the audio and video segment, and wait for the first device B to play to the 30th frame of the audio and video segment, to ensure that the playback progress of device A and device B is as synchronized as possible. It should be noted that if the M calculated according to the equation is not an integer, the integer closest to the result of the equation will be used as M.

[0115] In an optional embodiment of the present application, the method may further include:

[0116] For each first preset time period after the second preset time period, obtaining a first variance of the corresponding number of played frames based on the number of played frames of each device at each preset sampling moment in a second preset time period that is earlier than the first of the two preset time periods before the first preset time period, and obtaining a second variance of the corresponding number of played frames based on the number of played frames of each device at each preset sampling moment in a third preset time period that is later than the first preset time period.

[0117] For each preset sampling moment, compare the first variance and the second variance corresponding to the preset sampling moment, and obtain the proportion of the preset sampling moments in which the first variance is greater than the second variance;

[0118] If the ratio is less than the preset ratio value, obtaining a third device whose playback status is delayed in the third preset time period and whose playback progress has been adjusted in the third preset time period, and canceling the adjustment of the playback progress of the third device after the first preset time period starts;

[0119] If the ratio is not less than the preset ratio value, then obtain the fourth device with the least number of played frames at the last preset sampling moment in the third preset time period and the fifth device whose playback status is early playback in the third preset time period, and adjust the playback progress of the fifth device after the start of the first preset time period.

[0120] Among them, the first preset time period is each preset time period after the second preset time period from the start time of the audio and video, the second preset time period is the earlier preset time period of the two preset time periods before the first preset time period, and the third preset time period is the later preset time period of the two preset time periods before the first preset time period. The first variance is the variance of the number of played frames of each device at each preset sampling moment in the second preset time period, the second variance is the variance of the number of played frames of each device at each preset sampling moment in the third preset time period, and the preset ratio value is used to indicate the relationship between all first variances and all second variances. The third device is a device whose playback status in the third preset time period is playback lag and whose playback progress has been adjusted in the third preset time period. The fourth device is a device that has played the least number of frames in the third preset time period. The fifth device is a device whose playback status in the third preset time period is playback advance.

[0121] Specifically, for each first preset time period, first obtain the number of played frames of each device at each preset sampling moment of the corresponding second preset time period and third preset time period, and use the number of played frames of each device at each preset sampling moment as a group of data. Calculate the variance of each group of data to obtain the variance of each preset sampling moment. The variance of each preset sampling moment of the second preset time period is the first variance, and the variance of each preset sampling moment of the third preset time period is the second variance. Then compare the first variance with the second variance at the same position of the corresponding preset sampling moment, record the number of times the first variance is greater than the second variance, and then calculate the proportion of the number of times the first variance is greater than the second variance to the number of preset sampling moments in the preset time period.

[0122] Specifically, if the ratio value is less than the preset ratio value, it means that the adjustment of the device playback progress in the third preset time period may be unreasonable. Since the adjustment is to repeatedly play a certain number of frames of audio and video, that is, slowing down the playback progress of the device, the third device whose playback situation is delayed and has been adjusted in the third preset time period will be determined at this time. After the start of the first preset time period, the possible unreasonable adjustment of the third device will be canceled so that it no longer repeatedly plays a certain number of frames of audio and video; if the ratio value is not less than the preset ratio value, it means that the adjustment of the device playback progress in the third preset time period may be reasonable, then the same adjustment method will be adopted to continue to adjust the devices that need to be adjusted after the first preset time period, that is, determine the fourth device that has played the least number of frames in the third preset time period and the fifth device whose playback situation is advanced in the third preset time period, and make the fifth device repeat a certain number of frames of audio and video every time it plays a certain number of frames of audio and video after the start of the first preset time period to ensure that the playback progress of the fourth device and the fifth device are synchronized.

[0123] It should be noted that if it is found that there is no third device that meets the conditions during the process of obtaining the third device, it is considered that some devices may be interfered with by external factors or accidental factors within the device during the third preset time period. At this time, the same operation as satisfying the "ratio value is less than the preset ratio value" condition is taken, that is, the fourth device and the fifth device in the third preset time period are determined and the playback progress of the fifth device is adjusted after the start of the first preset time period.

[0124] It should be noted that, if it is found that no fifth device that meets the conditions exists during the process of acquiring the fifth device, no operation will be performed after the first preset time period starts.

[0125] For example, the number of preset sampling moments in each preset time period is 10, and the preset ratio value is 60%. At the beginning of the third preset time period, the number of played frames of each device at each preset sampling moment in the first two preset time periods (i.e., the first preset time period and the second preset time period) is obtained, and then the variance of each preset sampling moment is calculated based on the number of played frames of each device at each preset sampling moment. The first variance is the variance of each preset sampling moment in the first preset time period (i.e., the second preset time period), and the second variance is the variance of each preset sampling moment in the second preset time period (i.e., the third preset time period). Then, each first variance is compared with the second variance of the corresponding position ( For example, the variance of the fourth preset sampling moment of the first preset time period is compared with the variance of the fourth preset sampling moment of the second preset time period), and the number of times the first variance is greater than the second variance at the corresponding position is recorded as 8, accounting for 80% of the total number of preset sampling moments, exceeding the preset proportion value of 60%. At this time, device A will be determined from the second preset time period to be in a delayed playback state, and device A is set to repeat playing one frame of audio and video for every 30 frames of audio and video at the beginning of the second preset time period. In this case, device A is determined to be the third device, and after the start of the third preset time period, device A will stop repeating playing one frame of audio and video for every 30 frames of audio and video.

[0126] In an optional embodiment of the present application, obtaining a third device whose playback status in a third preset time period is playback lag and whose playback progress has been adjusted in the third preset time period includes:

[0127] For each preset sampling moment, obtain the median of the number of played frames of each device at the preset sampling moment, and obtain the relationship between the number of played frames of each device and the median;

[0128] Based on the relationship between the number of played frames of each device at each preset sampling moment and the corresponding median, a third device is determined from the devices whose play progress has been adjusted in the third preset time period.

[0129] Specifically, the process of calculating the median of the third preset time period and obtaining the relationship between the number of played frames and the median for each device is the same as the process of calculating the median of the first preset time period and obtaining the relationship between the number of played frames and the median for each device described above, and will not be further described here. Unlike the above description, the second device obtained above is a device with an advanced playback status, while the third device obtained here is a device with a delayed playback status and whose playback progress has been adjusted during the preset time period.

[0130] In an optional embodiment of the present application, determining the third device from the devices whose playback progress has been adjusted in the third preset time period based on the relationship between the number of played frames of each device at each preset sampling moment and the corresponding median includes:

[0131] For each preset sampling moment, use the first mark to mark the device whose number of played frames is less than the median corresponding to the preset sampling moment, and use the second mark to mark the device whose number of played frames is greater than the median corresponding to the preset sampling moment;

[0132] For each device, performing a signed rank sum test on the number of first marks and the second number of marks marked on the device within a third preset time period to obtain a test result corresponding to the device;

[0133] The third device is determined from the devices whose playback progress has been adjusted in the third preset time period by a device whose inspection result shows that the number of first marks exceeds the number of second marks which is a preset multiple.

[0134] Specifically, the process of marking the device in the third preset time period here is the same as the process of marking the device in the first preset time period described above, and the signed rank sum test method is also the same as described above, and will not be elaborated here.

[0135] It should be noted that, in the foregoing description, the device whose playback status is advanced is regarded as the second device, and here the device whose playback status is delayed and whose playback progress has been adjusted in the third preset time period is regarded as the third device.

[0136] Figure 4 This is a structural block diagram of a multi-device audio and video synchronization device provided in an embodiment of the present application, such as Figure 4 As shown, the multi-device audio and video synchronization apparatus 400 may include: a preset time period setting module 401, a playback frame number sampling module 402, a device to be adjusted determining module 403 and a playback progress adjusting module 404, wherein,

[0137] The preset time period setting module 401 is used to divide the audio and video into at least two preset time periods in sequence;

[0138] The playback frame number sampling module 402 is used to obtain the number of played frames of different devices at at least one preset sampling moment within a first preset time period;

[0139] The device-to-be-adjusted determining module 403 is configured to determine the device with the least number of played frames at the last preset sampling moment among the devices as the first device, and determine the second device whose playback condition is advanced based on the number of played frames of each device at each preset sampling moment;

[0140] The play progress adjustment module 404 is configured to adjust the play progress of each second device within a second preset time period.

[0141] The solution provided by this application achieves audio and video synchronization by dividing the audio and video into multiple preset time periods. At the end of each preset time period, the playback progress of devices with excessively fast playback speeds is adjusted, so that the playback speed of each device is as close to the same as possible at the beginning of the next preset time period. This makes it impossible for the user to physiologically perceive the difference in audio and video playback between different devices, thereby achieving the effect of audio and video synchronization. Furthermore, the solution of this application adjusts the playback progress of each device in each preset time period and is not affected by hardware and software performance differences between different devices or interference from external factors.

[0142] In an optional embodiment of the present application, the module for determining the device requiring adjustment is specifically configured to:

[0143] For each preset sampling moment, obtain the median of the number of played frames of each device at the preset sampling moment, and obtain the relationship between the number of played frames of each device and the median;

[0144] The second device is determined based on the relationship between the number of played frames of each device at each preset sampling moment and the corresponding median.

[0145] In an optional embodiment of the present application, the module for determining the device requiring adjustment is further configured to:

[0146] For each preset sampling moment, use the first mark to mark the device whose number of played frames is less than the median corresponding to the preset sampling moment, and use the second mark to mark the device whose number of played frames is greater than the median corresponding to the preset sampling moment;

[0147] For each device, performing a signed rank sum test on the number of first marks and the second number of marks marked on the device within a first preset time period to obtain a test result corresponding to the device;

[0148] The second device is determined as a device whose inspection result shows that the number of the second mark exceeds the preset multiple of the number of the first mark.

[0149] In an optional embodiment of the present application, the playback progress adjustment module is specifically configured to:

[0150] Obtaining the playback speed of the first device based on the number of frames played by the first device at the last preset sampling moment within the first preset time period, and obtaining the playback speed of the second device based on the number of frames played by the second device at the last preset sampling moment within the first preset time period;

[0151] Obtaining a first preset number of frames and a second preset number of frames based on a playback speed of the first device, a playback speed of the second device, and a preset number of repeated frames; wherein a ratio of the first preset number of frames to the playback speed of the first device is equal to a ratio of the second preset number of frames to the playback speed of the second device, and the second preset number of frames is equal to the sum of the number of repeated frames and the first preset number of frames;

[0152] The second device is configured to repeatedly play the audio and video with a preset number of repeated frames every time it plays the audio and video with a second preset number of frames after the start of the second preset time period, wherein the audio and video with a preset number of repeated frames is the audio and video with a preset number of repeated frames located at the end of the audio and video with the second preset number of frames.

[0153] In an optional embodiment of the present application, the playback progress adjustment module is further configured to:

[0154] For each first preset time period after the second preset time period, obtaining a first variance of the corresponding number of played frames based on the number of played frames of each device at each preset sampling moment in a second preset time period that is earlier than the first of the two preset time periods before the first preset time period, and obtaining a second variance of the corresponding number of played frames based on the number of played frames of each device at each preset sampling moment in a third preset time period that is later than the first preset time period.

[0155] For each preset sampling moment, compare the first variance and the second variance corresponding to the preset sampling moment, and obtain the proportion of the preset sampling moments in which the first variance is greater than the second variance;

[0156] If the ratio is less than the preset ratio value, obtaining a third device whose playback status is delayed in the third preset time period and whose playback progress has been adjusted in the third preset time period, and canceling the adjustment of the playback progress of the third device after the first preset time period starts;

[0157] If the ratio is not less than the preset ratio value, then obtain the fourth device with the least number of played frames at the last preset sampling moment in the third preset time period and the fifth device whose playback status is early playback in the third preset time period, and adjust the playback progress of the fifth device after the start of the first preset time period.

[0158] In an optional embodiment of the present application, the module for determining the device requiring adjustment may also be used to:

[0159] For each preset sampling moment, obtain the median of the number of played frames of each device at the preset sampling moment, and obtain the relationship between the number of played frames of each device and the median;

[0160] Based on the relationship between the number of played frames of each device at each preset sampling moment and the corresponding median, a third device is determined from the devices whose play progress has been adjusted in the third preset time period.

[0161] In an optional embodiment of the present application, the module for determining the device requiring adjustment may also be used to:

[0162] For each preset sampling moment, use the first mark to mark the device whose number of played frames is less than the median corresponding to the preset sampling moment, and use the second mark to mark the device whose number of played frames is greater than the median corresponding to the preset sampling moment;

[0163] For each device, performing a signed rank sum test on the number of first marks and the second number of marks marked on the device within a third preset time period to obtain a test result corresponding to the device;

[0164] The third device is determined from the devices whose playback progress has been adjusted in the third preset time period by a device whose inspection result shows that the number of first marks exceeds the number of second marks which is a preset multiple.

[0165] Reference below Figure 5 , which shows an electronic device suitable for implementing the embodiments of the present application (for example, Figure 1 The electronic devices in the embodiments of the present application may include, but are not limited to, mobile terminals such as mobile phones, laptop computers, digital broadcast receivers, PDAs (personal digital assistants), PADs (tablet computers), PMPs (portable multimedia players), in-vehicle terminals (such as in-vehicle navigation terminals), wearable devices, and fixed terminals such as digital TVs and desktop computers. Figure 5 The electronic device shown is merely an example and should not limit the functions and scope of use of the embodiments of the present application.

[0166] The electronic device includes: a memory and a processor, wherein the memory is used to store a program for executing the methods described in each of the above method embodiments; and the processor is configured to execute the program stored in the memory. The processor here may be referred to as the processing device 501 described below, and the memory may include at least one of the read-only memory (ROM) 502, the random access memory (RAM) 503, and the storage device 508 described below, as shown below:

[0167] like Figure 5As shown, the electronic device 500 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 501, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 502 or a program loaded from a storage device 508 into a random access memory (RAM) 503. Various programs and data required for the operation of the electronic device 500 are also stored in the RAM 503. The processing device 501, the ROM 502, and the RAM 503 are connected to each other via a bus 504. An input / output (I / O) interface 505 is also connected to the bus 504.

[0168] Typically, the following devices may be connected to the I / O interface 505: an input device 506 including, for example, a touch screen, a touchpad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 507 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 508 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 509. The communication device 509 may allow the electronic device 500 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 5 The electronic device is shown with various devices, but it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed instead.

[0169] In particular, according to an embodiment of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a non-transitory computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication device 509, or installed from the storage device 508, or installed from the ROM 502. When the computer program is executed by the processing device 501, the above-mentioned functions defined in the method of the embodiment of the present application are performed.

[0170] It should be noted that the computer-readable storage medium mentioned above in this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. The computer-readable storage medium can be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination of the above. More specific examples of computer-readable storage media can include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this application, a computer-readable storage medium can be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. This propagated data signal can take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium that can transmit, propagate, or transport a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination thereof.

[0171] In some embodiments, the client and server can communicate using any currently known or future developed network protocol, such as HTTP (HyperText Transfer Protocol), and can be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0172] The computer-readable medium may be included in the electronic device, or may exist independently without being incorporated into the electronic device.

[0173] The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device:

[0174] The audio and video are divided into at least two preset time periods in sequence; the number of played frames of different devices at at least one preset sampling moment in the first preset time period is obtained; the device with the least number of played frames at the last preset sampling moment among the devices is determined as the first device, and based on the number of played frames of each device at each preset sampling moment, the second device whose playback situation is advanced is determined; for each second device, the playback progress of the second device is adjusted within the second preset time period.

[0175] Computer program code for performing the operations of the present application may be written in one or more programming languages, or a combination thereof, including, but not limited to, object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0176] The flow charts and block diagrams in the accompanying drawings illustrate the possible architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In this regard, each box in the flow chart or block diagram can represent a module, program segment or a part of code, and the module, program segment or a part of code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in a different order than that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flow chart, and the combination of the boxes in the block diagram and / or flow chart can be implemented by a dedicated hardware-based system that performs the specified function or operation, or can be implemented by a combination of dedicated hardware and computer instructions.

[0177] The modules or units described in the embodiments of this application may be implemented in software or hardware. The name of a module or unit does not, in some cases, limit the unit itself. For example, the first constraint acquisition module may also be described as a "module for acquiring a first constraint."

[0178] The functions described above herein may be performed, at least in part, by one or more hardware logic components. For example, and without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0179] In the context of the present application, a machine-readable medium can be a tangible medium that can contain or store a program for use by an instruction execution system, device or equipment or used in combination with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0180] It should be understood that although the steps in the flowcharts of the accompanying drawings are shown in sequence as indicated by the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some of the steps in the flowcharts of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a portion of the sub-steps or stages of other steps.

[0181] The above descriptions are only partial embodiments of the present invention. It should be pointed out that ordinary technicians in this technical field can make several improvements and modifications without departing from the principles of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.

Claims

1. A multi-device audio and video synchronization method, characterized in that: include: Divide the audio and video into at least two preset time periods in sequence; Obtain the number of played frames of different devices at at least one preset sampling moment within a first preset time period; Determine the device with the least number of played frames at the last preset sampling moment among the devices as the first device, and determine the second device whose playback status is early playback based on the number of played frames of each device at each preset sampling moment; For each second device, adjusting the playback progress of the second device within a second preset time period; The method further comprises: For each first preset time period after the second preset time period, obtaining a first variance of the corresponding number of played frames based on the number of played frames of each device at each preset sampling moment in a second preset time period that is earlier than the first of the two preset time periods before the first preset time period; and obtaining a second variance of the corresponding number of played frames based on the number of played frames of each device at each preset sampling moment in a third preset time period that is later than the first preset time period; For each preset sampling moment, comparing the first variance and the second variance corresponding to the preset sampling moment, and obtaining the proportion of the preset sampling moments at which the first variance is greater than the second variance; If the ratio is less than the preset ratio value, obtaining a third device whose playback status is delayed during the third preset time period and whose playback progress has been adjusted during the third preset time period, and canceling the adjustment of the playback progress of the third device after the first preset time period starts; If the ratio is not less than the preset ratio value, then obtain the fourth device with the least number of played frames at the last preset sampling moment in the third preset time period and the fifth device whose playback status is early playback in the third preset time period, and adjust the playback progress of the fifth device after the start of the first preset time period.

2. The method according to claim 1, wherein The second device determining that the playback condition is playback advance based on the number of played frames of each device at each preset sampling moment includes: For each preset sampling moment, obtaining the median of the number of played frames of each device at the preset sampling moment, and obtaining the relationship between the number of played frames of each device and the median; The second device is determined based on the relationship between the number of played frames of each device at each preset sampling moment and the corresponding median.

3. The method according to claim 2, characterized in that The determining the second device based on the relationship between the number of played frames of each device at each preset sampling moment and the corresponding median includes: For each preset sampling moment, use a first mark to mark a device whose number of played frames is less than the median corresponding to the preset sampling moment, and use a second mark to mark a device whose number of played frames is greater than the median corresponding to the preset sampling moment; For each device, performing a signed rank sum test on the first number of marks and the second number of marks marked on the device within the first preset time period to obtain a test result corresponding to the device; The device whose inspection result shows that the number of the second mark exceeds the preset multiple of the number of the first mark is determined as the second device.

4. The method according to claim 1, wherein The step of adjusting the playback progress of each second device within the second preset time period includes: Obtaining a playback speed of the first device based on the number of frames played by the first device at the last preset sampling moment within the first preset time period, and obtaining a playback speed of the second device based on the number of frames played by the second device at the last preset sampling moment within the first preset time period; Obtaining a first preset number of frames and a second preset number of frames based on a playback speed of the first device, a playback speed of the second device, and a preset number of repeated frames; wherein a ratio of the first preset number of frames to the playback speed of the first device is equal to a ratio of the second preset number of frames to the playback speed of the second device, and the second preset number of frames is equal to the sum of the number of repeated frames and the first preset number of frames; The second device is configured to repeatedly play the audio and video with a preset number of repeated frames each time the audio and video with a second preset number of frames is played after the start of the second preset time period, wherein the audio and video with a preset number of repeated frames is the audio and video with a preset number of repeated frames located at the end of the audio and video with the second preset number of frames.

5. The method according to claim 1, wherein The acquiring of the third device that the playback condition in the third preset time period is playback lag and the playback progress has been adjusted in the third preset time period includes: For each preset sampling moment, obtaining the median of the number of played frames of each device at the preset sampling moment, and obtaining the relationship between the number of played frames of each device and the median; Based on the relationship between the number of played frames of each device at each preset sampling moment and the corresponding median, the third device is determined from the devices whose playback progress has been adjusted in the third preset time period.

6. The method according to claim 5, characterized in that The determining the third device from the devices whose playback progress has been adjusted in the third preset time period based on the relationship between the number of played frames of each device at each preset sampling moment and the corresponding median includes: For each preset sampling moment, use a first mark to mark a device whose number of played frames is less than the median corresponding to the preset sampling moment, and use a second mark to mark a device whose number of played frames is greater than the median corresponding to the preset sampling moment; For each device, performing a signed rank sum test on the first number of marks and the second number of marks marked on the device within the third preset time period to obtain a test result corresponding to the device; The third device is determined from the devices whose playback progress has been adjusted in the third preset time period by a device whose inspection result shows that the number of first marks exceeds the number of second marks which is a preset multiple.

7. A device for synchronizing audio and video of multiple devices, characterized in that: include: A preset time period setting module, used to divide the audio and video into at least two preset time periods in sequence; A playback frame number sampling module is used to obtain the number of played frames of different devices at at least one preset sampling moment within a first preset time period; The device determination module that needs to be adjusted is used to determine the device with the least number of played frames at the last preset sampling time among the devices as the first device, and determine the second device whose playback situation is ahead of schedule based on the number of played frames of each device at each preset sampling time; a playback progress adjustment module, configured to adjust the playback progress of each second device within a second preset time period; The playback progress adjustment module is further configured to: For each first preset time period after the second preset time period, obtaining a first variance of the corresponding number of played frames based on the number of played frames of each device at each preset sampling moment in a second preset time period that is earlier than the first of the two preset time periods before the first preset time period; and obtaining a second variance of the corresponding number of played frames based on the number of played frames of each device at each preset sampling moment in a third preset time period that is later than the first preset time period; For each preset sampling moment, comparing the first variance and the second variance corresponding to the preset sampling moment, and obtaining the proportion of the preset sampling moments at which the first variance is greater than the second variance; If the ratio is less than the preset ratio value, obtaining a third device whose playback status is delayed during the third preset time period and whose playback progress has been adjusted during the third preset time period, and canceling the adjustment of the playback progress of the third device after the first preset time period starts; If the ratio is not less than the preset ratio value, then obtain the fourth device with the least number of played frames at the last preset sampling moment in the third preset time period and the fifth device whose playback status is early playback in the third preset time period, and adjust the playback progress of the fifth device after the start of the first preset time period.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory, characterized in that: The processor executes the computer program to implement the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images