Audio and video synchronization method and device, electronic equipment and storage medium

Abstract

The invention discloses an audio and video synchronization method, which comprises the following steps of: firstly comparing whether the acquisition frame rates of audio and video are the same, if the acquisition frame rates are the same, considering that a picture with a target object is played, and dynamically adjusting the playing frame rate of a video frame, so that the audio frame and the video frame are kept synchronous in a playing process; specifically, the playing time difference between the audio frame and the video frame is calculated, and when the playing time difference exceeds a preset synchronization range, the refreshing interval of the next video frame is adjusted so that the audio frame and the video frame can be synchronized.

Description

Technical Field

[0001] The present invention belongs to video image processing technology, and in particular relates to an audio and video synchronization method, device, electronic equipment and storage medium. Background Art

[0002] The full real-time video recorder (AOV) can actively and regularly capture images at an extremely low frame rate (for example, 1fps / 1s) when there is no one or moving objects, instead of the device working after the PIR is triggered. When encapsulated into a video, the audio is filled with silent data. When the background algorithm recognizes that a target object appears on the monitoring screen, the AOV will use a certain frame rate to capture images and collect audio normally at the same time.

[0003] When playing back audio and video, the common method is to only play back at the original frame rate, which can easily cause the playback speed of the audio and video to not match during the playback process. As time goes by, the synchronization of audio and video will fail, affecting the viewing experience. Summary of the invention

[0004] In view of this, the present invention aims to provide an audio and video synchronization method, device, electronic device and storage medium, which can dynamically adjust the playback frame rate of audio frames and video frames to achieve synchronization between the two and ensure the smoothness of the AOV camera video playback screen.

[0005] In a first aspect, the present invention aims to provide an audio and video synchronization method, comprising:

[0006] Get the audio and video frame rates of the playback content;

[0007] Compare the audio capture frame rate and the video capture frame rate to see if they are the same, and if they are the same, perform the video frame playback adjustment process;

[0008] The video frame playback adjustment process includes:

[0009] Calculate the audio and video time difference between the audio frame and the video frame of the playback content at the current moment. When the audio and video time difference exceeds the preset synchronization range, adjust the playback frame rate of the video frame to synchronize it with the audio frame.

[0010] Further, when the audio and video time difference exceeds a preset synchronization range, adjusting the playback frame rate of the video frame to synchronize it with the audio frame includes:

[0011] Calculate the video frame time difference between the current video frame and the previous video frame;

[0012] The playback interval of the next video frame is calculated according to the video frame time difference, and the next video frame is refreshed and displayed according to the calculated playback interval.

[0013] Further, calculating the playback interval of the next video frame according to the video frame time difference includes:

[0014] Determine the relative position of the video frame and the audio frame at the current moment according to the audio and video time difference;

[0015] When the current video frame is later than the audio frame, the playback interval of the next video frame is calculated as the video frame time difference * n, 0 <n<1;

[0016] When the current video frame is earlier than the audio frame, the playback interval of the next video frame is calculated as the video frame time difference*m, where m>1.

[0017] Furthermore, the video frame playback adjustment process also includes:

[0018] When the time difference between the audio and video does not exceed the preset synchronization range, the playback frame rate of the video frame will not be adjusted.

[0019] Furthermore, the video frame playback adjustment process also includes:

[0020] When the audio and video time difference does not exceed the preset synchronization range, the playback interval of the next video frame is calculated based on the audio and video time difference and the video frame time difference.

[0021] Furthermore, the playback interval of the next video frame is the sum of the audio and video time difference and the video frame time difference.

[0022] Furthermore, the preset synchronization range includes -90ms~20ms.

[0023] Furthermore, the above method also includes:

[0024] When the audio capture frame rate is different from the video capture frame rate, adjust the video frame playback frame rate to 1fps.

[0025] Furthermore, obtaining the audio capture frame rate and the video capture frame rate of the playback content includes:

[0026] The audio data and video data of the playback content are decoded respectively to obtain the audio acquisition frame rate and the video acquisition frame rate.

[0027] In a second aspect, the present invention provides an audio and video synchronization device, comprising:

[0028] The acquisition frame rate acquisition module is used to acquire the audio acquisition frame rate and video acquisition frame rate of the playback content;

[0029] The frame rate difference calculation module is used to compare whether the audio acquisition frame rate and the video acquisition frame rate are the same;

[0030] The playback frame rate adjustment module is used to perform the following video frame playback adjustment process when the audio acquisition frame rate and the video acquisition frame rate are the same;

[0031] The video frame playback adjustment process includes:

[0032] Calculate the audio and video time difference between the audio frame and the video frame of the playback content at the current moment. When the audio and video time difference exceeds the preset synchronization range, adjust the playback frame rate of the video frame to synchronize it with the audio frame.

[0033] In a third aspect, the present invention provides an electronic device, comprising a memory storing computer executable instructions and a processor, wherein when the computer executable instructions are executed by the processor, the device executes each step of the audio and video synchronization method provided in the first aspect.

[0034] In a fourth aspect, the present invention provides a readable storage medium storing a computer executable program, which, when executed, can implement the various steps of the audio and video synchronization method provided in the first aspect.

[0035] It can be seen from the above technical solutions that the present invention has the following beneficial effects:

[0036] The present invention provides an audio and video synchronization method, which first compares whether the acquisition frame rates of audio and video are the same. If they are not the same, it can be considered that the captured scene is a picture of a scene without people or moving objects, and a low frame rate playback strategy can be adopted. If the acquisition frame rates are the same, it can be considered that there is a picture of a target object, and the playback frame rate of the video frame is dynamically adjusted to keep the audio frame and the video frame synchronized during the playback process. Specifically, the playback time difference between the audio frame and the video frame is judged. When the playback time difference exceeds a preset synchronization range, it is considered that the audio and video are seriously out of sync, and it is necessary to adjust the refresh interval of the next video frame to synchronize the audio frame and the video frame. The synchronization method provided by the present invention can better handle the drift problem of audio and video frames, and maintain the synchronization of audio and video by dynamically adjusting the playback frame rate of the video frame during the playback process. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying creative work.

[0038] Figure 1 This is a flow chart of an audio and video synchronization method provided by an embodiment of the present invention;

[0039] Figure 2 It is a schematic diagram of an AOV video recorder provided by an embodiment of the present invention using a dynamic acquisition frame rate for recording;

[0040] Figure 3 is a flow chart of an audio and video synchronization method provided by another embodiment of the present invention;

[0041] Figure 4 It is a structural schematic diagram of an audio and video synchronization device provided by an embodiment of the present invention;

[0042] Figure 5 This is a diagram of the electronic device architecture provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] See also Figure 1 An embodiment of the present invention provides an audio and video synchronization method, comprising the following steps:

[0045] Step S110: Obtain the audio capture frame rate and video capture frame rate of the playback content.

[0046] In this step, the AOV recorder supports recording the screen at a dynamic frame rate. When there is no target person or object in the screen, a low frame rate (for example, 1fps or 5fps) is often used for video recording to balance power consumption and the smoothness of the screen. When the target person or object appears in the screen, a higher frame rate video is used for recording. Then, the audio data and video data of the recorded content are decoded, and the audio acquisition frame rate and video acquisition frame rate are calculated. Figure 2 The figure illustrates a situation where an AOV recorder records a video at a dynamic frame rate. The I frame in the figure represents a key frame, and the P frame represents a forward reference frame.

[0047] Among them, the audio capture frame rate is usually expressed in the form of sampling rate (such as 44100 Hz or 48000 Hz). The decoded audio frame rate can be obtained by extracting the metadata of the audio file or decoding the original audio stream. The video capture frame rate is usually expressed in FPS (Frames Per Second) and can be obtained by decoding the metadata of the video stream.

[0048] Step S120: Compare the audio capture frame rate and the video capture frame rate to see if they are the same. If they are the same, proceed to step S130.

[0049] Specifically, the audio sampling rate and the video frame rate are compared. If the two are the same or the difference is within an acceptable range, it is determined that there may be a target object or person in the recorded image, that is, the audio frame corresponding to the video frame is not silent data. Since AOV recorders often use multiple frame rates combined with the captured image content, the frame rates of the video and audio are not completely consistent when the recorded content is fully played back. If the video playback frame rate is not dynamically adjusted during playback, the errors between the two will accumulate over time, resulting in audio and video asynchrony.

[0050] In a further embodiment, the support capability of the current playback environment for the video playback frame rate can be evaluated based on the performance parameters of the playback device to determine a suitable initial video playback frame rate. For example, when it is detected that the current playback end environment is insufficient due to insufficient network bandwidth or insufficient device decoding performance, a low frame rate playback instruction will be sent from the playback end to the recording terminal, and the recording terminal will only push low frame rate videos; when the playback end environment meets the high frame rate playback requirements, a high frame rate playback instruction will be sent from the playback end to the recording terminal, and the device terminal will push normal acquisition frame rate videos after receiving this instruction.

[0051] The playback device performance parameters used in the evaluation may include CPU usage, memory occupancy, GPU load, etc., and more preferably, playback network status parameters such as bandwidth, delay, jitter, etc. may also be considered.

[0052] Step S130. Calculate the audio and video time difference between the audio frame and the video frame of the playing content at the current moment, and when the audio and video time difference exceeds a preset synchronization range, adjust the playing frame rate of the video frame to synchronize it with the audio frame.

[0053] In this step, the playback timestamp (PTS) of the current playback moment can be obtained by decoding the audio stream and the video stream. These timestamps are used to calculate the playback time difference of the audio and video, and the time difference is compared with the preset synchronization range. If it exceeds the range (for example, ±0.05 seconds), the video playback frame rate needs to be adjusted by increasing or decreasing the playback rate of the video frame to synchronize it with the audio, which can be achieved by changing the video frame playback interval.

[0054] In a further embodiment, step S130 includes the following steps:

[0055] Step S131: Calculate the video frame time difference between the current video frame and the previous video frame.

[0056] Step S132: Calculate the playback interval of the next video frame according to the video frame time difference, and refresh and display the next video frame according to the calculated playback interval.

[0057] Synchronize the video frame with the audio frame, with the audio as the main time axis. During playback, determine how to change the playback frame rate of the video frame based on the playback time difference between the video frame and the audio frame to ensure that the playback time difference between the video frame and the audio frame is controlled within a certain range. Generally, when the playback offset between the audio and video frames is between -90 ms (in the embodiments of the present invention, it means that the audio frame is later than the video frame) and 20 ms (in the embodiments of the present invention, it means that the audio frame is earlier than the video frame), people can't feel the change in the audio-visual quality, and this interval can be considered as the synchronization area. Beyond this range, it is considered that the playback frame rate of the video frame needs to be adjusted.

[0058] Specifically, calculate the time difference between the current video frame and the previous video frame. This time difference represents the playback interval between the current frame and the previous frame. When the audio-video time difference exceeds the preset synchronization range, it is necessary to further determine the relative position between the video frame and the audio frame, that is, to determine whether the video frame is later or earlier than the audio frame. If the video frame is later than the audio frame, it is necessary to reduce the playback interval of the video frame so that the next frame can be displayed as soon as possible until the video frame catches up with the audio frame; if the video frame is earlier than the audio frame, it is necessary to increase the playback interval of the video frame so that the next frame is displayed with a delay until the audio frame catches up with the video frame.

[0059] In a further embodiment, the refresh interval of the next frame can be calculated based on the aforementioned video frame time difference, that is, the playback interval between the current video frame and the previous video frame. In a more preferred embodiment, when the video frame at the current moment is later than the audio frame, the playback interval of the next video frame is the video frame time difference * n, where 0 < n < 1; when the video frame at the current moment is earlier than the audio frame, the playback interval of the next video frame is the video frame time difference * m, where m > 1.

[0060] In a more preferred embodiment, when the playback time difference between the video frame and the audio frame exceeds the preset synchronization range and the video frame is later than the audio frame, the refresh interval of the next video frame is set to 0.8 * the video frame time difference. If the video frame is earlier than the audio frame, the refresh interval of the next video frame is set to 1.5 * the video frame time difference.

[0061] In a further embodiment, when the time difference between the audio and video is within the synchronization area, there are two cases. It may be that the audio frame and the video frame are absolutely synchronized, or there may be a certain distance between the audio frame and the video frame. Therefore, for the case where the preset synchronization range is not exceeded, the playback frame rate of the video frame (i.e., the refresh interval of the next frame) may not be adjusted, or the refresh interval of the next frame may be appropriately adjusted.

[0062] In a more preferred embodiment, the playback interval of the next video frame can be calculated based on the audio-video time difference and the video frame time difference. Specifically, the playback interval of the next video frame is set to the sum of the audio-video time difference and the video frame time difference.

[0063] In a further implementation, when the audio capture frame rate and the video capture frame rate are different, it can be considered that the picture recorded by the AOV recorder is a blank picture, and the playback frame rate of the video frame can be adjusted to a low frame rate such as 1fps.

[0064] See also Figure 3 An embodiment of the present invention provides a more specific audio and video synchronization method, including the following process:

[0065] Step S310: Detect performance parameters and network status parameters of the playback device.

[0066] Step S320: Evaluate the appropriate playback frame rate R based on the parameters detected in step S310.

[0067] Step S330: Decode the audio and video respectively to calculate the audio acquisition frame rate and the video acquisition frame rate.

[0068] Step S340: Compare whether the audio acquisition frame rate and the video acquisition frame rate are the same. If they are different, use a playback strategy of 1 frame per second, i.e. 1fps, to play the video. Otherwise, proceed to step S350.

[0069] Step S350: Calculate the video frame time difference delay between the current video frame and the previous video frame.

[0070] Step S360: Calculate the audio and video time difference diff between the audio frame and the video frame at the current moment.

[0071] Step S370. Determine whether the audio and video time difference diff exceeds the preset synchronization range, where the preset synchronization range is -90~20ms. If it does not exceed, go to step S380, otherwise go to step S390.

[0072] Step S380: Consider the playback frame rate R and set the refresh interval of the next frame delay'=delay+diff.

[0073] Step S390: When the video frame is later than the audio frame, the refresh interval of the next frame is delay'=0.8*delay; when the video frame is earlier than the audio frame, the refresh interval of the next frame is delay'=1.5*delay.

[0074] The above embodiments provide an audio and video synchronization method, which first compares whether the acquisition frame rates of the audio and video are the same. If they are not the same, it can be considered that the captured scene is a scene without people or moving objects, and a low frame rate playback strategy can be adopted. If the acquisition frame rates are the same, it can be considered that there is a target object. The playback frame rate of the video frame is dynamically adjusted to keep the audio frame and the video frame synchronized during the playback process. Specifically, the playback time difference between the audio frame and the video frame is determined. When the playback time difference exceeds the preset synchronization range, it is considered that the audio and video are seriously out of sync, and it is necessary to adjust the refresh interval of the next video frame to synchronize the audio frame and the video frame. The synchronization method provided by the present invention can better handle the drift problem of audio and video frames, and maintain the synchronization of audio and video by dynamically adjusting the playback frame rate of the video frame during the playback process.

[0075] The above method disclosed can be implemented by using various forms of equipment, so the present invention also discloses an audio and video synchronization device corresponding to the above method, and a specific embodiment is given below for detailed description.

[0076] like Figure 4 As shown, an embodiment of the present invention provides an audio and video synchronization device, including:

[0077] The acquisition frame rate acquisition module 402 is used to acquire the audio acquisition frame rate and the video acquisition frame rate of the playback content;

[0078] A frame rate difference calculation module 404 is used to compare whether the audio acquisition frame rate and the video acquisition frame rate are the same;

[0079] The playback frame rate adjustment module 406 is used to perform the following video frame playback adjustment process when the audio acquisition frame rate and the video acquisition frame rate are the same;

[0080] The video frame playback adjustment process includes:

[0081] Calculate the audio and video time difference between the audio frame and the video frame of the playback content at the current moment. When the audio and video time difference exceeds the preset synchronization range, adjust the playback frame rate of the video frame to synchronize it with the audio frame.

[0082] The implementation principle and technical effects of the device provided in the embodiments of the present application are the same as those of the aforementioned method embodiments. For the sake of brief description, for matters not mentioned in the device embodiment, reference may be made to the corresponding contents in the aforementioned method embodiments.

[0083] The methods and related devices mentioned in the above embodiments are described with reference to the method flow charts and / or structural diagrams provided in the embodiments of the present application. Specifically, each process and / or block in the method flow charts and / or structural diagrams, as well as the combination of processes and / or blocks in the flow charts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the process in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be stored in a computer-readable memory that can guide a computer or other programmable data processing device to work in a specific way, so that the instructions stored in the computer-readable memory produce a product including an instruction device, which implements the functions specified in the process. Figure 1 Schematic diagram of one or more processes and / or structures Figure 1 These computer program instructions can also be loaded onto a computer or other programmable data processing device, so that a series of operation steps are executed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide for implementing the process in the process. Figure 1 A flow or multiple flows and / or structures illustrate the steps of the functions specified in one block or multiple blocks.

[0084] The following embodiments are described by taking the method applied to a computer device as an example. It can be understood that the computer device can be any device with computing and processing functions, and can be but not limited to a server or a personal laptop computer, etc. In one embodiment, the computer device can be an application server, and the application server can be a server for running an application to be tested.

[0085] See also Figure 5 , which shows a hardware block diagram of an electronic device, the electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0086] like Figure 5As shown, the electronic device includes: at least one processor 1, at least one communication interface 2, at least one memory 3 and at least one communication bus 4;

[0087] In the embodiment of the present application, the number of the processor 1, the communication interface 2, the memory 3, and the communication bus 4 is at least one, and the processor 1, the communication interface 2, and the memory 3 communicate with each other through the communication bus 4;

[0088] The processor 1 may be a central processing unit CPU, or an application-specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention, etc.;

[0089] The memory 3 may include a high-speed RAM memory, and may also include a non-volatile memory (non-volatile memory), etc., such as at least one disk memory;

[0090] The memory stores a program, and the processor can call the program stored in the memory, and the program is used to: implement various processing flows of the aforementioned audio and video synchronization solution.

[0091] An embodiment of the present invention further provides a readable storage medium having a computer program stored thereon. When the computer program is executed by a processor, the processing flow of the audio and video synchronization solution provided in the above embodiment and / or any possible implementation method in combination with the embodiment is implemented.

[0092] The above-mentioned embodiments have described the present invention in particular detail with respect to possible scenarios, and those skilled in the art will recognize that the present invention can be practiced through other embodiments. The specific naming of components, the capitalization of terms, attributes, data structures, or any other programming or structural aspects are not mandatory or important, and the mechanisms or features of the present invention may have different names, forms, or procedures. The system may be implemented by a combination of hardware and software (as described), entirely by hardware elements, or entirely by software elements. The specific division of functions between the various system components described herein is exemplary only and not mandatory; on the contrary, the functions performed by a single system component may be performed by multiple components, or the functions performed by multiple components may be performed by a single component.

[0093] Those skilled in the art should understand that the various steps of the above disclosed method can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, optionally, they can be implemented with program codes executable by the computing device, so that they can be stored in a storage device and executed by the computing device, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the embodiments of the present invention are not limited to any specific combination of hardware and software.

[0094] These computing device executable programs (also referred to as programs, software, software applications, or code) include machine instructions for programmable processors, and these computing programs can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, device, and / or device (e.g., disk, optical disk, memory, programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0095] Certain aspects of the present invention include process steps and instructions described herein in the form of algorithms. It should be noted that the process steps and instructions of the present invention can be implemented in software, firmware and / or hardware, and when implemented by software, it can be downloaded, stored on different platforms used by various operating systems and operated from the platforms.

[0096] Those skilled in the art will understand that the structures shown in the accompanying drawings are merely block diagrams of partial structures related to the scheme of the present application, and do not constitute a limitation on the terminal device to which the scheme of the present application is applied. The specific terminal device may include more or fewer components than shown in the figures, or combine certain components, or have a different arrangement of components.

[0097] Some embodiments may be described using "coupled" and "connected" and their derivatives. It should be understood that these terms are not intended to be synonymous with each other. For example, some embodiments may be described using the word "connected" to indicate that two or more elements are in direct physical or electrical contact with each other. In another example, some embodiments may be described using the word "coupled" to indicate that two or more elements are in direct physical or electrical contact with each other. However, the term "coupled" may also mean that two or more elements are not in direct contact with each other, but still cooperate or interact with each other. The embodiments are not limited to this context.

[0098] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "possible design" etc. means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine the different embodiments or examples described in this specification and the features of the different embodiments or examples, without contradiction.

[0099] The above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit the same. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features thereof may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. An audio and video synchronization method, characterized in that: include: Get the audio and video frame rates of the playback content; Compare the audio acquisition frame rate and the video acquisition frame rate to see if they are the same, and if they are the same, perform a video frame playback adjustment process; The video frame playback adjustment process includes: The audio and video time difference between the audio frame and the video frame of the playing content at the current moment is calculated, and when the audio and video time difference exceeds a preset synchronization range, the playing frame rate of the video frame is adjusted to synchronize it with the audio frame.

2. The method according to claim 1, characterized in that When the audio and video time difference exceeds the preset synchronization range, adjusting the playback frame rate of the video frame to synchronize it with the audio frame includes: Calculate the video frame time difference between the current video frame and the previous video frame; The playback interval of the next video frame is calculated according to the video frame time difference, and the next video frame is refreshed and displayed according to the calculated playback interval.

3. The method according to claim 2, characterized in that The step of calculating the playback interval of the next video frame according to the video frame time difference comprises: Determine the relative position of the video frame and the audio frame at the current moment according to the audio and video time difference; When the current video frame is later than the audio frame, the playback interval of the next video frame is calculated as the video frame time difference * n, 0 <n<1; When the current video frame is earlier than the audio frame, the playback interval of the next video frame is calculated as the video frame time difference*m, where m>1.

4. The method according to claim 1, characterized in that The video frame playback adjustment process also includes: When the audio and video time difference does not exceed the preset synchronization range, the playback frame rate of the video frame is not adjusted.

5. The method according to claim 2, characterized in that: The video frame playback adjustment process also includes: When the audio and video time difference does not exceed the preset synchronization range, the playback interval of the next video frame is calculated according to the audio and video time difference and the video frame time difference.

6. The method according to claim 5, characterized in that The playback interval of the next video frame is the sum of the audio and video time difference and the video frame time difference.

7. The method according to claim 1, characterized in that The method further comprises: When the audio acquisition frame rate and the video acquisition frame rate are different, the playback frame rate of the video frame is adjusted to 1fps.

8. An audio and video synchronization device, characterized in that: include: The acquisition frame rate acquisition module is used to acquire the audio acquisition frame rate and video acquisition frame rate of the playback content; A frame rate difference calculation module, used to compare whether the audio acquisition frame rate and the video acquisition frame rate are the same; A playback frame rate adjustment module, used for performing a video frame playback adjustment process when the audio acquisition frame rate and the video acquisition frame rate are the same; The video frame playback adjustment process includes: The audio and video time difference between the audio frame and the video frame of the playing content at the current moment is calculated, and when the audio and video time difference exceeds a preset synchronization range, the playing frame rate of the video frame is adjusted to synchronize it with the audio frame.

9. An electronic device, characterized in that: The device comprises a memory storing computer executable instructions and a processor, and when the computer executable instructions are executed by the processor, the device executes the audio and video synchronization method according to any one of claims 1 to 7.

10. A readable storage medium, characterized in that: A computer executable program is stored, and when the program is executed, the audio and video synchronization method described in any one of claims 1 to 7 can be implemented.

Images