Method and device for synchronously processing sound and picture

Abstract

The present application is applicable to the technical field of intelligent audio-visual equipment, and provides a method and related device for synchronous processing of sound and picture applied to intelligent audio-visual equipment, aiming to solve the problem of asynchronism of picture and sound output by intelligent audio-visual equipment in the prior art. The method of the present application mainly includes: when receiving a picture delay instruction, increasing the cache number of picture frames of the picture played by the intelligent audio-visual equipment in the buffer according to the picture delay instruction; when receiving a picture advance instruction, reducing the cache number of picture frames of the picture played by the intelligent audio-visual equipment in the buffer according to the picture advance instruction.

Description

Technical Field

[0001] The present application belongs to the technical field of intelligent audio-visual equipment, and in particular, relates to a method and related device for synchronously processing sound and picture applied to intelligent audio-visual equipment. Background Art

[0002] The media resources played by smart TVs include audio and video resources that are on the same track in time series. In order to output the audio and video in the media resources, smart TVs need to first obtain the media resources, then parse the media resources through relevant protocols, decode audio and video, and render synchronously, and finally output the images through the smart TV screen and the audio through the smart TV speakers or external speakers. In the above process, starting from audio and video decoding, the video and audio are processed separately through different paths. Each link after the video and audio are processed separately may cause a certain time difference between the video and audio. Finally, when the images are output through the smart TV screen and the audio is output through the smart TV speakers or external speakers, the user may feel that the images and audio are not synchronized, affecting the viewing experience.

[0003] Furthermore, in the prior art, no solution has been found to provide users with a solution for synchronizing sound and picture on a smart TV. Summary of the invention

[0004] The purpose of the present application is to provide a method and related device for synchronous processing of sound and picture, aiming to solve the problem in the prior art that the smart audio and video equipment is not synchronized when finally outputting the picture and sound.

[0005] In the first aspect, a method for synchronously processing images and sounds is provided, which is applied to intelligent audio-visual equipment, comprising:

[0006] When receiving a picture delay instruction, increasing the number of cached picture frames of the picture played by the smart audio-visual device in the buffer according to the picture delay instruction;

[0007] When a picture advance instruction is received, the picture advance instruction is used in a buffer to reduce the number of cached picture frames of the playback picture of the smart audio-visual device.

[0008] Optionally, the method further includes:

[0009] receiving the synchronization processing instruction, wherein the synchronization processing instruction is used to start adjusting the output coordination of the picture and the sound on the intelligent audio-visual device;

[0010] The buffer in the locked state is activated according to the synchronization processing instruction to adjust the cache quantity of the picture frames of the picture played by the smart audio-visual device.

[0011] Optionally, the method further includes:

[0012] Receive a synchronization processing completion instruction;

[0013] The number of cached frames of the picture played by the smart audio-visual device in the current cache is locked according to the synchronization completion instruction.

[0014] Optionally, after activating the adjustment function of the buffer in the locked state for the cache quantity of the picture frames of the picture played by the smart audio-visual device according to the synchronization processing instruction, the method further includes:

[0015] Receiving, through a microphone, current sound emitted by the smart audio-visual device when playing a video;

[0016] Determining whether the current sound is aligned in time with a standard sound corresponding to a picture when the smart audio-visual device plays a video;

[0017] When the current sound is delayed compared to the standard sound corresponding to the picture when the smart audio-visual device plays the video, the picture delay instruction is generated;

[0018] When the current sound is aligned in time with the standard sound corresponding to the screen when the smart audio-visual device plays the video, the synchronization processing completion instruction is generated;

[0019] When the current sound is ahead of the standard sound corresponding to the picture when the smart audio-visual device plays the video, the picture advance instruction is generated.

[0020] Optionally, receiving, through a microphone, a current sound emitted by the smart audio and video device when playing a video includes:

[0021] receiving, through the microphone of the smart audio-visual device, the current sound emitted by the speaker of the smart audio-visual device when the smart audio-visual device plays a video; or,

[0022] The current sound emitted by the wirelessly connected speaker when the smart audio-visual device plays a video is received through the microphone of the smart audio-visual device.

[0023] Optionally, the smart audio and video device is one of a smart TV, a smart projector, a smart phone, a smart tablet, a laptop computer and a desktop computer.

[0024] Optionally, the receiving the synchronization processing instruction includes:

[0025] The synchronization processing instruction is generated by triggering a first physical button of the smart audio-visual device; or,

[0026] The synchronization processing instruction is generated by triggering the first virtual button of the intelligent audio-visual device;

[0027] The receiving synchronization processing completion instruction comprises:

[0028] The synchronization processing completion instruction is generated by triggering a second physical button of the smart audio-visual device; or,

[0029] The synchronization processing completion instruction is generated by triggering the second virtual button of the smart audio and video device.

[0030] In a second aspect, the present application provides a system for synchronously processing images and sounds, which is applied to intelligent audio-visual equipment, including:

[0031] an increasing unit, configured to increase the number of cached frames of the picture played by the smart audio-visual device in the buffer according to the picture delay instruction when receiving the picture delay instruction;

[0032] The reducing unit is used to reduce the cache quantity of the picture frames of the playback picture of the intelligent audio-visual device in the buffer according to the picture advance instruction when receiving the picture advance instruction.

[0033] Optionally, the system further includes:

[0034] A receiving unit, configured to receive the synchronization processing instruction, wherein the synchronization processing instruction is used to start adjusting the output coordination of the picture and the sound on the intelligent audio-visual device;

[0035] The activation unit is used to activate the buffer in the locked state to adjust the cache quantity of the picture frames of the picture played by the smart audio-visual device according to the synchronization processing instruction.

[0036] Optionally, the system further includes:

[0037] The receiving unit is further used to receive a synchronization processing completion instruction;

[0038] The locking unit is used to lock the cache quantity of the picture frames of the picture played by the smart audio-visual device in the current buffer according to the synchronization processing completion instruction.

[0039] Optionally, the system further includes:

[0040] The receiving unit is further used to receive the current sound emitted by the smart audio-visual device when playing the video through the microphone;

[0041] A determination unit, configured to determine whether the current sound is aligned in time with a standard sound corresponding to a picture when the smart audio-visual device plays a video;

[0042] A generating unit, configured to generate the picture delay instruction when the current sound is delayed compared to the standard sound corresponding to the picture when the smart audio-visual device plays the video;

[0043] The generating unit is further configured to generate the synchronization processing completion instruction when the current sound is aligned in time with the standard sound corresponding to the screen when the smart audio-visual device plays the video;

[0044] The generating unit is further configured to generate the picture advance instruction when the current sound is ahead of the standard sound corresponding to the picture when the smart audio-visual device plays the video.

[0045] Optionally, when the receiving unit receives the current sound emitted by the smart audio-visual device when playing a video through a microphone, it is specifically used to:

[0046] receiving, through the microphone of the smart audio-visual device, the current sound emitted by the speaker of the smart audio-visual device when the smart audio-visual device plays a video; or,

[0047] The current sound emitted by the wirelessly connected speaker when the smart audio-visual device plays a video is received through the microphone of the smart audio-visual device.

[0048] Optionally, the smart audio and video device is one of a smart TV, a smart projector, a smart phone, a smart tablet, a laptop computer and a desktop computer.

[0049] Optionally, when the receiving unit receives the synchronization processing instruction, it is specifically used to:

[0050] The synchronization processing instruction is generated by triggering a first physical button of the smart audio-visual device; or,

[0051] The synchronization processing instruction is generated by triggering the first virtual button of the intelligent audio-visual device;

[0052] When the receiving unit receives the synchronization processing completion instruction, it is specifically used to:

[0053] The synchronization processing completion instruction is generated by triggering a second physical button of the smart audio-visual device; or,

[0054] The synchronization processing completion instruction is generated by triggering the second virtual button of the smart audio and video device.

[0055] In a third aspect, the present application provides a computer device, including:

[0056] Processor, memory, bus, input and output interface, network interface;

[0057] The processor is connected to the memory, the input / output interface, and the network interface via the bus;

[0058] The memory stores a program;

[0059] When the processor executes the program stored in the memory, the method for synchronous processing of picture and sound as described in any one of the first aspects mentioned above is implemented.

[0060] In a fourth aspect, the present application provides a computer storage medium, wherein instructions are stored in the computer storage medium, and when the instructions are executed on a computer, the computer executes the method for synchronous processing of picture and sound as described in any one of the first aspects above.

[0061] In a fifth aspect, the present application provides a computer program product, which, when executed on a computer, enables the computer to execute the method for synchronous processing of picture and sound as described in any one of the first aspects above.

[0062] It can be seen from the above technical solutions that the embodiments of the present application have the following advantages:

[0063] The method for synchronous processing of picture and sound of the present application is applied to intelligent audio-visual equipment, which is provided with a buffer for caching a certain number of picture frames of the playback picture. When receiving a picture delay instruction, the cached number of cached picture frames of the picture played by the intelligent audio-visual equipment is increased in the buffer according to the picture delay instruction, so as to realize the further delay adjustment of the playback picture of the intelligent audio-visual equipment under the premise that the playback sound remains unchanged; when receiving a picture advance instruction, the cached number of cached picture frames of the playback picture of the intelligent audio-visual equipment is reduced in the buffer according to the picture advance instruction, so as to realize the reduction and delay adjustment of the playback picture of the intelligent audio-visual equipment under the premise that the playback sound remains unchanged; because in the prior art, the sound output of the intelligent audio-visual equipment when playing the media resource content is usually later than the picture output, the present implementation can realize the control of different delay time lengths of the picture by controlling the cached number of picture frames of the playback picture in the buffer, and adapt the sound in the intelligent audio-visual equipment that is delayed than the playback picture by controlling the different delay time lengths of the playback picture, so as to realize the synchronization of the picture and sound output by the intelligent audio-visual equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] Figure 1 A schematic diagram of a flow chart of an embodiment of a method for synchronously processing images and sounds of the present application;

[0065] Figure 2 This is a flow chart of another embodiment of the method for synchronously processing the picture and the sound of the present application;

[0066] Figure 3 This is a schematic diagram of the structure of an embodiment of the system for synchronously processing images and sounds of the present application;

[0067] Figure 4 This is a schematic diagram of the structure of an embodiment of the system for synchronously processing images and sounds of the present application;

[0068] Figure 5 A schematic diagram of the structure of an embodiment of a computer device of the present application;

[0069] Figure 6 This is a schematic diagram of the structural connection of an embodiment of the smart audio-visual device and the speaker of the present application;

[0070] Figure 7 The present invention is a schematic diagram of data flow direction of an embodiment of a smart audio-visual device playing media resources in the prior art. DETAILED DESCRIPTION

[0071] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0072] It should be noted that the smart audio-visual device of this embodiment refers to a device used to play media resources, and the media resources referred to in this embodiment refer to electronic resources with time-synchronized sound and images, such as TV series, movies and other videos. The smart audio-visual device that can play media resources can be one of a smart TV, a smart projector, a smart phone, a smart tablet, a laptop computer, a desktop computer, etc.

[0073] See also Figure 7, taking the smart audio-visual device as a smart TV as an example, when the smart TV plays the above media resources, in order to realize the separate output of the sound and the picture in the media resources, the smart TV needs to first obtain the media resource 800, and then perform the relevant protocol parsing, audio and video decoding, rendering and other operations on the media resource 800 to obtain: a picture frame sequence set 810 that has a chronological order, and a sound frame sequence combination 820 that has a chronological order; finally, the picture frame 810 in the picture frame sequence set 810 is displayed on the screen of the smart TV in the chronological order. 11, and play the sound frame 821 in the sound frame sequence combination 820 in chronological order through the speaker of the smart TV; when the smart TV is paired and connected with a speaker that also has a Bluetooth function through the Bluetooth function, the speaker can output the sound frame 821, specifically, the smart TV converts the sound frame 821 to be played by the sound frame sequence combination 820 into a sound frame 821" that can be transmitted through Bluetooth and transmits it to the speaker through Bluetooth. After the speaker receives the sound frame 821", it performs corresponding processing to obtain the sound frame 821 to be played, and the speaker then plays the sound frame 821. In the above process, starting from audio and video decoding, the video picture and video sound are processed separately through different paths. Each link after the video picture and video sound are processed separately may cause a certain time difference between the video picture and the video sound. Finally, when the video picture is output through the screen of the smart TV and the video sound is output through the speaker of the smart TV or an external speaker, the user may feel that the picture and sound are not synchronized, affecting the viewing experience. Especially when the smart TV is connected to an external speaker with Bluetooth function via Bluetooth wireless for playback, the smart TV needs to compress the sound signal and send it to the speaker via Bluetooth. The speaker then decompresses the compressed sound and plays it. This takes more time than playing the sound through the smart TV's own speakers. Users may feel that the picture played on the smart TV is faster than the sound heard from the speaker, affecting the user's viewing experience.

[0074] In order to solve the problem of asynchronism between the final output image and sound of smart audio-visual devices in the prior art, in particular, when the smart audio-visual device plays the sound of the media resource through an external speaker, the sound played by the speaker and the image displayed by the smart audio-visual device are not synchronized in time sequence. Figure 6, the intelligent audio-visual device of this embodiment adds a buffer to the traditional intelligent audio-visual device to solve the above technical problems. The method for realizing the delay of the final output picture of the intelligent audio-visual device can be completed by using a buffer to realize the frame buffer (FrameBuffer) of the picture frame. The picture frame sequence set 810 is composed of static pictures (picture frames 811) one by one, and each frame of the picture needs to pass through the frame buffer before display. The display module displays the pictures in the buffer frame by frame, and the dynamic picture can be seen. When the user wants the display module to realize the picture delay, the effect of adjusting the display picture delay can be achieved by adjusting the cache number of the picture frames in the buffer. Specifically, the intelligent audio-visual device 600 mainly includes: a processor 601, a speaker 602, a buffer 603, a display module 604, a communication module 605, a button module 606 and a power module (not shown in the figure). The processor 601 is mainly used to perform relevant protocol analysis, audio and video decoding, rendering and other operations on the media resource 800, thereby obtaining a picture frame sequence set 810 that is sequentially ordered based on a time sequence, and a sound frame sequence combination 820 that is sequentially ordered based on a time sequence; the processor 601 then sends the picture frame 811 in the picture frame sequence set 810 to the buffer 603 according to the time sequence sequence; and when the communication module 605 determines that there is no external speaker, the processor 601 also transmits the sound frame 821 in the sound frame sequence combination 820 to the speaker 602 according to the time sequence sequence; and when the communication module 605 determines that there is an external speaker 700, the processor 601 also transmits the sound frame 821 in the sound frame sequence combination 820 to the external speaker 700 through the communication module 605 according to the time sequence sequence. The speaker 602 is mainly used to play the sound frame 821 transmitted by the processor 601 to achieve sound output. The buffer 603 is mainly used to cache a number of picture frames according to the control logic adjusted by the processor 601 and then transmit them to the display module 604 in a first-in-first-out order. The display module 604 is mainly used to display the picture frames transmitted by the buffer 603. The communication module 605 is mainly used to determine whether to establish a connection with an external speaker. If a connection is successfully established with the speaker 700 (for example, a Bluetooth connection), it will feedback to the processor 601 that there is an external speaker 700, so that the processor 601 abandons the use of its own speaker to play the sound frame 821, and gives priority to the external speaker 700 to play the sound frame 821; if a connection is not established with the speaker 700 (for example, a Bluetooth connection), it will feedback to the processor 601 that there is no external speaker 700, so that the processor 601 uses its own speaker to play the sound frame 821. The button module 606 is mainly used to receive the user's operation instructions for the audio-visual device 600, and transmit the relevant operation instructions to the processor 601 for execution.The power module is mainly used to provide a suitable working voltage to the processor 601, the speaker 602, the buffer 603, the display module 604, the communication module 605, the key module 606 and the like.

[0075] Research shows that the above-mentioned intelligent audio-visual devices have the problem of being out of sync when finally outputting pictures and sounds, especially when the intelligent audio-visual devices play the sound of media resources through external speakers, the sound played by the speakers and the pictures displayed by the intelligent audio-visual devices are out of sync in time series, which is mainly manifested as: the pictures played by the intelligent audio-visual devices are faster than the sound. In the face of the technical problem that "the pictures played by the intelligent audio-visual devices are faster than the sound", there are usually two solutions: the first is to adjust the picture playback to a later time under the premise of keeping the sound playing continuously unchanged in the time series order, so that the user feels that the output of the picture and the sound is synchronized; the second is to adjust the sound playback in advance under the premise of keeping the picture playing continuously unchanged in the time series order, so that the user also feels that the output of the picture and the sound is synchronized. However, in reality, the effect of the second solution is relatively limited, especially when the intelligent audio-visual device plays the sound of media resources through an external speaker, because the intelligent audio-visual device takes a long time to send the sound to the external speaker for playback, and the time available for adjusting the sound playback in advance is not large, which is usually not enough to make up for the time difference between the asynchronous picture and the sound. Therefore, this embodiment mainly adopts the above-mentioned first solution.

[0076] Therefore, based on the above understanding of the intelligent audio-visual device 600 of the present application, please refer to Figure 1 The process of solving the above technical problem is shown in the following embodiment process. An embodiment of the method for synchronously processing the picture and sound of the intelligent audio-visual device applied in the present application includes:

[0077] 101. When a picture delay instruction is received, the number of cached picture frames for playing pictures of the smart audio-visual device is increased in the buffer according to the picture delay instruction.

[0078] When the user feels that the picture is faster than the sound when watching the video played by the smart audio-visual device, this step can receive the picture delay instruction triggered by the user through the button module 606, and then increase the cached number of picture frames of the playback picture of the smart audio-visual device video in the buffer 603 according to the picture delay instruction, so as to achieve the re-delay adjustment of the playback picture of the smart audio-visual device under the premise that the playback sound remains unchanged. For example, each time the user triggers the picture delay instruction (receiving the picture delay instruction issued by the user from the physical button through the button module 606), this step stores multiple X frames of picture frames in the buffer 603, where X is a positive integer greater than or equal to 1. Normally, the picture played by the smart audio-visual device is 60Hz, that is, 60 frames per second. Storing 1 picture frame in the buffer 603 can bring a picture delay of 1 / 60 seconds, and 6 frame caches can bring a delay of 1 / 10 seconds (100ms). Based on the 150ms delay of the external speaker connected via Bluetooth, approximately 9 frames of images can be stored in the buffer 603 to achieve synchronization between the output image of the smart audio-visual device and the output sound of the speaker. 102. When receiving a picture advance instruction, the number of picture frames of the playback picture of the smart audio-visual device is reduced in the buffer according to the picture advance instruction.

[0079] When the user stores too many picture frames in the buffer 603 during the operation of step 101, the user may feel that the picture is slower than the sound when watching the video played by the smart audio-visual device. In this step, the picture advance instruction triggered by the user can be received through the key module 606, and then the number of picture frames cached in the buffer 603 of the picture played by the smart audio-visual device can be reduced according to the picture advance instruction, so as to achieve the relative advance adjustment of the picture played by the smart audio-visual device under the premise of unchanged sound. For example, each time the picture advance instruction is triggered by the user (the picture advance instruction issued by the user from the physical button is received through the key module 606), this step releases a number of X frames of picture frames from the buffer 603, where X is a positive integer greater than or equal to 1, until there is no picture frame cached in the buffer 603, and the limit of the advance picture adjustment is reached. Generally, the picture played by the smart audio-visual device is 60Hz, that is, 60 frames per second. The buffer 603 releases 1 picture frame to bring 1 / 60 second of picture advance, and 6 frame caches can bring 1 / 10 second (100ms) of advance.

[0080] It can be seen that since in the prior art the sound output of smart audio and video devices when playing media resource content is usually delayed relative to the picture output, the present implementation can achieve control of different delay durations of the picture by controlling the number of cached picture frames of the playback picture in the buffer, and adapt to the sound in the smart audio and video devices that is delayed compared to the playback picture by controlling the different delay durations of the playback picture, thereby achieving synchronization of the picture and sound output by the smart audio and video devices.

[0081] See also Figure 2 Another embodiment of the method for synchronously processing the picture and sound of the intelligent audio-visual device of the present application includes:

[0082] 201. Receive a synchronization processing instruction, where the synchronization processing instruction is used to start adjusting the output coordination of the picture and the sound on the intelligent audio-visual device.

[0083] It is understandable that since the synchronization calibration of the picture and sound of the smart audio and video equipment is a matter that has certain technical requirements and will directly affect the user experience, for safety reasons, this step needs to clearly receive the user's synchronization processing instruction to the smart audio and video device before the relevant program will be activated. The synchronization processing instruction in this step refers to the output coordination of the smart audio and video device used to start adjusting the picture and sound, so as to achieve the user-level feeling of picture and sound synchronization. For example, the process of receiving the synchronization processing instruction triggered by the user in this step can be: the first physical button of the smart audio and video device is triggered to generate the synchronization processing instruction; or, the first virtual button of the smart audio and video device (the virtual button on the touch screen) is triggered to generate the synchronization processing instruction. There is no limitation on the form of receiving the synchronization processing instruction here.

[0084] 202. Activate a function of adjusting the cache quantity of picture frames of the intelligent audio-visual device playing pictures by activating the cache in the locked state according to the synchronization processing instruction.

[0085] For security reasons, this step requires that when a synchronization processing instruction is clearly received from the user to the smart audio and video device in step 201, the buffer 603 in the locked state is activated according to the synchronization processing instruction to adjust the cache number of picture frames of the picture played by the smart audio and video device, thereby allowing subsequent steps to increase or decrease the cache number of picture frames stored in the buffer 603.

[0086] 203. Receive the current sound emitted by the smart audio and video device when playing the video through the microphone.

[0087] In some embodiments, the "automatic synchronization of picture and sound" function of the smart audio and video device can also be implemented. When the smart audio and video device has a microphone recording function, after activating the locked buffer in step 202 to adjust the cache quantity of the picture frames of the picture played by the smart audio and video device, this step can receive the current sound emitted by the smart audio and video device when playing the video through the microphone (emitted by the smart audio and video device's own speaker or the wirelessly connected speaker).

[0088] 204. Determine whether the current sound is aligned in time with the standard sound corresponding to the picture when the smart audio and video device plays the video. If the current sound is delayed than the standard sound corresponding to the picture when the smart audio and video device plays the video, execute step 205; if the current sound is earlier than the standard sound corresponding to the picture when the smart audio and video device plays the video, execute step 207; if the current sound is aligned in time with the standard sound corresponding to the picture when the smart audio and video device plays the video, execute step 209.

[0089] It is worth noting that when the smart audio-visual device decodes the audio and video of the media resource 800, it can know the correspondence between each frame of the picture and the corresponding standard sound in the time series order. Then, after knowing the current sound emitted by the smart audio-visual device when playing the video (played by the smart audio-visual device's own speakers or external speakers) in step 203, this step can determine whether the current sound is aligned in time with the standard sound corresponding to the picture when the smart audio-visual device plays the video. If the current sound is delayed than the standard sound corresponding to the picture when the smart audio-visual device plays the video, the time difference between the current sound and the standard sound corresponding to the picture can be calculated, and then the number of picture frames required to store the time difference in the buffer can be further calculated; if the current sound is ahead of the standard sound corresponding to the picture when the smart audio-visual device plays the video, the time difference between the current sound and the standard sound corresponding to the picture can be calculated, and then the number of picture frames required to be released from the buffer for the time difference can be further calculated; if the current sound is aligned in time with the standard sound corresponding to the picture when the smart audio-visual device plays the video, it proves that the picture and the current sound are synchronized when the smart audio-visual device plays the video.

[0090] 205. Generate a screen delay instruction.

[0091] When it is determined in step 204 that the current sound is delayed compared to the standard sound corresponding to the picture when the smart audio and video device plays the video, this step actively generates or calls a preset picture delay instruction, which is used to increase the cache quantity of picture frames in the buffer.

[0092] 206. Increase the number of cached frames of the picture played by the intelligent audio-visual device in the buffer according to the picture delay instruction.

[0093] This step is based on the picture delay instruction of step 205, and usually carries the calculated number of picture frames that need to be stored in the buffer together with the picture delay instruction. Then this step increases the cache number of the corresponding number of picture frames for the picture played by the smart audio and video device in the buffer, and continues to trigger the execution of step 204.

[0094] 207. Generate screen advance instructions.

[0095] When it is determined in step 204 that the current sound is ahead of the standard sound corresponding to the picture when the smart audio-visual device plays the video, this step actively generates or calls a preset picture advance instruction, which is used to reduce the cache quantity of picture frames from the buffer.

[0096] 208. Reduce the number of cached frames of the playback image of the intelligent audio-visual device in the buffer according to the picture advance instruction.

[0097] This step is based on the picture advance instruction of step 207, and usually carries the calculated number of cached frames that need to be reduced from the buffer together with the picture advance instruction. Therefore, this step reduces the cached number of picture frames for playing pictures on the smart audio and video device by a corresponding number in the buffer, and continues to trigger the execution of step 204.

[0098] 209. Generate a synchronization processing completion instruction.

[0099] After it is determined in step 204 that the current sound is aligned in time with the standard sound corresponding to the picture when the smart audio and video device plays the video, it is proved that the picture and the current sound are synchronized when the smart audio and video device plays the video. This step actively generates or calls a preset synchronization processing completion instruction, which is used to lock the cache number of picture frames in the cache that cache the picture played by the smart audio and video device.

[0100] It should be noted that, in some embodiments, the synchronization processing completion instruction of this step can also be: triggered and generated by a second physical button of the smart audio and video device; or, triggered and generated by a second virtual button (virtual button on the touch screen) of the smart audio and video device.

[0101] 210. Lock the cache quantity of the picture frames of the picture played by the intelligent audio-visual device in the current buffer according to the synchronization processing completion instruction.

[0102] This step completes the instruction to lock the cache number of picture frames of the smart audio-visual device playing pictures in the cache according to the synchronization processing of step 209, and completes the "automatic synchronization of picture and sound" of the smart audio-visual device.

[0103] The above embodiment describes the method for synchronously processing the picture and sound of the intelligent audio-visual device applied by the present application. The following describes the system for synchronously processing the picture and sound of the intelligent audio-visual device applied by the present application. Figure 3 , the present application is applied to an embodiment of a system for synchronously processing images and sounds of intelligent audio-visual equipment, including:

[0104] The adding unit 301 is used to increase the cache quantity of the picture frames of the picture played by the smart audio-visual device in the buffer according to the picture delay instruction when receiving the picture delay instruction;

[0105] The reducing unit 302 is used to reduce the number of cached frames of the playback picture of the smart audio-visual device in the buffer according to the picture advance instruction when receiving the picture advance instruction.

[0106] The operation performed by the synchronous processing system of the picture and sound applied to the intelligent audio-visual device in this embodiment is similar to the above-mentioned Figure 1 The operations performed in the embodiment are similar and will not be described in detail here.

[0107] The picture and sound synchronization processing system of the present application is applied to intelligent audio-visual equipment, which is provided with a buffer for caching a certain number of picture frames of the playback picture. When receiving a picture delay instruction, the cached number of cached picture frames of the picture played by the intelligent audio-visual equipment is increased in the buffer according to the picture delay instruction, so as to achieve further delay adjustment of the playback picture of the intelligent audio-visual equipment under the premise that the playback sound remains unchanged; when receiving a picture advance instruction, the cached number of cached picture frames of the playback picture of the intelligent audio-visual equipment is reduced in the buffer according to the picture advance instruction, so as to achieve reduction and delay adjustment of the playback picture of the intelligent audio-visual equipment under the premise that the playback sound remains unchanged; because in the prior art, the sound output of the intelligent audio-visual equipment when playing media resource content is usually later than the picture output, this implementation can achieve control of different delay time lengths of the picture by controlling the cached number of picture frames of the playback picture in the buffer, and adapt to the sound in the intelligent audio-visual equipment that is delayed than the playback picture by controlling the different delay time lengths of the playback picture, so as to achieve synchronization of the picture and sound output by the intelligent audio-visual equipment.

[0108] See also Figure 4 Another embodiment of the system for synchronously processing images and sounds of intelligent audio-visual equipment of the present application includes:

[0109] The adding unit 401 is used to increase the cache quantity of the picture frames of the picture played by the smart audio-visual device in the buffer according to the picture delay instruction when receiving the picture delay instruction;

[0110] The reducing unit 402 is used for reducing the number of cached frames of the playback picture of the smart audio-visual device in the buffer according to the picture advance instruction when receiving the picture advance instruction.

[0111] Optionally, the system further includes:

[0112] A receiving unit 403 is used to receive the synchronization processing instruction, where the synchronization processing instruction is used to start adjusting the output coordination of the picture and the sound on the intelligent audio-visual device;

[0113] The activation unit 404 is used to activate the buffer in the locked state to adjust the cache quantity of the picture frames of the picture played by the smart audio-visual device according to the synchronization processing instruction.

[0114] Optionally, the system further includes:

[0115] The receiving unit 403 is further configured to receive a synchronization processing completion instruction;

[0116] The locking unit 405 is used to lock the cache quantity of the picture frames of the picture played by the smart audio-visual device in the current buffer according to the synchronization processing completion instruction.

[0117] Optionally, the system further includes:

[0118] The receiving unit 403 is further configured to receive, through a microphone, the current sound emitted by the smart audio-visual device when playing a video;

[0119] A determination unit 406 is used to determine whether the current sound is aligned in time with the standard sound corresponding to the screen when the smart audio-visual device plays the video;

[0120] The generating unit 407 is used to generate the picture delay instruction when the current sound is delayed compared with the standard sound corresponding to the picture when the smart audio-visual device plays the video;

[0121] The generating unit 407 is further configured to generate the synchronization processing completion instruction when the current sound is aligned in time with the standard sound corresponding to the screen when the smart audio-visual device plays the video;

[0122] The generating unit 407 is further configured to generate the picture advance instruction when the current sound is ahead of the standard sound corresponding to the picture when the smart audio-visual device plays the video.

[0123] Optionally, when the receiving unit 403 receives the current sound emitted by the smart audio-visual device when playing a video through a microphone, it is specifically used to:

[0124] receiving, through the microphone of the smart audio-visual device, the current sound emitted by the speaker of the smart audio-visual device when the smart audio-visual device plays a video; or,

[0125] The current sound emitted by the wirelessly connected speaker when the smart audio-visual device plays a video is received through the microphone of the smart audio-visual device.

[0126] Optionally, the smart audio and video device is one of a smart TV, a smart projector, a smart phone, a smart tablet, a laptop computer and a desktop computer.

[0127] Optionally, when the receiving unit 403 receives the synchronization processing instruction, it is specifically used to:

[0128] The synchronization processing instruction is generated by triggering a first physical button of the smart audio-visual device; or,

[0129] The synchronization processing instruction is generated by triggering the first virtual button of the intelligent audio-visual device;

[0130] When the receiving unit 403 receives the synchronization processing completion instruction, it is specifically used to:

[0131] The synchronization processing completion instruction is generated by triggering a second physical button of the smart audio-visual device; or,

[0132] The synchronization processing completion instruction is generated by triggering the second virtual button of the smart audio and video device.

[0133] The operation performed by the synchronous processing system of the picture and sound applied to the intelligent audio-visual device in this embodiment is similar to the above-mentioned Figure 1 or Figure 2 The operations performed in the embodiment are similar and will not be described in detail here.

[0134] The computer device in the embodiment of the present application is described below. Figure 5 , an embodiment of the computer device in the embodiment of the present application includes:

[0135] The computer device 500 may include one or more processors (central processing units, CPU) 501 and a memory 502, in which one or more application programs or data are stored. The memory 502 is a volatile storage or a persistent storage. The program stored in the memory 502 may include one or more modules, and each module may include a series of instruction operations in the computer device. Furthermore, the processor 501 may be configured to communicate with the memory 502 to execute a series of instruction operations in the memory 502 on the computer device 500. The computer device 500 may also include one or more network interfaces 503, one or more input and output interfaces 504, and / or one or more operating systems, such as Windows Server, Mac OS, Unix, Linux, FreeBSD, etc. The processor 501 may execute the aforementioned Figure 1 or Figure 2 The operations performed in the illustrated embodiment will not be described in detail here.

[0136] In several embodiments provided in the embodiments of the present application, those skilled in the art should understand that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the unit is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.

[0137] In addition, each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of a software functional unit. If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, read-only memory), random access memory (RAM, random access memory), disk or optical disk and other media that can store program code.

[0138] The above description is only a preferred embodiment of the present application and is not intended to limit the present application. Any modifications, equivalent substitutions or improvements made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for synchronously processing images and sounds, characterized in that: Applied to intelligent audio and video equipment, including: Receive a synchronization processing instruction, the synchronization processing instruction is used to start adjusting the output coordination of the picture and the sound of the intelligent audio-visual device; Activating, according to the synchronization processing instruction, a function of adjusting the number of cached frames of the picture played by the intelligent audio-visual device by a cache in a locked state; When receiving a picture delay instruction, the number of cached picture frames of the picture played by the smart audio-visual device is increased in the buffer according to the picture delay instruction, and each time the picture delay instruction is received, X frames of picture frames are stored in the buffer, where X is a positive integer greater than or equal to 1; When receiving a picture advance instruction, the number of picture frames cached in the buffer for the playback picture of the smart audio-visual device is reduced according to the picture advance instruction, and each time the picture advance instruction is received, X frames of picture frames are released to the buffer; The method further comprises: Receive a synchronization processing completion instruction; According to the synchronization processing completion instruction, the number of cached frames of the picture played by the smart audio-visual device is locked in the current cache; The receiving the synchronization processing instruction comprises: The synchronization processing instruction is generated by triggering a first physical button of the smart audio-visual device; The receiving synchronization processing completion instruction comprises: The synchronization processing completion instruction is generated by triggering the second physical button of the smart audio and video device.

2. The synchronization processing method according to claim 1, characterized in that: After activating the adjustment function of the buffer in the locked state for the cache quantity of the picture frames of the picture played by the smart audio-visual device according to the synchronization processing instruction, the method further includes: Receiving, through a microphone, current sound emitted by the smart audio-visual device when playing a video; Determining whether the current sound is aligned in time with a standard sound corresponding to a picture when the smart audio-visual device plays a video; When the current sound is delayed compared to the standard sound corresponding to the picture when the smart audio-visual device plays the video, the picture delay instruction is generated; When the current sound is aligned in time with the standard sound corresponding to the screen when the smart audio-visual device plays the video, the synchronization processing completion instruction is generated; When the current sound is ahead of the standard sound corresponding to the picture when the smart audio-visual device plays the video, the picture advance instruction is generated.

3. The synchronization processing method according to claim 2, characterized in that: The receiving, through the microphone, the current sound emitted by the smart audio-visual device when playing the video includes: receiving, through the microphone of the smart audio-visual device, the current sound emitted by the speaker of the smart audio-visual device when the smart audio-visual device plays a video; or, The current sound emitted by the wirelessly connected speaker when the smart audio-visual device plays a video is received through the microphone of the smart audio-visual device.

4. The synchronous processing method according to claim 2, characterized in that: The smart audio-visual device is one of a smart TV, a smart projector, a smart phone, a smart tablet, a laptop computer and a desktop computer.

5. A synchronous processing system for images and sounds, characterized in that: Applied to intelligent audio and video equipment, including: A receiving unit, configured to receive a synchronization processing instruction, wherein the synchronization processing instruction is used to start adjusting the output coordination of the picture and the sound in the intelligent audio-visual device; An activation unit, configured to activate, according to the synchronization processing instruction, a function of adjusting the cache quantity of the picture frames of the picture played by the intelligent audio-visual device by the buffer in the locked state; an adding unit, configured to increase the number of cached frames of the picture played by the smart audio-visual device in the buffer according to the picture delay instruction when receiving the picture delay instruction, and store X frames of the picture into the buffer each time the picture delay instruction is received, where X is a positive integer greater than or equal to 1; a reducing unit, configured to reduce the number of cached frames of the playback screen of the intelligent audio-visual device in the buffer according to the picture advance instruction when receiving the picture advance instruction, and release X frames of picture frames to the buffer each time the picture advance instruction is received; The system further comprises: The receiving unit is further used to receive a synchronization processing completion instruction; A locking unit, configured to lock the number of frames of the picture currently cached by the buffer for playing the picture of the smart audio-visual device according to the synchronization processing completion instruction; When the receiving unit receives the synchronization processing instruction, it is specifically used to: The synchronization processing instruction is generated by triggering a first physical button of the smart audio-visual device; When the receiving unit receives the synchronization processing completion instruction, it is specifically used to: The synchronization processing completion instruction is generated by triggering the second physical button of the smart audio and video device.

6. A computer device, characterized in that: include: Processor, memory, bus, input and output interface, network interface; The processor is connected to the memory, the input / output interface, and the network interface via the bus; The memory stores a program; When the processor executes the program stored in the memory, the method for synchronous processing of picture and sound as claimed in any one of claims 1 to 4 is implemented.

7. A computer storage medium, characterized in that: The computer storage medium stores instructions, and when the instructions are executed on a computer, the computer executes the method for synchronously processing pictures and sounds as claimed in any one of claims 1 to 4.

Images