Audio and video synchronization method based on airborne entertainment system

Abstract

The invention relates to an audio and video synchronization method based on an airborne entertainment system, and the method comprises the steps: a media server carries out the constant code stream processing of a to-be-played film source, carries out the audio and video separation, and obtains a first video stream, a second video stream, and an uncompressed audio stream; the media server pushes the first video stream to a hoisting display unit and pushes the second video stream to a handrail display unit, and pushes an uncompressed audio stream to a passenger cabin broadcasting unit and a passenger control unit after time T1; after receiving the first video stream, the hoisting display unit starts playing after waiting for time T2; after receiving the second video stream, the armrest display unit directly plays the second video stream; after receiving the uncompressed audio stream, the passenger cabin broadcasting unit directly plays the uncompressed audio stream; and after receiving the uncompressed audio stream, the passenger control unit directly plays the uncompressed audio stream. According to the invention, the audio and video streams pushed by the media server can be synchronized on different devices.

Description

Technical Field

[0001] This invention relates to the field of airborne streaming media technology, and in particular to a method for synchronizing audio and video based on an airborne entertainment system. Background Technology

[0002] With the development of communication technology, network streaming media technology is increasingly used in aircraft, especially in in-flight entertainment systems. Streaming media refers to media formats, such as audio, video, or multimedia files, that are continuously played in real-time over a network using streaming transmission technology. In-flight entertainment systems mainly include media servers, pylon displays, armrest displays, passenger control units, and passenger announcements. Pylon and armrest displays play video, while passenger control units and passenger announcements play audio. Due to the special nature of in-flight entertainment system equipment, audio and video streams need to be pushed separately to achieve audio-video synchronization on different devices. This necessitates a complete audio-video synchronization method to achieve the synchronized audio-video effect of the in-flight equipment. Summary of the Invention

[0003] The technical problem to be solved by the present invention is to provide an audio and video synchronization method based on an in-flight entertainment system, which can synchronize the audio and video streams pushed by the media server on different devices.

[0004] The technical solution adopted by this invention to solve its technical problem is: to provide an audio and video synchronization method based on an in-flight entertainment system, comprising the following steps:

[0005] The media server performs constant bitrate processing on the video source to be played, and then performs audio-video separation on the video source after constant bitrate processing to obtain a first video stream, a second video stream, and an uncompressed audio stream; wherein, the second video stream contains a compressed audio stream;

[0006] The media server pushes the first video stream to the hoisting display unit and the second video stream to the handrail display unit, and pushes an uncompressed audio stream to the cabin broadcast unit and the passenger control unit after time T1; wherein, time T1 is the difference between the video decoding and display time and the audio playback time;

[0007] After receiving the first video stream, the hoisting display unit starts playing after a waiting time T2; during the waiting time T2, the received video frames are sent to the queue buffer, and the first frame in the queue buffer is taken as the latest video frame; the time T2 is the difference between the display time of the hoisting display unit and the display time of the handrail display unit.

[0008] After receiving the second video stream, the armrest display unit directly plays the second video stream and decodes the compressed audio stream contained in the second video stream through the built-in audio decoding chip;

[0009] Upon receiving the uncompressed audio stream, the cabin broadcast unit directly plays the uncompressed audio stream.

[0010] After receiving the uncompressed audio stream, the passenger control unit plays the uncompressed audio stream directly.

[0011] Before the media server performs constant bitrate processing on the source material to be played, the following steps are also included: the media server synchronizing its power-on time with the avionics system, the hoisting display unit, the armrest display unit, and the passenger control unit synchronizing their time with the media server, and the cabin broadcast synchronizing its time with the avionics system.

[0012] The video data of the first and second video streams are encoded in H.264 format, with the frame rate constantly controlled at 30 frames per second, and are encapsulated into MPEG2-TS format bitstreams.

[0013] The audio data of the uncompressed audio stream is an uncompressed PCM audio file with a sampling rate of 48kHz.

[0014] When the media server pushes uncompressed audio streams to the cabin broadcast unit and passenger control unit, it pushes 384 bytes of audio data each time, and the push interval is 4ms.

[0015] When the hoisted display unit plays the first video stream, it further includes: determining whether the interval between the currently playing video frame and the latest video frame in the queue buffer exceeds 10ms; if it does, discarding the first video frame in the queue buffer.

[0016] When the passenger control unit plays the uncompressed audio stream, it further includes: determining whether the interval between the currently played audio frame and the latest received audio frame exceeds 35ms; if it exceeds 35ms, determining whether the time corresponding to the currently played audio frame is greater than the time corresponding to the latest received audio frame; if so, adding an empty audio frame before the latest received audio frame; if not, discarding the latest received audio frame.

[0017] Beneficial effects

[0018] Due to the adoption of the above technical solutions, this invention has the following advantages and positive effects compared with the prior art: This invention processes the video source using a constant bitstream method and separates audio and video into specific formats to achieve audio and video synchronization. Furthermore, this invention also adopts a time-locking method to achieve audio and video synchronization of each terminal based on the latest data frame. Through the method of this invention, the video frame error can be within 10ms and the audio frame error can be within 35ms, thereby ensuring audio and video synchronization of each terminal device. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a flowchart of an embodiment of the present invention based on an audio-video synchronization method for an in-flight entertainment system;

[0020] Figure 2 This is a system connection architecture diagram in an embodiment of the present invention;

[0021] Figure 3 This is a system data flow diagram in an embodiment of the present invention;

[0022] Figure 4 This is a timeline diagram of the system in an embodiment of the present invention. Detailed Implementation

[0023] The present invention will be further illustrated below with reference to specific embodiments. It should be understood that these embodiments are for illustrative purposes only and are not intended to limit the scope of the invention. Furthermore, it should be understood that after reading the teachings of this invention, those skilled in the art can make various alterations or modifications to the invention, and these equivalent forms also fall within the scope defined by the appended claims.

[0024] The embodiments of the present invention relate to an audio-video synchronization method based on an in-flight entertainment system, which can be applied to, for example... Figure 2 The system architecture shown adopts an Ethernet ring network structure, which saves cabling on the aircraft. The media server is connected to the first cabin broadcast unit via network port 1. The cabin broadcast unit is connected to the next cabin broadcast unit via Ethernet, and finally to the media server's network port 2. The media server is connected to the hoisting display unit via Ethernet port 3. The hoisting display unit is connected to the next hoisting display unit via Ethernet, and finally to the media server's Ethernet port 4. The media server's network port 5 is connected to the armrest display unit. The armrest display unit is connected to the next armrest display unit via Ethernet. The last armrest display unit is connected to the passenger control unit via Ethernet. The passenger control unit is connected to the next passenger control unit via Ethernet, and finally to the media server's network port 6.

[0025] In this embodiment, the media server uses a Linux operating system, the hoisted display unit uses a Linux operating system, the handrail display unit uses an Android operating system, the passenger control unit uses a Linux operating system, and the cabin broadcast unit uses a VxWorks real-time operating system.

[0026] like Figure 1 As shown, the audio-video synchronization method based on an in-flight entertainment system in this embodiment specifically includes the following steps:

[0027] S1: After the media server is powered on, it synchronizes its time with the avionics system; the hoisting display unit, armrest display unit, and passenger control unit synchronize their time with the media server unit; and the cabin broadcast unit synchronizes its time with the avionics system.

[0028] S2: The media server performs constant bitrate processing on the video source, ensuring that the bitrate of the video source remains constant. After processing, the data packets pushed each time can be kept at a constant value.

[0029] S3: The processed video source is separated into audio and video streams, resulting in a first video stream, a second video stream, and an uncompressed audio stream. The video is separated into TS format video, encoded in 720P H.264 at 30 frames per second; the audio is separated into 16-bit, 48kHz uncompressed PCM audio. The video data in both the first and second video streams is encoded in H.264, with a constant frame rate of 30 frames per second, and encapsulated into an MPEG2-TS format bitstream. The uncompressed audio stream consists of uncompressed PCM audio files at a 48kHz frame rate to ensure that 384 bytes of audio data are pushed at a time interval of 4ms. The second video stream also contains an AAC format audio stream.

[0030] S4: As Figure 3 As shown, the media server pushes the audio stream to the cabin broadcast unit, the first video stream to the hoisting display unit, the second video stream to the armrest display unit, and the audio stream to the passenger control unit. The second video stream pushed to the armrest display unit contains an AAC format audio stream, which can be decoded using the built-in audio decoding function of the armrest display unit.

[0031] S5: The media server sends a playback command via multicast. After receiving the command, the hoisting display unit, handrail display unit, passenger control unit, and cabin broadcast unit adjust the player to playback mode and wait for the media server to push the audio and video stream.

[0032] S6: The media server pushes audio streams via RTP multicast. The video stream address is 239.128.100.3, port number 60786; the audio stream address is 239.192.128.1, port number 60784; (e.g., ...) Figure 4As shown, the video stream is pushed first, and the audio stream is pushed after time T1. The video decoding and display speed is slower than the audio playback speed; therefore, time T1 is the difference between the video decoding and display time and the audio playback time. The playback speed of the hoisting display unit is T2 time faster than that of the armrest display unit. Therefore, the hoisting display unit needs to wait for time T2 before playing the video, while the armrest display unit plays the video immediately after receiving it. The cabin broadcast unit plays the audio immediately after receiving it, and the passenger control unit also plays the audio stream data immediately upon receiving it.

[0033] S7: After receiving the first video stream, the hoisted display unit waits for time T2 before starting playback. Time T2 is the difference between the display time of the hoisted display unit and the display time of the handrail display unit. During the waiting time T2, the hoisted display unit sends the received video frames into the queue buffer, and the queue buffer uses the first frame as the latest video frame. During playback, the hoisted display unit uses the latest video frame as a reference to determine whether the interval between the currently playing video frame and the latest video frame exceeds 10ms. If it exceeds 10ms, the first video frame in the queue buffer is discarded.

[0034] S8: After receiving the second video stream, the armrest display unit plays it directly. This second video stream also contains AAC format audio data, which the armrest display unit decodes locally using its built-in audio decoding chip. The armrest display unit's player has a time lock function, prioritizing the most recently received video frame to ensure synchronized video playback.

[0035] S9: The cabin broadcast unit uses the VxWorks real-time operating system. The audio stream is uncompressed PCM format audio data, which can be played directly. As it is a real-time system, playing the latest audio data can achieve a synchronization effect.

[0036] S10: The passenger control unit receives an uncompressed PCM audio stream. Since the passenger control unit is not a real-time operating system, audio playback will slow down over time. The passenger control unit uses a time lock to determine if the time interval between the latest received audio frame and the currently playing audio frame exceeds 35ms. If it does, it further checks if the time corresponding to the currently playing audio frame is greater than the time corresponding to the latest received audio frame. If so, it means the audio playback is too fast, so an empty audio frame is added before the latest received audio frame. If not, it means the audio playback is too slow, and the latest received audio frame needs to be discarded. This method ensures audio-video synchronization.

[0037] It is easy to see that this invention processes the video source using a constant bitstream method and separates audio and video into specific formats to achieve audio and video synchronization. Furthermore, this invention also uses a time-locking method to achieve audio and video synchronization of each terminal based on the latest data frame. Through the method of this invention, the video frame error can be kept within 10ms and the audio frame error within 35ms, thereby ensuring audio and video synchronization of each terminal device.

Claims

1. A method for synchronizing audio and video based on an in-flight entertainment system, characterized in that: The following steps are involved: The media server performs constant bitstream processing on the video source to be played, and performs audio and video separation on the video source to be played after the constant bitstream processing, to obtain a first video stream, a second video stream and an uncompressed audio stream; wherein the second video stream contains a compressed audio stream; The media server pushes the first video stream to the hanging display unit and the second video stream to the armrest display unit, and pushes the uncompressed audio stream to the cabin broadcast unit and the passenger control unit after time T1; wherein time T1 is the difference between the time of video decoding and display and the time of audio playback; After receiving the first video stream, the hanging display unit starts playing after waiting time T2; wherein, during the waiting time T2, the received video frame is sent to the queue buffer area, and the front frame in the queue buffer area is used as the latest video frame; the time T2 is the difference between the display time of the hanging display unit and the display time of the handrail display unit; After receiving the second video stream, the armrest display unit directly plays the second video stream and decodes the compressed audio stream contained in the second video stream through a built-in audio decoding chip; After receiving the uncompressed audio stream, the cabin broadcast unit directly plays the uncompressed audio stream; After receiving the uncompressed audio stream, the passenger control unit directly plays the uncompressed audio stream.

2. The method for synchronizing audio and video based on an in-flight entertainment system according to claim 1, characterized in that: Before the media server performs constant code stream processing on the video source to be played, the media server is powered on to synchronize time with the avionics system, the hanging display unit, the armrest display unit and the passenger control unit synchronize time with the media server, and the cabin broadcast synchronizes time with the avionics system.

3. The method for synchronizing audio and video based on an in-flight entertainment system according to claim 1, characterized in that: The video data of the first video stream and the second video stream adopt the H.264 encoding format, the frame rate is constantly controlled at 30 frames per second, and are encapsulated into a code stream in MPEG2-TS format.

4. The method for synchronizing audio and video based on an in-flight entertainment system according to claim 1, characterized in that: The audio data of the uncompressed audio stream is a PCM uncompressed audio file, and the sampling rate is 48KHz.

5. The method for synchronizing audio and video based on an in-flight entertainment system according to claim 1, characterized in that: When the media server pushes the uncompressed audio stream to the cabin broadcast unit and the passenger control unit, 384 bytes of audio data are pushed each time and the push time interval is 4ms.

6. The method for synchronizing audio and video based on an in-flight entertainment system according to claim 1, characterized in that: When playing the first video stream, the hanging display unit also includes: determining whether the interval between the currently played video frame and the latest video frame in the queue buffer area exceeds 10ms, and if so, discarding the frontmost video frame in the queue buffer area.

7. The method for synchronizing audio and video based on an in-flight entertainment system according to claim 1, characterized in that: When the passenger control unit plays the uncompressed audio stream, it also includes: determining whether the interval between the currently played audio frame and the latest received audio frame exceeds 35ms; if so, determining whether the time corresponding to the currently played audio frame is greater than the time corresponding to the latest received audio frame, and if so, adding an empty audio frame before the latest received audio frame, and if not, discarding the latest received audio frame.

Images