In-vehicle viewing system and in-vehicle viewing method

Abstract

The present invention relates to an in-vehicle viewing system, an in-vehicle viewing method, and a computer storage medium. The in-vehicle viewing system according to one aspect of the present invention includes: a first audio-video data processing module, which is disposed on the vehicle head unit and includes a first clock unit for generating a first audio timestamp signal and a first video timestamp signal; and a second audio-video data processing module, which is disposed on the augmented reality device side or the virtual reality device side and includes a second clock unit for generating a second audio timestamp signal and a second video timestamp signal; wherein the first clock unit and the second clock unit are communicatively connected such that the first clock unit controls the second audio timestamp signal and the second video timestamp signal to be synchronized with the first audio timestamp signal and the first video timestamp signal, respectively.

Description

Technical Field

[0001] The present invention relates to the field of communication technologies, and more particularly to an in-vehicle viewing system, an in-vehicle viewing method, and a computer storage medium. Background Art

[0002] Currently, in in-vehicle entertainment systems, local audio-visual playback devices are usually used to provide viewing services for vehicle occupants. However, with the continuous development of autonomous driving and advanced driver assistance, a single local audio-visual playback device is increasingly unable to meet the entertainment needs of vehicle occupants.

[0003] In recent years, virtual reality and / or augmented reality technologies have been used to enhance the user experience in various fields such as gaming, healthcare, retail, education, social media, etc. Virtual reality technology is a computer simulation technology that can create a virtual reality environment, which can use a computer to generate a virtual reality environment. The basic implementation method of virtual reality technology is that a computer obtains data for creating a virtual reality environment (such as images, sounds, videos, etc.) and converts it into an electronic signal, and a virtual reality display device receives the electronic signal and constructs a virtual reality environment based on the electronic signal. Virtual reality and / or augmented reality technologies usually use a virtual reality display device to display images or videos to a user, and the displayed images or videos can include a completely digital view (virtual reality) or can include information overlaid on the user's field of view (augmented reality), thereby being able to enhance the authenticity and interest of the displayed images or videos.

[0004] Therefore, it is desirable that virtual reality and / or augmented reality technologies can also be applied to vehicles to meet the entertainment needs of vehicle occupants. In addition, in some driving situations, to ensure driving safety, it is desirable to switch the audio played by the in-vehicle audio to the speakers of an augmented reality device or a virtual reality device of other vehicle passengers, so as to ensure that the driver is not disturbed by the audio. In this process, how to ensure that the audio automatically remains synchronized with the video played on the augmented reality device or the virtual reality device after being switched from the in-vehicle audio to the speakers of the augmented reality device or the virtual reality device is also a problem that needs to be solved urgently at present. Summary of the Invention

[0005] To solve or at least alleviate one or more of the above problems, the following technical solutions are provided.

[0006] According to a first aspect of the present invention, there is provided an in-vehicle viewing system, the in-vehicle viewing system comprising: a first audio-video data processing module, which is arranged on the vehicle head unit and includes a first clock unit for generating a first audio timestamp signal and a first video timestamp signal; and a second audio-video data processing module, which is arranged on an augmented reality device side or a virtual reality device side and includes a second clock unit for generating a second audio timestamp signal and a second video timestamp signal; wherein the first clock unit and the second clock unit are communicatively connected such that the first clock unit controls the second audio timestamp signal and the second video timestamp signal to be synchronized with the first audio timestamp signal and the first video timestamp signal respectively.

[0007] For the in-vehicle viewing system according to an embodiment of the present invention, wherein the first clock unit is configured to: receive the second audio timestamp signal and the second video timestamp signal from the second clock unit; generate a control instruction based on a comparison between the second audio timestamp signal and the first audio timestamp signal and between the second video timestamp signal and the first video timestamp signal; and send the control instruction to the second clock unit, the control instruction being used to control the second audio timestamp signal and the second video timestamp signal to be synchronized with the first audio timestamp signal and the first video timestamp signal respectively.

[0008] For the in-vehicle viewing system according to an embodiment of the present invention or any one of the above embodiments, wherein the first audio-video data processing module further comprises: a first audio receiving unit, which is configured to obtain a decoded audio data stream and a first audio timestamp signal and send the obtained decoded audio data stream and the first audio timestamp signal to a first audio player on the vehicle head unit, such that the first audio player plays the decoded audio data stream based on the first audio timestamp signal; and a first video rendering unit, which is configured to obtain a decoded video data stream and a first video timestamp signal and send the obtained decoded video data stream and the first video timestamp signal to a first video player on the vehicle head unit, such that the first video player plays the decoded video data stream based on the first video timestamp signal.

[0009] The in-vehicle viewing system according to an embodiment of the present invention or any one of the above embodiments, wherein the first audio-visual data processing module further includes: a first audio sending unit configured to encapsulate an undecoded audio data stream into an audio data packet based on the Real-Time Transport Protocol (RTP) and send the encapsulated audio data packet based on the Real-Time Transport Protocol (RTP) to the second audio-visual data processing module; and a first video sending unit configured to encapsulate an undecoded video data stream into a video data packet based on the Real-Time Transport Protocol (RTP) and send the encapsulated video data packet based on the Real-Time Transport Protocol (RTP) to the second audio-visual data processing module.

[0010] The in-vehicle viewing system according to an embodiment of the present invention or any one of the above embodiments, wherein the second audio-visual data processing module further includes: a second audio receiving unit configured to obtain a decoded audio data stream and a second audio timestamp signal and send the obtained decoded audio data stream and the second audio timestamp signal to a second audio player at the augmented reality device side or the virtual reality device side, such that the second audio player plays the decoded audio data stream based on the second audio timestamp signal; and a second video rendering unit configured to obtain a decoded video data stream and a second video timestamp signal and send the obtained decoded video data stream and the second video timestamp signal to a second video player at the augmented reality device side or the virtual reality device side, such that the second video player plays the decoded video data stream based on the second video timestamp signal.

[0011] The in-vehicle viewing system according to an embodiment of the present invention or any one of the above embodiments, wherein the first audio timestamp signal, the first video timestamp signal, the second audio timestamp signal, and the second video timestamp signal are generated incrementally over time.

[0012] The in-vehicle viewing system according to an embodiment of the present invention or any one of the above embodiments, wherein the first clock unit is configured to: stop generating the first audio timestamp signal in response to the decoded audio data stream stopping to play; and stop generating the first video timestamp signal in response to the decoded video data stream stopping to play.

[0013] The in-vehicle viewing system according to an embodiment of the present invention or any one of the above embodiments, wherein the in-vehicle viewing system is configured to respond to a user input: switch the decoded audio data stream being played on a first audio player to a second audio player for playing, or switch the decoded audio data stream being played on the second audio player to the first audio player for playing; and switch the decoded video data stream being played on a first video player to a second video player for playing, or switch the decoded video data stream being played on the second video player to the first video player for playing.

[0014] According to a second aspect of the present invention, there is provided an in-vehicle viewing method, the method comprising: generating a first audio timestamp signal and a first video timestamp signal via a first clock unit at a vehicle head unit; generating a second audio timestamp signal and a second video timestamp signal via a second clock unit at an augmented reality device end or a virtual reality device end; and setting the first clock unit and the second clock unit to be communicatively connected such that the first clock unit controls the second audio timestamp signal and the second video timestamp signal to be synchronized with the first audio timestamp signal and the first video timestamp signal respectively.

[0015] The in-vehicle viewing method according to an embodiment of the present invention, wherein the method further comprises, via the first clock unit: receiving the second audio timestamp signal and the second video timestamp signal from the second clock unit; generating a control instruction based on a comparison between the second audio timestamp signal and the first audio timestamp signal and between the second video timestamp signal and the first video timestamp signal; and sending the control instruction to the second clock unit, the control instruction being used to control the second audio timestamp signal and the second video timestamp signal to be synchronized with the first audio timestamp signal and the first video timestamp signal respectively.

[0016] The in-vehicle viewing method according to an embodiment of the present invention or any one of the above embodiments, wherein the method further comprises: acquiring a decoded audio data stream and a first audio timestamp signal and sending the acquired decoded audio data stream and first audio timestamp signal to a first audio player at the vehicle head unit, such that the first audio player plays the decoded audio data stream based on the first audio timestamp signal; and acquiring a decoded video data stream and a first video timestamp signal and sending the acquired decoded video data stream and first video timestamp signal to a first video player at the vehicle head unit, such that the first video player plays the decoded video data stream based on the first video timestamp signal.

[0017] The in-vehicle viewing method according to an embodiment of the present invention or any one of the above embodiments, wherein the method further includes: encapsulating the undecoded audio data stream into an audio data packet based on the Real-Time Transport Protocol (RTP) and sending the encapsulated audio data packet based on the Real-Time Transport Protocol (RTP) to the augmented reality device or the virtual reality device; and encapsulating the undecoded video data stream into a video data packet based on the Real-Time Transport Protocol (RTP) and sending the encapsulated video data packet based on the Real-Time Transport Protocol (RTP) to the augmented reality device or the virtual reality device.

[0018] The in-vehicle viewing method according to an embodiment of the present invention or any one of the above embodiments, wherein the method further includes: obtaining the decoded audio data stream and the second audio timestamp signal and sending the obtained decoded audio data stream and the second audio timestamp signal to a second audio player at the augmented reality device or the virtual reality device, so that the second audio player plays the decoded audio data stream based on the second audio timestamp signal; and obtaining the decoded video data stream and the second video timestamp signal and sending the obtained decoded video data stream and the second video timestamp signal to a second video player at the augmented reality device or the virtual reality device, so that the second video player plays the decoded video data stream based on the second video timestamp signal.

[0019] The in-vehicle viewing method according to an embodiment of the present invention or any one of the above embodiments, wherein the first audio timestamp signal, the first video timestamp signal, the second audio timestamp signal, and the second video timestamp signal are generated incrementally over time.

[0020] The in-vehicle viewing method according to an embodiment of the present invention or any one of the above embodiments, wherein the method further includes via the first clock unit: stopping generating the first audio timestamp signal in response to the decoded audio data stream stopping playback; and stopping generating the first video timestamp signal in response to the decoded video data stream stopping playback.

[0021] The in-vehicle viewing method according to an embodiment of the present invention or any one of the above embodiments, wherein the method further includes in response to a user input: switching the decoded audio data stream being played on the first audio player to the second audio player for playback, or switching the decoded audio data stream being played on the second audio player to the first audio player for playback; and switching the decoded video data stream being played on the first video player to the second video player for playback, or switching the decoded video data stream being played on the second video player to the first video player for playback.

[0022] According to a third aspect of the present invention, there is provided a computer storage medium, the computer storage medium comprising instructions, the instructions when run, executing the in-vehicle movie viewing method according to the second aspect of the present invention.

[0023] According to one or more embodiments of the present invention, the in-vehicle viewing solution can apply virtual reality and / or augmented reality technology to the vehicle to provide an immersive entertainment experience for vehicle users. In addition, according to one or more embodiments of the present invention, the in-vehicle viewing solution can switch the audio and / or video played on the vehicle side to the augmented reality device side or the virtual reality device side for playback or switch the audio and / or video played on the augmented reality device side or the virtual reality device side to the vehicle side for playback according to user selection, and generate an audio timestamp signal and a video timestamp signal by setting independent clock units on the vehicle side and the augmented reality device side or the virtual reality device side, so that the playback on the vehicle side can be kept synchronized with the playback on the augmented reality device side or the virtual reality device side during the switching process, so that the vehicle user will not notice the playback delay caused by the switching process and affect the viewing experience, thereby providing a rich immersive entertainment experience for the vehicle user while ensuring driving safety. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] The above and / or other aspects and advantages of the present invention will become clearer and easier to understand through the following description of various aspects in conjunction with the accompanying drawings, in which the same or similar units are represented by the same reference numerals. In the accompanying drawings:

[0025] Figure 1 A schematic diagram of an in-vehicle movie viewing system according to one or more embodiments of the present invention is shown.

[0026] Figure 2 A schematic diagram of an in-vehicle movie viewing system according to one or more embodiments of the present invention is shown.

[0027] Figure 3 A flow chart of an in-vehicle movie viewing method according to one or more embodiments of the present invention is shown. DETAILED DESCRIPTION

[0028] The description of the following specific embodiments is merely exemplary in nature and is not intended to limit the disclosed technology or the application and use of the disclosed technology. In addition, it is not intended to be bound by any express or implied theory presented in the aforementioned technical field, background technology or the following specific embodiments.

[0029] In the following detailed description of the embodiments, numerous specific details are set forth in order to provide a more thorough understanding of the disclosed technology. However, it will be apparent to one of ordinary skill in the art that the disclosed technology may be practiced without these specific details. In other instances, well-known features have not been described in detail to avoid unnecessarily complicating the description.

[0030] Terms such as "comprising" and "including" mean that the technical solutions of the present invention do not exclude the case of having other units and steps that are not directly or explicitly stated in addition to the units and steps directly and explicitly stated in the specification. Terms such as "first" and "second" do not denote the order of the units in terms of time, space, size, etc., but are merely used to distinguish the units.

[0031] In the context of the present invention, the Real-Time Transport Protocol (RTP) is a connectionless data transport protocol. The real-time feature of the RTP can meet the real-time requirements for the transmission of streaming media data. The RTP packet contains fields such as data type, sequence number, timestamp, and synchronization source identifier, which provide strong support for detecting data loss and achieving data stream synchronization.

[0032] Hereinafter, various exemplary embodiments according to the present invention will be described in detail with reference to the accompanying drawings.

[0033] Figure 1 A schematic diagram of an in-vehicle viewing system according to one or more embodiments of the present invention is shown.

[0034] As Figure 1As shown in the figure, the in-vehicle viewing system 100 includes a first audio-video data processing module 110 and a second audio-video data processing module 120. The first audio-video data processing module 110 is disposed on the vehicle head unit side and includes a first clock unit 1101, and the first clock unit 1101 is configured to generate a first audio timestamp signal and a first video timestamp signal. The second audio-video data processing module 120 is disposed on the augmented reality device side or the virtual reality device side and includes a second clock unit 1201, and the second clock unit 1201 is configured to generate a second audio timestamp signal and a second video timestamp signal. The first clock unit 1101 and the second clock unit 1201 are communicatively connected such that the first clock unit 1101 controls the second audio timestamp signal and the second video timestamp signal to be synchronized with the first audio timestamp signal and the first video timestamp signal, respectively. It should be noted that the audio timestamp signal can be used to configure a corresponding timestamp for each frame of audio data included in the audio data, and the timestamp corresponding to each audio frame data represents the start time of the corresponding audio frame data. Similarly, the video timestamp signal can be used to configure a corresponding timestamp for each frame of video data included in the video data, and the timestamp corresponding to each video frame data represents the start time of the corresponding video frame data.

[0035] Optionally, the first clock unit 1101 can be configured to receive the second audio timestamp signal and the second video timestamp signal from the second clock unit 1201, generate a control instruction based on the comparison between the second audio timestamp signal and the first audio timestamp signal and between the second video timestamp signal and the first video timestamp signal, and send the generated control instruction to the second clock unit 1201, and the control instruction is used to control the second audio timestamp signal and the second video timestamp signal to be synchronized with the first audio timestamp signal and the first video timestamp signal, respectively.

[0036] Optionally, the first audio timestamp signal and the first video timestamp signal, and the second audio timestamp signal and the second video timestamp signal are generated incrementally over time, making it easier to control the second audio timestamp signal and the second video timestamp signal to be synchronized with the first audio timestamp signal and the first video timestamp signal, respectively. Exemplarily, if the audio data stream includes audio data A1, audio data A2, audio data A3, and audio data A4, and the timestamp of audio data A1 is T1, then in the case where the frame length is t, the timestamp of audio data A2 after audio data A1 is T1 + t, the timestamp of audio data A3 after audio data A2 is T1 + 2t, and the timestamp of audio data A4 after audio data A3 is T1 + 3t.

[0037] Optionally, the first clock unit 1101 and the second clock unit 1201 can establish a communication connection through a TCP long connection communication link, so that the communication link connection is not closed after each transmission of the audio timestamp signal, the video timestamp signal, and the control instruction, to achieve connection multiplexing and real-time communication between the first clock unit 1101 and the second clock unit 1201, thereby saving the time for creating a communication link connection between the in-vehicle device side and the augmented reality device side or the virtual reality device side.

[0038] Optionally, the first clock unit can be configured to stop generating the first audio timestamp signal in response to the decoded audio data stream stopping playing, and to stop generating the first video timestamp signal in response to the decoded video data stream stopping playing, so as to ensure that the audio and / or video played on the in-vehicle device side is synchronized with the audio and / or video played on the augmented reality device side or the virtual reality device side.

[0039] It should be noted that the second audio and video data processing module 120 can be set on multiple augmented reality device sides or virtual reality device sides, so that the audio and video data played on the in-vehicle device side can be shared with multiple augmented reality device sides or virtual reality devices, and by respectively setting independent clock units on the in-vehicle device side and the augmented reality device side or the virtual reality device side to generate audio timestamp signals and video timestamp signals, it is possible to keep the audio and / or video played on the in-vehicle device side synchronized with the audio and / or video played on the augmented reality device side or the virtual reality device side, thereby realizing synchronized viewing for multiple people.

[0040] Figure 2 The schematic diagram of an in-vehicle viewing system according to one or more embodiments of the present invention is shown.

[0041] As Figure 2 shown, the in-vehicle viewing system 200 includes a first audio and video data processing module 210 and a second audio and video data processing module 220.

[0042] The first audio and video data processing module 210 is set on the in-vehicle device side and includes a receiving unit 2101, a demultiplexing unit 2102, a first audio sending unit 2103, a first audio decoding unit 2104, a first video decoding unit 2105, a first video sending unit 2106, a first audio receiving unit 2107, a first clock unit 2108, and a first video rendering unit 2109. The first clock unit 2108 is used to generate a first audio timestamp signal and a first video timestamp signal. Optionally, the first audio and video data processing module 210 can be set in the in-vehicle entertainment system.

[0043] The second audio and video data processing module 220 is provided at the augmented reality device side or the virtual reality device side and includes a second audio processing unit 2201, a second video processing unit 2202, a second audio receiving unit 2203, a second clock unit 2204, and a second video rendering unit 2205. The second clock unit 2204 is configured to generate a second audio timestamp signal and a second video timestamp signal.

[0044] The first clock unit 2108 and the second clock unit 2204 are communicatively connected such that the first clock unit 2108 controls the second audio timestamp signal and the second video timestamp signal to be synchronized with the first audio timestamp signal and the first video timestamp signal, respectively. Optionally, the first clock unit 2108 may be configured to receive the second audio timestamp signal and the second video timestamp signal from the second clock unit 2204, generate a control instruction based on the comparison between the second audio timestamp signal and the first audio timestamp signal and between the second video timestamp signal and the first video timestamp signal, and send the generated control instruction to the second clock unit 2204, where the control instruction is used to control the second audio timestamp signal and the second video timestamp signal to be synchronized with the first audio timestamp signal and the first video timestamp signal, respectively.

[0045] The receiving unit 2101 may be configured to receive the audio and video data to be played and sequentially read the received audio and video data to be played.

[0046] The demultiplexing unit 2102 may be configured to demultiplex the audio and video data to be played to obtain an audio data stream and a video data stream, and selectively send the obtained audio data stream to the first audio sending unit 2103 and / or the first audio decoding unit 2104, and selectively send the obtained video data stream to the first video decoding unit 2105 and / or the first video sending unit 2106, depending on whether the playback is at the augmented reality device side or the virtual reality device side or at the in-vehicle device side. It can be understood that the audio data stream and the video data stream output by the demultiplexing unit 2102 are undecoded audio data stream and video data stream.

[0047] When playing audio at the augmented reality device side or the virtual reality device side, the demultiplexing unit 2102 may send the obtained undecoded audio data stream to the first audio sending unit 2103.

[0048] The first audio sending unit 2103 can be configured to encapsulate the undecoded audio data stream into an audio data packet based on the Real-Time Transport Protocol (RTP) and send the encapsulated audio data packet based on the RTP to the second audio processing unit 2201 of the second audio and video data processing module 220. Optionally, the first audio sending unit 2103 can be configured to cache the undecoded audio data stream and encapsulate the cached audio data stream into an audio data packet based on the RTP at a predetermined moment. The first audio sending unit 2103 directly encapsulates the undecoded audio data stream into an audio data packet based on the RTP and sends it to the second audio processing unit 2201 of the second audio and video data processing module 220, omitting the operations of decoding and playing the audio data stream at the in-vehicle device end and then encoding and transmitting it, significantly reducing the computing resources consumed at the in-vehicle device end.

[0049] The second audio processing unit 2201 can be configured to unpack the received encapsulated audio data packet based on the RTP to obtain an audio data stream, decode the audio data stream obtained by unpacking to obtain a decoded audio data stream, and send the decoded audio data stream to the second audio receiving unit 2203. The second audio receiving unit 2203 can be configured to obtain the decoded audio data stream from the second audio processing unit 2201 and the second audio timestamp signal from the second clock unit 2204, and send the obtained decoded audio data stream and the second audio timestamp signal to an audio player at the augmented reality device end or the virtual reality device end (for example, a speaker at the augmented reality device end or the virtual reality device end), so that the audio player plays the decoded audio data stream based on the second audio timestamp signal.

[0050] When playing audio at the in-vehicle device end, the demultiplexing unit 2102 can send the obtained undecoded audio data stream to the first audio decoding unit 2104.

[0051] The first audio decoding unit 2104 can be configured to decode the obtained undecoded audio data stream to obtain a decoded audio data stream, and send the decoded audio data stream to the first audio receiving unit 2107. Exemplarily, decoding the obtained undecoded audio data stream can include decoding the audio frame data contained in the audio data stream to restore the sampled audio sampling signal, so as to obtain the audio sampling data corresponding to each frame of audio. The first audio receiving unit 2107 can be configured to obtain the decoded audio data stream from the first audio decoding unit 2104 and the first audio timestamp signal from the first clock unit 2108, and send the obtained decoded audio data stream and the first audio timestamp signal to an audio player at the in-vehicle device end (for example, a car audio), so that the audio player plays the decoded audio data stream based on the first audio timestamp signal.

[0052] When playing a video on an augmented reality device or a virtual reality device, the demultiplexing unit 2102 may send the acquired undecoded video data stream to the first video sending unit 2106.

[0053] The first video sending unit 2106 may be configured to encapsulate the undecoded video data stream into video data packets based on the Real-Time Transport Protocol (RTP) and send the encapsulated video data packets based on the Real-Time Transport Protocol (RTP) to the second video processing unit 2202 of the second audio-video data processing module 220. Optionally, the first video sending unit 2106 may be configured to cache the undecoded video data stream and encapsulate the cached video data stream into video data packets based on the Real-Time Transport Protocol (RTP) at a predetermined moment. The first video sending unit 2106 directly encapsulates the undecoded video data stream into video data packets based on the Real-Time Transport Protocol (RTP) and sends them to the second video processing unit 2202 of the second audio-video data processing module 220, omitting the operations of decoding and playing the video data stream on the in-vehicle device and then encoding and transmitting it, significantly reducing the computing resources consumed by the in-vehicle device.

[0054] The second video processing unit 2202 may be configured to unpack the received encapsulated video data packets based on the Real-Time Transport Protocol (RTP) to obtain a video data stream, decode the video data stream obtained by unpacking to obtain a decoded video data stream, and send the decoded video data stream to the second video rendering unit 2205. The second video rendering unit 2205 may be configured to obtain the decoded video data stream from the second video processing unit 2202 and the second video timestamp signal from the second clock unit 2204, and send the obtained decoded video data stream and the second video timestamp signal to a video player (e.g., a display of an augmented reality device or a virtual reality device) at the augmented reality device end or the virtual reality device end, so that the video player plays the decoded video data stream based on the second video timestamp signal.

[0055] When playing a video on the in-vehicle device, the demultiplexing unit 2102 may send the acquired undecoded video data stream to the first video decoding unit 2105.

[0056] The first video decoding unit 2105 may be configured to decode the acquired undecoded video data stream to obtain a decoded video data stream, and send the decoded video data stream to the first video rendering unit 2109. Exemplarily, decoding the acquired undecoded video data stream may include decoding the video frame data included in the video data stream according to a decoding rule corresponding to the video encoding rule to restore the image signal, and obtaining the image data corresponding to each frame of video. The first video rendering unit 2109 may be configured to obtain the decoded video data stream from the first video decoding unit 2105 and obtain the first video timestamp signal from the first clock unit 2108, and send the obtained decoded video data stream and the first video timestamp signal to the video player (e.g., in-vehicle screen) of the vehicle head unit, so that the video player plays the decoded video data stream based on the first video timestamp signal.

[0057] Optionally, the in-vehicle viewing system 200 may be configured to, in response to a user input, switch the decoded audio data stream played by the audio player at the vehicle head unit to the audio player at the augmented reality device side or the virtual reality device side for playing, or switch the decoded audio data stream played by the audio player at the augmented reality device side or the virtual reality device side to the audio player at the vehicle head unit for playing, and switch the decoded video data stream played by the video player at the vehicle head unit to the video player at the augmented reality device side or the virtual reality device side for playing, or switch the decoded video data stream played by the video player at the augmented reality device side or the virtual reality device side to the video player at the vehicle head unit for playing. By flexibly switching the playback of audio and video data between the vehicle head unit and the augmented reality device side or the virtual reality device side in response to a user input, and ensuring that vehicle users are not aware of the playback delay caused by this switching process and thus not affecting the viewing experience, the needs of different vehicle users can be met, providing vehicle users with a diverse and immersive entertainment experience.

[0058] In one embodiment, when playing audio data and video data at the vehicle head unit and playing video data at the augmented reality device side or the virtual reality device side, in some driving situations, the in-vehicle viewing system 200 may be configured to, in response to a user input, switch the audio data played at the vehicle head unit to the audio player at the augmented reality device side or the virtual reality device side for playing, thereby preventing the driver from being disturbed by the audio sound and thus improving driving safety.

[0059] It should be noted that Figure 2 the block diagram of the in-vehicle viewing system according to one or more embodiments of the present invention shown Figure 2Each unit shown can be integrated into one unit or separated into other sub-units. Alternatively, other components different from the units shown in Figure 2 can also be used to implement the in-vehicle viewing system according to one or more embodiments of the present invention.

[0060] The in-vehicle viewing system proposed according to one aspect of the present invention can apply virtual reality and / or augmented reality technologies to a vehicle to provide an immersive entertainment experience for vehicle users. In addition, according to the in-vehicle viewing solution of one or more embodiments of the present invention, the audio and / or video played on the in-vehicle device can be switched to the augmented reality device side or the virtual reality device side for playing according to user selection, or the audio and / or video played on the augmented reality device side or the virtual reality device side can be switched to the in-vehicle device side for playing, and independent clock units are respectively set on the in-vehicle device side and the augmented reality device side or the virtual reality device side to generate audio timestamp signals and video timestamp signals, so that the playback on the in-vehicle device side can be kept synchronized with the playback on the augmented reality device side or the virtual reality device side during this switching process, so that vehicle users will not perceive the playback delay caused by this switching process and affect the viewing experience, thus ensuring driving safety while providing a rich immersive entertainment experience for vehicle users.

[0061] Figure 3 The flowchart of the in-vehicle viewing method according to one or more embodiments of the present invention is shown.

[0062] As Figure 3 shown, the in-vehicle viewing method according to one or more embodiments of the present invention includes the following steps:

[0063] Step S310: Generate a first audio timestamp signal and a first video timestamp signal on the in-vehicle device via a first clock unit;

[0064] Step S320: Generate a second audio timestamp signal and a second video timestamp signal on the augmented reality device side or the virtual reality device side via a second clock unit; and

[0065] Step S330: Set the first clock unit and the second clock unit to be communicatively connected, so that the first clock unit controls the second audio timestamp signal and the second video timestamp signal to be synchronized with the first audio timestamp signal and the first video timestamp signal respectively.

[0066] Optionally, in step S330, receive a second audio timestamp signal and a second video timestamp signal from the second clock unit via the first clock unit, generate a control instruction based on the comparison between the second audio timestamp signal and the first audio timestamp signal and between the second video timestamp signal and the first video timestamp signal, and send the control instruction to the second clock unit. The control instruction is used to control the second audio timestamp signal and the second video timestamp signal to be synchronized with the first audio timestamp signal and the first video timestamp signal respectively.

[0067] Optionally, the in-vehicle movie-watching method according to one or more embodiments of the present invention may further include switching the decoded audio data stream played by the audio player on the in-vehicle device to the audio player on the augmented reality device side or the virtual reality device side in response to a user input, or switching the decoded audio data stream played by the audio player on the augmented reality device side or the virtual reality device side to the audio player on the in-vehicle device side, and switching the decoded video data stream played by the video player on the in-vehicle device to the video player on the augmented reality device side or the virtual reality device side, or switching the decoded video data stream played by the video player on the augmented reality device side or the virtual reality device side to the video player on the in-vehicle device side. By flexibly switching the playback of audio-visual data between the in-vehicle device and the augmented reality device side or the virtual reality device side in response to a user input, the needs of different vehicle users can be met, and a diverse immersive entertainment experience can be provided for vehicle users.

[0068] Figure 3 Each step of the in-vehicle movie-watching method shown in can be implemented with reference to the in-vehicle movie-watching system 100 and the in-vehicle movie-watching system 200 described above, and the relevant content is incorporated herein by reference.

[0069] The in-vehicle movie-watching method proposed according to one aspect of the present invention can apply virtual reality and / or augmented reality technologies to vehicles to provide an immersive entertainment experience for vehicle users. In addition, according to the in-vehicle movie-watching solution of one or more embodiments of the present invention, the audio and / or video played on the in-vehicle device can be switched to the augmented reality device side or the virtual reality device side for playback according to user selection, or the audio and / or video played on the augmented reality device side or the virtual reality device side can be switched to the in-vehicle device side for playback. By respectively setting independent clock units on the in-vehicle device and the augmented reality device side or the virtual reality device side to generate audio timestamp signals and video timestamp signals, the playback on the in-vehicle device can be kept synchronized with the playback on the augmented reality device side or the virtual reality device side during this switching process, so that vehicle users will not perceive the playback delay caused by this switching process and affect the movie-watching experience, thus ensuring driving safety while providing a rich immersive entertainment experience for vehicle users.

[0070] In addition, as described above, the present invention can also be implemented as a computer storage medium in which is stored a method for in-vehicle movie viewing that causes a computer to execute in accordance with an aspect of the present invention.

[0071] Here, as the computer storage medium, various types of computer storage media such as disk types (e.g., magnetic disks, optical disks, etc.), card types (e.g., memory cards, optical cards, etc.), semiconductor memory types (e.g., read-only memories, non-volatile memories, etc.), tape types (e.g., magnetic tapes, cassette tapes, etc.) can be adopted.

[0072] In applicable cases, various embodiments provided by the present disclosure can be implemented using hardware, software, or a combination of hardware and software. Moreover, in applicable cases, without departing from the scope of the present disclosure, the various hardware components and / or software components described herein can be combined into composite components including software, hardware, and / or both. In applicable cases, without departing from the scope of the present disclosure, the various hardware components and / or software components described herein can be divided into sub-components including software, hardware, or both. Additionally, in applicable cases, it is contemplated that software components can be implemented as hardware components, and vice versa.

[0073] Software in accordance with the present disclosure (such as program code and / or data) can be stored on one or more computer storage media. It is also contemplated that one or more general-purpose or special-purpose computers and / or computer systems connected via a network and / or in other ways can be used to implement the software identified herein. In applicable cases, the order of the various steps described herein can be changed, combined into composite steps, and / or divided into sub-steps to provide the features described herein.

[0074] The embodiments and examples provided herein are for the purpose of best illustrating the embodiments in accordance with the present invention and its specific applications, and thereby enabling those skilled in the art to implement and use the present invention. However, those skilled in the art will know that the above description and examples are provided only for the sake of convenience of illustration and exemplification. The described presentation is not intended to cover all aspects of the present invention or to limit the present invention to the precise forms disclosed.

Claims

1. A vehicle-mounted viewing system, characterized in that, The in-vehicle viewing system includes: A first audio-video data processing module, which is arranged on the vehicle head unit and includes a first clock unit, and the first clock unit is used to generate a first audio timestamp signal and a first video timestamp signal; and A second audio-video data processing module, which is arranged on the augmented reality device side or the virtual reality device side and includes a second clock unit, and the second clock unit is used to generate a second audio timestamp signal and a second video timestamp signal; Wherein the first clock unit and the second clock unit are communicatively connected, so that the first clock unit controls the second audio timestamp signal and the second video timestamp signal to be synchronized with the first audio timestamp signal and the first video timestamp signal respectively, Wherein the first audio timestamp signal and the second audio timestamp signal are used to configure corresponding timestamps for each frame of audio data included in the audio data, and the first video timestamp signal and the second video timestamp signal are used to configure corresponding timestamps for each frame of video data included in the video data.

2. The in-vehicle viewing system according to claim 1, wherein the first clock unit is configured to: Receive the second audio timestamp signal and the second video timestamp signal from the second clock unit; Generate a control instruction according to the comparison between the second audio timestamp signal and the first audio timestamp signal and between the second video timestamp signal and the first video timestamp signal; and Send the control instruction to the second clock unit, and the control instruction is used to control the second audio timestamp signal and the second video timestamp signal to be synchronized with the first audio timestamp signal and the first video timestamp signal respectively.

3. The in-vehicle viewing system according to claim 1, wherein the first audio-video data processing module further includes: A first audio receiving unit, which is configured to obtain the decoded audio data stream and the first audio timestamp signal and send the obtained decoded audio data stream and the first audio timestamp signal to the first audio player on the vehicle head unit, so that the first audio player plays the decoded audio data stream based on the first audio timestamp signal; And A first video rendering unit, which is configured to obtain the decoded video data stream and the first video timestamp signal and send the obtained decoded video data stream and the first video timestamp signal to the first video player on the vehicle head unit, so that the first video player plays the decoded video data stream based on the first video timestamp signal.

4. The in-vehicle viewing system according to claim 1, wherein the first audio-video data processing module further includes: A first audio sending unit, which is configured to encapsulate the undecoded audio data stream into an audio data packet based on the Real-Time Transport Protocol (RTP) and send the encapsulated audio data packet based on the Real-Time Transport Protocol (RTP) to the second audio-video data processing module; And A first video sending unit, which is configured to encapsulate the undecoded video data stream into a video data packet based on the Real-Time Transport Protocol (RTP) and send the encapsulated video data packet based on the Real-Time Transport Protocol (RTP) to the second audio-video data processing module.

5. The in-vehicle viewing system according to claim 1, wherein the second audio-video data processing module further comprises: A second audio receiving unit configured to obtain a decoded audio data stream and a second audio timestamp signal and send the obtained decoded audio data stream and second audio timestamp signal to a second audio player at an augmented reality device end or a virtual reality device end, such that the second audio player plays the decoded audio data stream based on the second audio timestamp signal; And A second video rendering unit configured to obtain a decoded video data stream and a second video timestamp signal and send the obtained decoded video data stream and second video timestamp signal to a second video player at an augmented reality device end or a virtual reality device end, such that the second video player plays the decoded video data stream based on the second video timestamp signal.

6. The in-vehicle viewing system according to claim 1, wherein the first audio timestamp signal, the first video timestamp signal, the second audio timestamp signal, and the second video timestamp signal are generated incrementally over time.

7. The in-vehicle viewing system according to claim 3 or 5, wherein the first clock unit is configured to: Stop generating the first audio timestamp signal in response to the decoded audio data stream stopping playing; and Stop generating the first video timestamp signal in response to the decoded video data stream stopping playing.

8. The in-vehicle viewing system according to claim 3 or 5, wherein the in-vehicle viewing system is configured to respond to a user input: Switch the decoded audio data stream being played by the first audio player to the second audio player for playing, or switch the decoded audio data stream being played by the second audio player to the first audio player for playing; and Switch the decoded video data stream being played by the first video player to the second video player for playing, or switch the decoded video data stream being played by the second video player to the first video player for playing.

9. A vehicle-mounted movie viewing method, characterized in that, The method comprises: Generating a first audio timestamp signal and a first video timestamp signal via a first clock unit at a vehicle head unit; Generating a second audio timestamp signal and a second video timestamp signal via a second clock unit at an augmented reality device end or a virtual reality device end; and Setting the first clock unit and the second clock unit to be communicatively connected, such that the first clock unit controls the second audio timestamp signal and the second video timestamp signal to be synchronized with the first audio timestamp signal and the first video timestamp signal respectively, wherein the first audio timestamp signal and the second audio timestamp signal are used to configure corresponding timestamps for each frame of audio data included in the audio data, and the first video timestamp signal and the second video timestamp signal are used to configure corresponding timestamps for each frame of video data included in the video data.

10. The in-vehicle viewing method according to claim 9, wherein the method further comprises, via the first clock unit: Receive a second audio timestamp signal and a second video timestamp signal from the second clock unit; Generate a control instruction based on the comparison between the second audio timestamp signal and the first audio timestamp signal and between the second video timestamp signal and the first video timestamp signal; and Send the control instruction to the second clock unit, where the control instruction is used to control the second audio timestamp signal and the second video timestamp signal to be synchronized with the first audio timestamp signal and the first video timestamp signal respectively.

11. The in-vehicle movie viewing method according to claim 9, wherein the method further comprises: Obtain a decoded audio data stream and a first audio timestamp signal and send the obtained decoded audio data stream and first audio timestamp signal to a first audio player at the in-vehicle device end, so that the first audio player plays the decoded audio data stream based on the first audio timestamp signal; And Obtain a decoded video data stream and a first video timestamp signal and send the obtained decoded video data stream and first video timestamp signal to a first video player at the in-vehicle device end, so that the first video player plays the decoded video data stream based on the first video timestamp signal.

12. The in-vehicle movie viewing method according to claim 9, wherein the method further comprises: Encapsulate an undecoded audio data stream into an audio data packet based on the Real-Time Transport Protocol (RTP) and send the encapsulated audio data packet based on the Real-Time Transport Protocol (RTP) to the augmented reality device end or the virtual reality device; And Encapsulate an undecoded video data stream into a video data packet based on the Real-Time Transport Protocol (RTP) and send the encapsulated video data packet based on the Real-Time Transport Protocol (RTP) to the augmented reality device end or the virtual reality device.

13. The in-vehicle movie viewing method according to claim 9, wherein the method further comprises: Obtain a decoded audio data stream and a second audio timestamp signal and send the obtained decoded audio data stream and second audio timestamp signal to a second audio player at the augmented reality device end or the virtual reality device end, so that the second audio player plays the decoded audio data stream based on the second audio timestamp signal; And Obtain a decoded video data stream and a second video timestamp signal and send the obtained decoded video data stream and second video timestamp signal to a second video player at the augmented reality device end or the virtual reality device end, so that the second video player plays the decoded video data stream based on the second video timestamp signal.

14. The in-vehicle movie viewing method according to claim 9, wherein the first audio timestamp signal, the first video timestamp signal, the second audio timestamp signal, and the second video timestamp signal are generated incrementally over time.

15. The in-vehicle movie viewing method according to claim 11 or 13, wherein the method further comprises via the first clock unit: Stop generating the first audio timestamp signal in response to the decoded audio data stream stopping playback; and Stop generating the first video timestamp signal in response to the stop of playing the decoded video data stream.

16. The in-vehicle viewing method according to claim 11 or 13, wherein the method further comprises, in response to a user input: Switch the decoded audio data stream played on the first audio player to the second audio player for playing, or switch the decoded audio data stream played on the second audio player to the first audio player for playing; and Switch the decoded video data stream played on the first video player to the second video player for playing, or switch the decoded video data stream played on the second video player to the first video player for playing.

17. A computer storage medium, characterized in that, The computer storage medium includes instructions that, when running, execute the in-vehicle viewing method according to any one of claims 9 to 16.

Images