A method for vehicle-mounted DLP projection

Abstract

This application discloses an in-vehicle DLP projection method, an in-vehicle infotainment system projection method, an in-vehicle DLP projection synchronization method, an in-vehicle infotainment system projection synchronization method, and a vehicle, comprising: the in-vehicle DLP receiving a video frame queue from the in-vehicle infotainment system via RTP packets; after RTP unpacking and decoding, obtaining the video frame queue and marking it with a DLP-side timestamp; based on the DLP-side timestamp and the in-vehicle infotainment system timestamp, obtaining clock time difference information of the same video frame queue before the in-vehicle infotainment system receives RTP packets and after the in-vehicle DLP decodes the frames; after sending the clock time difference information of the video queue, in which the DLP-side timestamp and the in-vehicle infotainment system timestamp correspond, the in-vehicle DLP instantly plays the video queue on the in-vehicle DLP playback terminal; wherein the clock time difference information includes timestamp information. Through the above scheme, playback synchronization on the in-vehicle DLP and the in-vehicle infotainment system playback terminal is achieved.

Description

Technical Field

[0001] This application relates to the field of projection, and more particularly to a vehicle-mounted DLP projection method, a vehicle-mounted system projection method, a vehicle-mounted DLP projection synchronization method, a vehicle-mounted system projection synchronization method, and a vehicle. Background Technology

[0002] To improve the vehicle's ability to handle oncoming traffic, DLP headlights are installed, which control the illumination area. However, currently, DLP headlights are only used as headlights and have no other purpose.

[0003] On the other hand, as the main control system of in-vehicle equipment, the vehicle's infotainment system is becoming increasingly powerful, with entertainment and multimedia functions. If DLP headlights are used as terminals for playing video files, theoretically, they can be expanded into powerful outdoor camping party cinemas and karaoke entertainment halls.

[0004] However, before being integrated into in-vehicle equipment, DLP headlights, although used as video projection devices, could not provide high-quality entertainment in vehicles because of a synchronization issue during file playback. There was a slight delay between the video stream played from the DLP headlights and the audio stream played by the vehicle's infotainment system, which affected the user experience.

[0005] Therefore, a solution is needed to simultaneously output the same video file to be played on different output devices, thereby improving the user's sensory experience. Summary of the Invention

[0006] The purpose of this invention is to provide an in-vehicle DLP projection method, an in-vehicle projector projection method, an in-vehicle DLP projection system, an in-vehicle DLP projection synchronization method, an in-vehicle projector projection synchronization method, and a vehicle, thereby solving at least one of the aforementioned technical problems.

[0007] This invention provides the following solution:

[0008] According to one aspect of the present invention, a vehicle-mounted projection method is provided, wherein the in-vehicle DLP is synchronized with the vehicle's clock via the NTP protocol, the vehicle-mounted projection method comprising:

[0009] Get the video file to be played;

[0010] The video file to be played is digitally decoded to separate the video frame queue and the audio frame queue;

[0011] Mark the video frame queue and audio frame queue with the vehicle-mounted timestamp;

[0012] The video frame queue is sent to the vehicle-mounted DLP via RTP packets;

[0013] The video frame queue and audio frame queue are separated into an image queue and a PCM queue that correspond to each other in a frame-to-frame relationship;

[0014] The image queue and PCM queue are equipped with vehicle-mounted timestamps.

[0015] The image queue and PCM queue stack are stored in the vehicle's infotainment system for playback.

[0016] Receive clock difference information from the video frame queue of the vehicle-mounted DLP;

[0017] The clock time difference information corresponds to the image queue and PCM queue that mark the timestamp on the vehicle terminal;

[0018] Based on the clock time difference information and the corresponding image queue and PCM queue timestamp information on the vehicle terminal, the delayed playback information is obtained;

[0019] Based on the delayed playback information, control the storage time of the image queue and PCM queue stacks;

[0020] The in-vehicle infotainment system plays the image queue and PCM queue after a stack storage delay.

[0021] According to a second aspect of the present invention, an in-vehicle DLP projection method is provided, wherein the in-vehicle DLP is synchronized with the vehicle's clock via the NTP protocol, the in-vehicle DLP projection method comprising:

[0022] The in-vehicle DLP receives the video frame queue from the vehicle's infotainment system via RTP packets;

[0023] After RTP unpacking, the video frame queue is obtained through decoding and the DLP end timestamp is marked.

[0024] Based on the timestamps of the DLP end and the vehicle terminal, obtain the clock time difference information of the same video frame queue before the vehicle terminal passes through the RTP packet and after the vehicle-mounted DLP decodes it;

[0025] After sending the clock difference information of the video queue, which corresponds to the timestamps of the DLP terminal and the vehicle terminal, the vehicle-mounted DLP will play the video queue in real time on the playback terminal of the vehicle-mounted DLP.

[0026] The clock time difference information includes timestamp information.

[0027] According to three aspects of the present invention, an in-vehicle DLP projection system is provided, the in-vehicle DLP projection system comprising: an in-vehicle head unit and an in-vehicle DLP;

[0028] The vehicle infotainment system includes an underlying service module;

[0029] The underlying service modules include an NTP server module and an RTP server module;

[0030] The NTP server module is used as the NTP server on the vehicle-mounted system side to synchronize the vehicle-mounted system with the vehicle-mounted DLP clock.

[0031] The RTP server module is used to send the video file to be played from the vehicle's infotainment system to the in-vehicle DLP terminal.

[0032] Based on the clock synchronization between the vehicle's infotainment system and the in-vehicle DLP by the NTP server module, the same video file to be played is marked with a timestamp on the vehicle's infotainment system.

[0033] The in-vehicle DLP marks the video file to be played with a DLP-side timestamp.

[0034] The vehicle-mounted DLP generates clock time difference information based on the timestamp on the vehicle-mounted device and the timestamp on the DLP, which are marked on the video file to be played on the vehicle-mounted DLP, and sends it to the vehicle-mounted device.

[0035] The vehicle's infotainment system receives clock time difference information from the in-vehicle DLP.

[0036] The vehicle system delays the playback of the video file to be played on the vehicle system playback terminal based on the clock time difference information.

[0037] Furthermore, the vehicle infotainment system also includes: a user service module;

[0038] The user service module includes a local player module and a third-party player module, which are used to manage video files to be played.

[0039] The underlying service module also includes a remote playback service module;

[0040] The remote playback service module is used to store the same video file to be played in the vehicle's infotainment system and send it to the in-vehicle DLP, respectively.

[0041] The local player module or the third-party player module sends a request to the remote playback service module to play the video file to be played.

[0042] The remote playback service module marks the video file to be played with a timestamp on the vehicle terminal according to the request to play the video file sent by the local player module or the third-party player module, and stores it in the vehicle terminal and sends it to the vehicle DLP.

[0043] The remote playback service module sends the video file to be played, which has a timestamp from the vehicle's infotainment system, to the in-vehicle DLP via the RTP server module.

[0044] Furthermore, the underlying service module also includes: a multimedia extraction module and an audio decoding module;

[0045] The multimedia extraction module is used to extract the video files to be played into an image queue before storing the video file stack;

[0046] The audio decoding module is used to strip the video file to be played into a PCM queue before storing the video file stack;

[0047] The image queue and the PCM queue are marked with the vehicle-mounted timestamp of the video file to be played, and are stored in the vehicle-mounted stack before playback.

[0048] Furthermore, the audio decoding module also includes: an audio encoding / decoding module and an audio tracking module;

[0049] The audio encoding / decoding module is used to process the video file to be played and obtain the PCM queue;

[0050] The audio tracking module is used to control the stack storage time of the PCM queue according to the clock time difference information, and to play the PCM queue from the vehicle audio playback terminal after the stack storage delay.

[0051] According to four aspects of the present invention, a method for synchronizing vehicle-mounted DLP projection is provided, comprising:

[0052] After the vehicle-mounted DLP is woken up or started, it sends an NTP clock synchronization request to the vehicle's infotainment system.

[0053] The in-vehicle DLP receives a queue of video frames from the vehicle's infotainment system;

[0054] The vehicle-mounted DLP obtains the average value of the latency of a single video frame in the video frame queue based on the latency from the vehicle's head unit to the vehicle-mounted DLP head unit, and returns it to the vehicle's head unit.

[0055] The in-vehicle DLP batch processes and plays video frame queues from the vehicle's infotainment system;

[0056] The vehicle-mounted DLP periodically selects a group of video frames with vehicle-mounted timestamps at preset time intervals to obtain the average video latency of a single frame in the video frame queue.

[0057] According to five aspects of the present invention, a method for synchronizing vehicle-mounted system projection is provided, comprising:

[0058] The vehicle-mounted system receives an NTP clock synchronization request from the vehicle-mounted DLP and aligns its clock with the vehicle-mounted DLP according to the NTP clock synchronization request.

[0059] The vehicle-mounted system sends a queue of video frames to the in-vehicle DLP.

[0060] The vehicle-mounted unit receives the average delay of a single video frame from the video frame queue returned by the vehicle-mounted DLP, and refreshes the record of clock difference information.

[0061] The vehicle-mounted system sends a batch of video frame queues to the in-vehicle DLP.

[0062] The vehicle's infotainment system delays the playback of video frame queues locally based on clock time difference information;

[0063] The vehicle-mounted system periodically selects a set of video frame queues, marks the timestamps on the vehicle-mounted system, and sends them to the in-vehicle DLP at preset time intervals.

[0064] The vehicle system periodically refreshes its clock time difference records based on the acquired clock time difference information.

[0065] According to six aspects of the present invention, a vehicle is provided, comprising: a vehicle infotainment system, a central control screen, an instrument panel screen, a DLP projection headlight, a microphone, and a digital power amplifier system;

[0066] The digital amplifier system includes multiple speakers arranged in the car cabin;

[0067] The vehicle infotainment system is connected to the central control screen, instrument panel, and DLP projection headlights, and synchronized with the NTP clock.

[0068] The vehicle-mounted system is connected to the DLP projection headlight via Ethernet, and transmits a video file to be played that is marked with a timestamp from the vehicle-mounted system to the DLP projection headlight.

[0069] The DLP projection lamp receives the timestamp of the video file to be played from the DLP end.

[0070] The clock difference information of the video file to be played under NTP clock synchronization is obtained based on the timestamps of the vehicle-mounted system and the DLP-mounted system.

[0071] The DLP projection headlight transmits the clock time difference information of the video file to be played back to the vehicle's infotainment system.

[0072] The vehicle system delays the playback of the video file to be played on the central control screen, instrument panel, and digital amplifier system based on the clock time difference information of the video file to be played.

[0073] The microphone is used to record sound, which is then played back in real time by the digital amplifier system.

[0074] Furthermore, the DLP projection headlight includes a deserializer and a dual-lamp fusion module:

[0075] The deserializer is used to process the video file to be played into multiple synchronized video signals output by DLP projector lamps.

[0076] The dual-lamp fusion module is used to control multiple DLP projection lamps to focus the same video pixels onto a screen at a preset distance.

[0077] The deserializer processes the video file to be played and outputs it to the dual-lamp fusion module, which controls multiple DLP projection lamps to output video.

[0078] The dual-lamp fusion module controls the video image pixels of multiple DLP projection lamps to focus on the screen at a preset distance. Through this solution, the following beneficial technical effects are achieved:

[0079] This application uses the NTP clock synchronization method to first establish synchronization between the vehicle's infotainment system and the in-vehicle DLP system. Based on this, the file to be played is timestamped, so that the file to be played is played after delay compensation in the vehicle's infotainment system, thus achieving synchronized playback on the in-vehicle DLP and the in-vehicle playback terminal.

[0080] This application eliminates device latency by unifying the management of the display device and audio playback device under the vehicle system. It processes the playback file and microphone audio separately with delayed playback and instant playback, thus eliminating the sense of delay and misalignment between singing audio and background audio and video when used as a karaoke room function.

[0081] This application fully utilizes the hardware resources of the vehicle having two DLP devices by controlling the pixel focal length of the two DLP headlights, achieving a stereoscopic effect that the original single DLP device could not achieve, making the image presented on the screen vivid and realistic. Attached Figure Description

[0082] Figure 1 This is a flowchart of a vehicle-mounted projection method provided by one or more embodiments of the present invention.

[0083] Figure 2 This is a flowchart of an in-vehicle DLP projection method provided by one or more embodiments of the present invention.

[0084] Figure 3 This is a structural block diagram of an in-vehicle DLP projection system provided by one or more embodiments of the present invention.

[0085] Figure 4 This is a flowchart of a vehicle-mounted DLP projection synchronization method provided by one or more embodiments of the present invention.

[0086] Figure 5 This is a flowchart of a vehicle-mounted projection synchronization method provided by one or more embodiments of the present invention.

[0087] Figure 6 This is a structural block diagram of a vehicle provided by one or more embodiments of the present invention.

[0088] Figure 7This is a structural block diagram of a DLP projection headlight provided in one or more embodiments of the present invention.

[0089] Figure 8 This is a flowchart of an audio-video synchronization scheme according to a specific embodiment of the present invention.

[0090] Figure 9 This is an audio-video synchronization timing diagram of a specific embodiment of the present invention.

[0091] Figure 10 This is a block diagram of an electronic device structure for a vehicle projection method provided in one or more embodiments of the present invention. Detailed Implementation

[0092] The technical solution of the present invention will now be clearly and completely described with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0093] Figure 1 This is a flowchart of a vehicle-mounted projection method provided by one or more embodiments of the present invention.

[0094] like Figure 1 As shown, the in-vehicle DLP synchronizes with the vehicle's clock via the NTP protocol, and the projection methods for the in-vehicle system include:

[0095] Step S01: Obtain the video file to be played;

[0096] Step S01: Digitally decode the video file to be played, separating the video frame queue and the audio frame queue;

[0097] Step S03: Mark the video frame queue and audio frame queue with the vehicle-mounted timestamp;

[0098] Step S04: Send the video frame queue to the vehicle-mounted DLP via RTP packets;

[0099] Step S05: Separate the video frame queue and audio frame queue into the image queue and PCM queue, which are in a frame correspondence relationship.

[0100] Step S06, wherein the image queue and the PCM queue are equipped with vehicle-mounted timestamps;

[0101] Step S07: Store the image queue and PCM queue stack to the vehicle's infotainment system for playback.

[0102] Step S08: Receive clock time difference information from the video frame queue of the vehicle-mounted DLP;

[0103] Step S09, wherein the clock time difference information corresponds to the image queue and PCM queue that are marked with timestamps on the vehicle terminal;

[0104] Step S10: Obtain delayed playback information based on the clock time difference information and the timestamp information of the corresponding image queue and PCM queue on the vehicle terminal;

[0105] Step S11: Based on the delayed playback information, control the storage time of the image queue and PCM queue stacks;

[0106] Step S12: Play the image queue and PCM queue after stack storage delay on the playback terminal of the vehicle's infotainment system.

[0107] Specifically, when processing the same video or audio stream file, devices with different performance levels will accumulate latency due to variations in encoding / decoding efficiency and processing difficulty when playing the same file at the actual output or playback end. Although each device has a local buffering mechanism, it only serves the smooth playback of the local device itself and cannot synchronize the actual playback of multiple playback devices. For example, directly playing the same video stream on a car stereo and an in-vehicle DLP device will result in accumulated latency.

[0108] Therefore, it is necessary to integrate different devices under the same clock system to eliminate latency between devices for the same audio and video streams. To maintain communication and information processing synchronization between devices, clock synchronization is required, which can be achieved using technologies such as NTP. Network Time Protocol (NTP) is an application layer protocol within the TCP / IP protocol suite, used to synchronize the clocks between clients and servers, providing highly accurate time correction. The NTP server receives precise Coordinated Universal Time (UTC) from an authoritative clock source (such as atomic clocks or GPS), and the client then requests and receives the time from the server.

[0109] For example, in the Network Time Protocol (NTP), the client first sends an NTP request message to the server, which includes the timestamp t1 of the message leaving the client. The NTP request message arrives at the NTP server at time t2. When the server receives the message, it processes it and sends an NTP response message at time t3. This response message carries the timestamps t1 (when the message left the NTP client), t2 (when it arrived at the NTP server), and t3 (when it left the NTP server). When the client receives the response message, it records the timestamp t4 of the message's return. Using these four timestamps, the client can calculate two key parameters: the round-trip delay (delay) of the NTP message from the client to the server.

[0110] delay = (t4 - t1)(t3 - t2)

[0111] Obtain the time difference offset between the client and the server.

[0112] Then, according to the system of equations:

[0113]

[0114] The time difference can be solved as follows:

[0115]

[0116] The NTP client adjusts its clock based on the calculated offset to synchronize with the NTP server's clock.

[0117] Therefore, the vehicle's system clock can be used as the authoritative clock, and NTP technology can be used to establish system clock synchronization between the vehicle's infotainment system and the in-vehicle DLP.

[0118] Under NTP technology, incoming and outgoing messages carry timestamps or clock stamps. The clock stamps under NTP technology can be used as a reference to mark audio and video stream files, so that when the vehicle head unit and the vehicle DLP play the same audio and video stream file (such as the video file to be played), the output at the playback end is synchronized, thereby balancing the playback asynchrony caused by differences in device performance, encoding and decoding efficiency, etc.

[0119] Since the in-vehicle infotainment system manages both video and audio playback, the video file to be played must first be separated into video frame queues and audio frame queues for playback on different devices. Additionally, the in-vehicle DLP plays video streams but does not require audio streams, which need to be filtered out. The file to be played can be segmented into video frame queues of a certain length according to a specific beat cycle. Each video frame queue undergoes encoding and decoding processes such as encapsulation and decapsulation, allowing the in-vehicle DLP to obtain the video frame queues. This is equivalent to the in-vehicle DLP obtaining a copy of the video stream containing the same frame content as the one on the in-vehicle infotainment system. When the video frame queues and audio frame queues are on the in-vehicle infotainment system side, timestamps are added indicating when the video frame queues and audio frame queues were generated or processed on the in-vehicle infotainment system. When the video frame queues are on the in-vehicle DLP side, timestamps are added indicating when they were processed on the in-vehicle DLP side. After the video frame queue is sent to the vehicle-mounted DLP terminal, it is temporarily stored locally (such as in a stack, first-in-first-out) while waiting for the vehicle-mounted DLP to return information with the vehicle-mounted DLP terminal's timestamp (such as clock time difference information). Based on the clock time difference information, the video frame queue and audio frame queue with the same frame content can be delayed and waited on the vehicle-mounted terminal, so that when the vehicle-mounted DLP plays the video on the playback terminal, it can retrieve the image queue and PCM queue in the stack frame by frame to achieve simultaneous playback alignment.

[0120] Since the packet sizes of the video frame queue and the audio frame queue are not uniquely fixed, but are played frame by frame on the in-vehicle infotainment system, the video frame queue and the audio frame queue are aligned by timestamps, and then further processed on the in-vehicle infotainment system into a unified form of image queue and PCM queue, thus eliminating the impact of inconsistent packet sizes.

[0121] Timestamps can be used as identifiers for the same frame queue, ensuring that the car stereo and the in-vehicle DLP playback file are the same.

[0122] Figure 2 This is a flowchart of an in-vehicle DLP projection method provided by one or more embodiments of the present invention.

[0123] like Figure 2 As shown, the in-vehicle DLP synchronizes with the vehicle's clock via the NTP protocol. The in-vehicle DLP projection methods include:

[0124] Step S21: The vehicle-mounted DLP receives a queue of video frames from the vehicle's infotainment system via RTP packets.

[0125] Step S22: After RTP unpacking, the video frame queue is obtained and marked with DLP timestamps after decoding;

[0126] Step S23: Based on the DLP end timestamp and the vehicle end timestamp, obtain the clock time difference information of the same video frame queue before the vehicle end passes through the RTP packet and after the vehicle DLP decodes it;

[0127] Step S24: After sending the clock difference information of the video queue whose timestamps correspond to those of the DLP terminal and the vehicle terminal, the vehicle DLP will play the video queue in real time on the playback terminal of the vehicle DLP.

[0128] Step S25, wherein the clock time difference information includes timestamp information.

[0129] Specifically, the timestamp marked on the vehicle-mounted terminal is the timestamp before RTP packet encapsulation. After encapsulation, the video frame queue enters the temporary stack storage stage. The timestamp marked on the vehicle-mounted DLP is the timestamp when the video frame queue available for playback is obtained after unpacking and decoding. The vehicle-mounted DLP obtains the clock difference information through the timestamps at both ends. Since the video frame queue corresponding to RTP packet encapsulation, unpacking, encoding, and decoding is not a unique fixed size, the clock difference needs to be dynamically adjusted. However, the return clock difference is fixed, and the resulting latency is fixed, which can be added as a fixed constant to the clock difference information processing. The vehicle-mounted system obtains the clock difference information along with the timestamp information marked on both ends. It can find the corresponding video frame queue in the stack based on the timestamp on the vehicle-mounted terminal, and can also control the release of the video frame queue from the stack for playback based on the clock difference information.

[0130] Figure 3 This is a structural block diagram of an in-vehicle DLP projection system provided by one or more embodiments of the present invention.

[0131] like Figure 3 As shown, the in-vehicle DLP projection system includes: a vehicle-mounted projector and an in-vehicle DLP;

[0132] The vehicle infotainment system includes underlying service modules;

[0133] The underlying service modules include an NTP server module and an RTP server module;

[0134] The NTP server module is used as the NTP server on the vehicle-mounted system side to synchronize the vehicle-mounted system clock with the vehicle-mounted DLP clock.

[0135] The RTP server module is used to send the video files to be played from the vehicle's infotainment system to the in-vehicle DLP terminal.

[0136] Based on the clock synchronization between the vehicle's infotainment system and the in-vehicle DLP via the NTP server module, the same video file to be played is marked with a timestamp on the vehicle's infotainment system.

[0137] The in-vehicle DLP tagged the video file to be played with a DLP-side timestamp.

[0138] The vehicle-mounted DLP generates clock time difference information based on the timestamp on the video file to be played on the vehicle-mounted DLP terminal and the timestamp on the DLP terminal, and sends it to the vehicle-mounted terminal.

[0139] The vehicle's infotainment system receives clock time difference information from the in-vehicle DLP.

[0140] The in-vehicle infotainment system delays the playback of video files on its playback device based on clock time difference information.

[0141] Specifically, the NTP server module synchronizes the clocks of the vehicle's infotainment system and the in-vehicle DLP system, while the RTP server module transmits the files to be played as video frame queues to the in-vehicle DLP system. Each module handles different tasks. By marking timestamps on both ends of the files containing the same frame content, the system obtains the latency information for delayed playback on the vehicle's infotainment system, thus achieving playback synchronization.

[0142] In this embodiment, the vehicle infotainment system further includes: a user service module;

[0143] The user service module includes a local player module and a third-party player module, which are used to manage video files to be played.

[0144] The underlying service module also includes a remote playback service module;

[0145] The remote playback service module is used to stack the same video file to be played in the vehicle's infotainment system and send it to the in-vehicle DLP separately;

[0146] The local player module or a third-party player module sends a request to the remote playback service module to play the video file to be played.

[0147] The remote playback service module marks the video file to be played with a timestamp on the vehicle's infotainment system based on the request to play the video file sent by the local player module or a third-party player module. The timestamp is then stored in the vehicle's infotainment system and sent to the in-vehicle DLP.

[0148] The remote playback service module sends the video file to be played, which has the vehicle's head unit's timestamp, to the in-vehicle DLP via the RTP server module.

[0149] Specifically, the decision to play a file is initiated by the player, such as when a user selects a video file to play on the UI. Before the video file is sent to the in-vehicle DLP, it is timestamped and then stored on the vehicle's infotainment system and sent to the in-vehicle DLP, ensuring both systems receive the file. The system then waits for the in-vehicle DLP to return a delay message for the currently stored video file. After this delay, the vehicle's infotainment system begins playing the file.

[0150] The timestamp on the vehicle's infotainment system indicates which video frame queue to retrieve from the stack for playback, while the timestamp on the DLP (Digital Perceptron) system reveals the latency difference between the file currently being played on the DLP and the file being played on the vehicle's infotainment system.

[0151] In this embodiment, the underlying service module further includes: a multimedia extraction module and an audio decoding module;

[0152] The multimedia extraction module is used to extract images into a queue before storing the video file stack to be played.

[0153] The audio decoding module is used to strip the video file to be played into a PCM queue before storing it in a stack.

[0154] The image queue and PCM queue are marked with the vehicle's onboard timestamp of the video file to be played, and are stored in the vehicle's stack before playback.

[0155] Specifically, through timestamp marking, the video stream playback on the in-vehicle DLP terminal is first aligned with that on the vehicle's infotainment system, and then the audio stream playback on the vehicle's infotainment system is aligned with that on the video stream.

[0156] By aligning the image queue and PCM queue frame by frame, the audio stream and video stream are played in the vehicle's infotainment system in a frame-by-frame manner.

[0157] In this embodiment, the audio decoding module further includes: an audio encoding / decoding module and an audio tracking module;

[0158] The audio codec module is used to process the video file to be played and obtain the PCM queue;

[0159] The audio tracking module is used to control the stack storage time of the PCM queue based on the clock time difference information, and plays the PCM queue from the vehicle audio playback terminal after the stack storage delay.

[0160] Specifically, the PCM queue can indirectly obtain alignment playback information from the image queue, or it can directly control the stack storage time of the PCM queue through clock time difference information to achieve audio and video alignment. For example, if the vehicle's infotainment system only activates the audio playback module while the video playback module is not activated, the image queue and PCM queue can be aligned for playback based solely on timestamps, without being limited to the requirement that both files be video stream files or video queues to handle delayed playback control flow.

[0161] Figure 4 This is a flowchart of a vehicle-mounted DLP projection synchronization method provided by one or more embodiments of the present invention.

[0162] like Figure 4 As shown, the vehicle-mounted DLP projection synchronization method includes:

[0163] Step S31: After the vehicle-mounted DLP is woken up or started, it sends an NTP clock synchronization request to the vehicle's infotainment system.

[0164] Step S32: The vehicle-mounted DLP receives a queue of video frames from the vehicle's infotainment system;

[0165] Step S33: The vehicle-mounted DLP obtains the average value of the latency of a single video frame in the video frame queue based on the latency from the vehicle's head unit to the vehicle-mounted DLP head unit, and returns it to the head unit.

[0166] Step S34: The in-vehicle DLP batch processes and plays the video frame queue from the vehicle's infotainment system;

[0167] Step S35: The vehicle-mounted DLP periodically selects a group of video frames with vehicle terminal timestamps at preset time intervals to obtain the average value of the single-frame video latency in the video frame queue.

[0168] Specifically, after starting the in-vehicle DLP, it first establishes clock synchronization with the vehicle's infotainment system before receiving the video frame queue. Since the packet length of each video frame queue varies, the delay information also differs each time. The delay time can be averaged across each video frame, and delay processing can be performed frame-by-frame on the file to be played. Because processing timestamps consumes computational resources, it's unnecessary to perform extensive delay information calculations constantly. Delay information from a single instance can be stored in a temporary memory, and subsequent video and audio frame queues can be processed accordingly. After a preset time interval, the timestamp information is collected again, continuously updating the playback delay record and forming a periodic correction control operation.

[0169] Figure 5 This is a flowchart of a vehicle-mounted projection synchronization method provided by one or more embodiments of the present invention.

[0170] like Figure 5 As shown, the vehicle-mounted projection synchronization method includes:

[0171] Step S41: The vehicle unit receives the NTP clock synchronization request from the vehicle-mounted DLP and aligns the clock with the vehicle-mounted DLP according to the NTP clock synchronization request.

[0172] Step S42: The vehicle's infotainment system sends a queue of video frames to the in-vehicle DLP.

[0173] Step S43: The vehicle unit receives the average delay of a single video frame in the video frame queue returned by the vehicle-mounted DLP, and refreshes the record of clock time difference information.

[0174] Step S44: The vehicle's infotainment system sends a batch of video frame queues to the in-vehicle DLP.

[0175] Step S45: The vehicle's infotainment system delays the playback of the video frame queue locally based on the clock time difference information;

[0176] Step S46: The vehicle-mounted system periodically selects a set of video frame queues, marks the timestamp on the vehicle-mounted system, and sends them to the vehicle-mounted DLP at preset time intervals;

[0177] Step S47: The vehicle system periodically refreshes the clock time difference information record based on the acquired clock time difference information.

[0178] Specifically, after aligning its clock with the in-vehicle DLP, the vehicle's infotainment system sends a queue of video frames to the in-vehicle DLP. The infotainment system then uses the returned single-frame delay information to control the delay of the video file playback. The infotainment system doesn't need to rely on receiving single-frame delay information from the in-vehicle DLP for delay control every time; it can store the single-frame delay information locally for subsequent video stream playback. Updating and correcting only the single-frame delay information is sufficient to meet the synchronization requirements of the playback end.

[0179] After a preset time, new single-frame delay information is obtained to refresh the stored records.

[0180] Figure 6 This is a structural block diagram of a vehicle provided by one or more embodiments of the present invention.

[0181] like Figure 6 As shown, the vehicle includes: in-vehicle infotainment system, central control screen, instrument panel, DLP projection headlights, microphone, and digital amplifier system;

[0182] The digital amplifier system includes multiple speakers arranged in the car cabin;

[0183] The vehicle's infotainment system is connected to the central control screen, instrument panel, and DLP projection headlights, and synchronized with the NTP clock.

[0184] The vehicle's infotainment system is connected to the DLP projection headlight via Ethernet, transmitting video files to be played that are marked with timestamps from the vehicle's infotainment system to the DLP projection headlight.

[0185] The DLP projector headlight receives the video file to be played and marks the DLP end with a timestamp.

[0186] The clock difference information of the video file to be played under NTP clock synchronization is obtained based on the timestamps of the vehicle-mounted system and the DLP-mounted system.

[0187] The DLP projector headlights transmit the clock time difference information of the video file to be played back to the vehicle's infotainment system.

[0188] The vehicle's infotainment system delays the playback of the video file on the central control screen, instrument panel, and digital amplifier system based on the clock time difference information of the video file to be played.

[0189] The microphone is used to record sound, which is then played back in real time by the digital amplifier system.

[0190] Specifically, the central control screen, instrument panel screen, and DLP projection headlights act as video playback terminals, playing the video stream of the file to be played. The digital amplifier system acts as the audio playback terminal, playing the audio stream of the file to be played. Through NTP clock synchronization, the video stream output from the DLP projection headlights and the central control screen and instrument panel is synchronized, as is the audio stream output from the digital amplifier system. However, the microphone's audio comes from the user's singing, synchronized with the video and audio from the video playback terminals. The microphone's audio playback strategy is real-time playback and cannot be mixed with the playback synchronization between the DLP projection headlights and the vehicle's infotainment system.

[0191] If the audio picked up by the microphone must be mixed with the audio in the video file to be played, it needs to be mixed with the PCM audio frame queue that will be played after a delay before being output to ensure that the sound picked up by the microphone is output in real time.

[0192] Figure 7 This is a structural block diagram of a DLP projection headlight provided in one or more embodiments of the present invention.

[0193] like Figure 7 As shown, the DLP projection headlight includes a deserializer and a dual-lamp blending module:

[0194] The deserializer is used to process the video file to be played into multiple synchronized video signals output by DLP projector lamps;

[0195] The dual-lamp blending module is used to control multiple DLP projector lamps to focus the same video pixels onto a screen at a preset distance;

[0196] The deserializer processes the video file to be played and outputs it to the dual-lamp blending module, which controls multiple DLP projector lamps to output video.

[0197] The dual-lamp fusion module controls multiple DLP projector lamps to focus the video image pixels onto the screen at a preset distance.

[0198] Specifically, the vehicle-mounted system includes a serializer that works with the DLP projection headlights, and a deserializer to output multiple DLP projection headlights. The vehicle-mounted system sends video frame queues to two DLP projection headlights respectively. Through a dual-lamp fusion module, the output is synchronized pixel by pixel, and each pixel is focused on a screen at a preset distance to achieve a stereoscopic image effect.

[0199] Figure 8 This is a flowchart of an audio-video synchronization scheme according to a specific embodiment of the present invention.

[0200] Figure 9 This is an audio-video synchronization timing diagram of a specific embodiment of the present invention.

[0201] like Figure 8 , 9 As shown, the overall audio-video synchronization solution is as follows: First, the IVI (in-vehicle infotainment system) and DLP (in-vehicle DLP, DLP projection headlights) synchronize their system time, using the NTP time synchronization protocol. Using the system's master clock as a reference, the audio playback time is adjusted to achieve synchronization. Audio and video are delayed on the in-vehicle infotainment system, while video plays normally on the DLP system. The delay time is the time it takes for video to play normally from the in-vehicle infotainment system to the DLP system.

[0202] The IVI (in-vehicle infotainment system) decapsulates the read video files, separates the audio frame queue and the video frame queue, and temporarily stores the audio frame queue after decomposing it into a PCM queue.

[0203] The video frame queue is broken down into an image queue within the vehicle's infotainment system, with a clock stamp temporarily stored. Another queue undergoes RTP packet processing and is sent to the DLP (Digital Perceptron) terminal. After RTP depacketization and H.264 decoding, a video stream with the same frame content as the vehicle's infotainment system is obtained. The DLP terminal also marks the video stream with a clock stamp and returns the clock difference information to the vehicle's infotainment system. The infotainment system uses this clock difference information to control the temporary storage time of the image queue. The timestamp and clock difference information are obtained using the clock protocol under the NTP protocol. The PCM (Packet Module) queue and image queue are delayed according to the clock difference information before being played on the playback terminal. After the DLP terminal returns the clock difference information, the image is played directly on the DLP headlights, thus eliminating playback asynchrony issues.

[0204] When the DLP connection to the vehicle's infotainment system is activated, it first sends an NTP clock synchronization request, and the infotainment system and DLP establish clock synchronization. Video can be sent from the infotainment system to the DLP in queues of 10 video frames. The DLP returns single-frame time difference information, and the infotainment system processes the video and audio streams played locally in batches based on the single-frame time difference with delay.

[0205] It can refresh the single frame time difference every second, maintaining batch processing of video and audio streams with delayed playback.

[0206] Within the H.262 framework, video streams sent to the DLP end can be encoded and decoded.

[0207] Figure 10 This is a block diagram of an electronic device structure for a vehicle projection method provided in one or more embodiments of the present invention.

[0208] like Figure 10 As shown, this application provides an electronic device, including: a processor, a communication interface, a memory, and a communication bus, wherein the processor, the communication interface, and the memory communicate with each other through the communication bus;

[0209] The memory stores a computer program, which, when executed by the processor, causes the processor to perform the steps of the vehicle projection method.

[0210] This application also provides a computer-readable storage medium storing a computer program executable by an electronic device, which, when run on the electronic device, causes the electronic device to perform the steps of the vehicle projection method.

[0211] This application also provides a vehicle, specifically comprising:

[0212] Electronic equipment used to implement vehicle-mounted projection methods;

[0213] The processor runs a program, and when the program runs, it executes the steps of the vehicle projection method based on data output from the electronic device.

[0214] Storage medium for storing programs that, when running, execute the steps of the vehicle projection method based on data output from electronic devices.

[0215] The communication bus mentioned in the above electronic devices can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. This communication bus can be divided into address bus, data bus, control bus, etc. For ease of illustration, only one thick line is used to represent it in the diagram, but this does not mean that there is only one bus or one type of bus.

[0216] The electronic device comprises a hardware layer, an operating system layer running on top of the hardware layer, and an application layer running on the operating system. The hardware layer includes hardware such as a central processing unit (CPU), a memory management unit (MMU), and memory. The operating system can be any one or more computer operating systems that control the electronic device through processes, such as Linux, Unix, Android, iOS, or Windows. Furthermore, in this embodiment of the invention, the electronic device can be a smartphone, tablet computer, or other handheld device, or a desktop computer, portable computer, or other electronic device; there is no particular limitation in this embodiment.

[0217] In this embodiment of the invention, the executing entity for electronic device control can be an electronic device itself, or a functional module within an electronic device capable of calling and executing a program. The electronic device can obtain the firmware corresponding to the storage medium. This firmware is provided by the supplier, and different storage media may have the same or different firmware; no limitation is made here. After obtaining the firmware corresponding to the storage medium, the electronic device can write this firmware into the storage medium; specifically, it burns the firmware corresponding to the storage medium into the storage medium. The process of burning the firmware into the storage medium can be implemented using existing technology, and will not be elaborated upon in this embodiment of the invention.

[0218] Electronic devices can also obtain reset commands corresponding to the storage media. The reset commands corresponding to the storage media are provided by the supplier. The reset commands corresponding to different storage media can be the same or different, and no restrictions are imposed here.

[0219] At this time, the storage medium of the electronic device is a storage medium on which the corresponding firmware has been written. The electronic device can respond to the reset command corresponding to the storage medium on which the corresponding firmware has been written, thereby resetting the storage medium on which the corresponding firmware has been written according to the reset command. The process of resetting the storage medium according to the reset command can be implemented by existing technology and will not be described in detail in this embodiment of the invention.

[0220] For ease of description, the above devices are described separately by function as various units and modules. Of course, in implementing this application, the functions of each unit and module can be implemented in one or more software and / or hardware.

[0221] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the meaning consistent with their meaning in the context of the prior art, and should not be interpreted in an idealized or overly formal sense unless specifically defined.

[0222] For the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0223] As can be seen from the above description of the embodiments, those skilled in the art can clearly understand that this application can be implemented by means of software plus necessary general-purpose hardware platforms. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in various embodiments or some parts of the embodiments of this application.

[0224] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A vehicle-mounted projection method, characterized in that, The in-vehicle DLP synchronizes with the vehicle's clock via the NTP protocol, and the in-vehicle projection method includes: Get the video file to be played; The video file to be played is digitally decoded to separate the video frame queue and the audio frame queue; Mark the video frame queue and audio frame queue with the vehicle-mounted timestamp; The video frame queue is sent to the vehicle-mounted DLP via RTP packets; The video frame queue and audio frame queue are separated into an image queue and a PCM queue that correspond to each other in a frame-to-frame relationship; The image queue and PCM queue are equipped with vehicle-mounted timestamps. The image queue and PCM queue stack are stored in the vehicle's infotainment system for playback. Receive clock difference information from the video frame queue of the vehicle-mounted DLP; The clock time difference information corresponds to the image queue and PCM queue that mark the timestamp on the vehicle terminal; Based on the clock time difference information and the corresponding image queue and PCM queue timestamp information on the vehicle terminal, the delayed playback information is obtained; Based on the delayed playback information, control the storage time of the image queue and PCM queue stacks; The in-vehicle infotainment system plays the image queue and PCM queue after a stack storage delay.

2. A vehicle-mounted DLP projection method, characterized in that, Based on the vehicle-mounted projection method according to claim 1, the vehicle-mounted DLP projection method includes: The in-vehicle DLP receives the video frame queue from the vehicle's infotainment system via RTP packets; After RTP unpacking, the video frame queue is obtained through decoding and the DLP end timestamp is marked. Based on the timestamps of the DLP end and the vehicle terminal, obtain the clock time difference information of the same video frame queue before the vehicle terminal passes through the RTP packet and after the vehicle-mounted DLP decodes it; After sending the clock difference information of the video queue whose timestamps correspond to those of the vehicle-mounted system, the vehicle-mounted DLP will play the video queue in real time on the playback terminal of the vehicle-mounted DLP. The clock time difference information includes timestamp information.

3. A vehicle-mounted DLP projection system, characterized in that, The vehicle-mounted DLP projection system includes: a vehicle-mounted unit and a vehicle-mounted DLP; The vehicle infotainment system includes an underlying service module; The underlying service modules include an NTP server module and an RTP server module; The NTP server module is used as the NTP server on the vehicle-mounted system side to synchronize the vehicle-mounted system with the vehicle-mounted DLP clock. The RTP server module is used to send the video file to be played from the vehicle's infotainment system to the in-vehicle DLP terminal. Based on the clock synchronization between the vehicle's infotainment system and the in-vehicle DLP by the NTP server module, the same video file to be played is marked with a timestamp on the vehicle's infotainment system. The in-vehicle DLP marks the video file to be played with a DLP-side timestamp. The vehicle-mounted DLP generates clock time difference information based on the timestamp on the vehicle-mounted device and the timestamp on the DLP, which are marked on the video file to be played on the vehicle-mounted DLP, and sends it to the vehicle-mounted device. The vehicle's infotainment system receives clock time difference information from the in-vehicle DLP. The vehicle system delays the playback of the video file to be played on the vehicle system playback terminal based on the clock time difference information.

4. The vehicle-mounted DLP projection system according to claim 3, characterized in that, The vehicle infotainment system also includes: a user service module; The user service module includes a local player module and a third-party player module, which are used to manage video files to be played. The underlying service module also includes a remote playback service module; The remote playback service module is used to store the same video file to be played in the vehicle's infotainment system and send it to the in-vehicle DLP, respectively. The local player module or the third-party player module sends a request to the remote playback service module to play the video file to be played. The remote playback service module marks the video file to be played with a timestamp on the vehicle terminal according to the request to play the video file sent by the local player module or the third-party player module, and stores it in the vehicle terminal and sends it to the vehicle DLP. The remote playback service module sends the video file to be played, which has a timestamp from the vehicle's infotainment system, to the in-vehicle DLP via the RTP server module.

5. The vehicle-mounted DLP projection system according to claim 4, characterized in that, The underlying service module also includes: a multimedia extraction module and an audio decoding module; The multimedia extraction module is used to extract the video files to be played into an image queue before storing the video file stack; The audio decoding module is used to strip the video file to be played into a PCM queue before storing the video file stack; The image queue and the PCM queue are marked with the vehicle-mounted timestamp of the video file to be played, and are stored in the vehicle-mounted stack before playback.

6. The vehicle-mounted DLP projection system according to claim 5, characterized in that, The audio decoding module further includes: an audio encoding / decoding module and an audio tracking module; The audio encoding / decoding module is used to process the video file to be played and obtain the PCM queue; The audio tracking module is used to control the stack storage time of the PCM queue according to the clock time difference information, and to play the PCM queue from the vehicle audio playback terminal after the stack storage delay.

7. A method for synchronizing vehicle-mounted DLP projection, characterized in that, include: After the vehicle-mounted DLP is woken up or started, it sends an NTP clock synchronization request to the vehicle's infotainment system. The in-vehicle DLP receives a queue of video frames from the vehicle's infotainment system and marks the DLP's timestamp. Based on the timestamps from the DLP terminal and the vehicle terminal, obtain the clock difference information for the same video frame queue; The vehicle-mounted DLP obtains the average value of the latency of a single video frame in the video frame queue based on the latency from the vehicle's head unit to the vehicle-mounted DLP head unit, and returns it to the vehicle's head unit. The in-vehicle DLP batch processes and plays video frame queues from the vehicle's infotainment system; The vehicle-mounted DLP periodically selects a group of video frames with vehicle-mounted timestamps at preset time intervals to obtain the average video latency of a single frame in the video frame queue.

8. A method for synchronizing vehicle-mounted system projection, characterized in that, include: The vehicle-mounted system receives an NTP clock synchronization request from the vehicle-mounted DLP and aligns its clock with the vehicle-mounted DLP according to the NTP clock synchronization request. The vehicle-mounted system sends a queue of video frames to the in-vehicle DLP. The vehicle-mounted unit receives the average delay of a single video frame from the video frame queue returned by the vehicle-mounted DLP, and refreshes the record of clock difference information. The vehicle-mounted system sends a batch of video frame queues to the in-vehicle DLP. The vehicle's infotainment system delays the playback of video frame queues locally based on clock time difference information; The vehicle-mounted system periodically selects a set of video frame queues, marks the timestamps on the vehicle-mounted system, and sends them to the in-vehicle DLP at preset time intervals. The vehicle system periodically refreshes its clock time difference records based on the acquired clock time difference information.

9. A vehicle, characterized in that, include: In-vehicle infotainment system, central control screen, instrument panel, DLP projection headlights, microphone, digital amplifier system; The digital amplifier system includes multiple speakers arranged in the car cabin; The vehicle infotainment system is connected to the central control screen, instrument panel, and DLP projection headlights, and synchronized with the NTP clock. The vehicle-mounted system is connected to the DLP projection headlight via Ethernet, and transmits a video file to be played that is marked with a timestamp from the vehicle-mounted system to the DLP projection headlight. The DLP projection lamp receives the timestamp of the video file to be played from the DLP end. The clock difference information of the video file to be played under NTP clock synchronization is obtained based on the timestamps of the vehicle-mounted system and the DLP-mounted system. The DLP projection headlight transmits the clock time difference information of the video file to be played back to the vehicle's infotainment system. The vehicle system delays the playback of the video file to be played on the central control screen, instrument panel, and digital amplifier system based on the clock time difference information of the video file to be played. The microphone is used to record sound, which is then played back in real time by the digital amplifier system.

10. The vehicle according to claim 9, characterized in that, The DLP projection headlight includes a deserializer and a dual-lamp fusion module. The deserializer is used to process the video file to be played into multiple synchronized video signals output by DLP projector lamps. The dual-lamp fusion module is used to control multiple DLP projection lamps to focus the same video pixels onto a screen at a preset distance. The deserializer processes the video file to be played and outputs it to the dual-lamp fusion module, which controls multiple DLP projection lamps to output video. The dual-lamp fusion module controls multiple DLP projection lamps to focus the video image pixels onto the screen at a preset distance.

Images