Vehicle machine screen projection method, device and system, medium and vehicle

Through the combination of USB data acquisition card and libuvc library functions, the problem of unsmooth audio and video signal transmission in the car computer screen projection is solved, and the effect of low latency, high fluency and audio and video synchronization is achieved, which improves the overall screen projection of the car computer screen performance.

CN118945403APending Publication Date: 2024-11-12CHONGQING CHANGAN AUTOMOBILE CO LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202411043573.2
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-12

AI Technical Summary

Technical Problem

In the vehicle screen projection technology, the audio and video signals transmitted by the host device to the vehicle are not smooth, resulting in screen delay and synchronization problems.

Method used

Use USB data acquisition card to collect audio and video signals of the host device and pre-process them in the vehicle computer, including decoding, recoding, compression and synchronization processing, dynamically adjust the frame rate to ensure the stability of the data frame number, and use the libuvc library function for detailed details Granular control ensures low latency and high fluency of audio and video data.

Benefits of technology

It realizes low latency and high fluency in the projection of the car machine, ensures the synchronization effect of audio and video data, reduces the difficulty of projection and improves the application range.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118945403A_ABST
    Figure CN118945403A_ABST
Patent Text Reader

Abstract

The invention relates to the technical field of vehicle multimedia, and discloses a vehicle machine screen projection method, device and system, a medium and a vehicle. The method comprises the steps that collected video data and collected audio data sent by a USB data collection card are received, and the USB data collection card is used for collecting original video data and original audio data of host equipment; preprocessing the original video data and the original audio data to respectively obtain collected video data and collected audio data; carrying out fluency improvement processing on the collected video data; and playing the collected audio data and displaying the processed collected video data. According to the invention, the picture fluency of screen projection from the host equipment to the vehicle machine is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of vehicle multimedia technology, and in particular to a vehicle-machine screen projection method, device, system, medium and vehicle. Background Art

[0002] In recent years, with the improvement of car computer hardware and software conditions, the emergence of various ecological applications in car computers has become an inevitable trend. These ecological applications not only enrich the driving and riding experience of car users, but also promote the intelligent development and innovation of the automotive industry. In the future, with the continuous advancement of technology and the continuous expansion of the market, car computer ecological applications will be more diversified and personalized, bringing car users a more colorful driving and riding experience. Applying screen projection technology to cars is a new study on car computer multimedia applications. From the perspective of in-car games, games do play an indispensable role in in-car ecological entertainment, and the screen projection technology of game consoles will also bring new possibilities and experiences to this field. It is a valuable study to smoothly project the audio and video data of game consoles and other host devices onto the car screen. Summary of the invention

[0003] In view of this, the present invention provides a vehicle-mounted screen projection method, device, system, medium and vehicle to solve the problem of unsmooth screen projection from a host device to a vehicle-mounted screen.

[0004] In a first aspect, the present invention provides a method for projecting a screen on a vehicle computer, which is applied to a vehicle computer, and the method includes: receiving collected video data and collected audio data sent by a USB data acquisition card, the USB data acquisition card being used to collect original video data and original audio data of a host device, and pre-processing the original video data and the original audio data to obtain collected video data and collected audio data, respectively; processing the collected video data to improve fluency; playing the collected audio data and displaying the processed collected video data.

[0005] According to the above technical means, the present invention collects the audio and video signals of the host device through the USB data acquisition card and transmits them to the vehicle computer. The USB data acquisition card technology is mature and easy to implement, which reduces the difficulty of transmitting the audio and video signals of the host device to the vehicle computer. At the same time, before the vehicle computer projects the collected video data and the collected audio data, the data is also processed to improve the fluency, thereby ensuring that the optimized display screen and audio have the advantages of low latency and high fluency.

[0006] In some optional implementations, the collected video data is processed to improve fluency, including: reading the latest frame of data each time the display framework interface reads the collected video data from the user cache; the display framework interface is a service called by the display driver for reading the collected video data, the display driver is used to display the read collected video data, and the user cache is a space in the vehicle computer for storing the collected video data sent by the USB data acquisition card; initializing the maximum frame rate and the write frame rate; dynamically adjusting the write frame rate without exceeding the maximum frame rate, so that when the collected video data of the USB data acquisition card is written to the user cache according to the write frame rate, the number of collected video data frames in the user cache remains stable.

[0007] According to the above technical means, the display framework interface is configured to read the latest frame of video data from the user cache each time, thereby ensuring that the displayed frame number is the latest and avoiding the high latency problem caused by the lack of synchronization between the screen and the host device screen. In addition, in order to reduce the latency while ensuring that no frames are lost, the number of data frames in the user cache is kept stable by dynamically adjusting the write frame rate, so that the data writing and reading in the user cache are maintained at a uniform and stable state, avoiding large differences in the data screen each time the latest frame of video data is read, and improving the smoothness of the screen projection.

[0008] In some optional implementations, the method also includes: stabilizing the number of frames of captured video data in the user cache to 1 frame all the time by dynamically adjusting the write frame rate; maintaining a first speed at which the audio framework interface reads the captured audio data from the user cache each time, and maintaining a second speed at which the captured audio data of the USB data acquisition card is written into the user cache.

[0009] According to the above technical means, the number of data frames in the user cache is stabilized at 1 frame all the time, and the transmission speed of audio data is maintained stable, which can make the video delay around 120-180ms, and the audio delay is also within tens of milliseconds, so that the delay difference of audio and video data is basically stabilized at around 100ms. For the user's senses, the effect of audio and video synchronization can be achieved, and the synchronization of audio data and video data after screen projection is improved.

[0010] In some optional implementations, receiving collected video data and collected audio data sent by a USB data acquisition card includes: writing the collected video data into a user cache through a libuvc library function; and writing the collected audio data into the user cache through a libusb library function.

[0011] According to the above-mentioned technical means, on the basis of the libusb protocol, the libuvc library function is used to write the collected video data into the user cache. The libuvc library function supports fine-grained control of USB video devices that export standard USB video class interfaces, enabling developers to write screen projection drivers for more different car models, thereby improving the application scope of the screen projection solution provided by the present invention.

[0012] In some optional implementations, the collected audio data is played and the processed collected video data is displayed, including: reading the collected video data from the user cache to the display driver through the display framework interface; enabling the vehicle display screen to display the read collected video data through the display driver; reading the collected audio data from the user cache to the audio driver through the audio framework interface; enabling the vehicle speaker to play the read collected audio data through the audio driver.

[0013] In some optional implementations, the step of preprocessing the original video data and the original audio data through the USB data acquisition card includes: decoding, re-encoding, compressing and synchronizing the original video data and the original audio data through the USB data acquisition card.

[0014] According to the above-mentioned technical means, the present invention also decodes, re-encodes, compresses and synchronizes the collected original video data and original audio data through a USB data acquisition card. On the one hand, it matches the data format of the host device and the data format of the vehicle computer. On the other hand, it reduces the space occupied by the collected data and improves the data transmission speed. In addition, the audio and video are preliminarily synchronized to reduce the problem of sound and picture being out of sync after screen projection.

[0015] In the second aspect, the present invention provides a car screen projection device, which is applied to the car computer, and the device includes: a data receiving module, which is used to receive collected video data and collected audio data sent by a USB data acquisition card, the USB data acquisition card is used to collect original video data and original audio data of the host device, and pre-process the original video data and original audio data to obtain collected video data and collected audio data respectively; a smoothness processing module, which is used to improve the smoothness of the collected video data; a screen projection module, which is used to play the collected audio data and display the processed collected video data.

[0016] In a third aspect, the present invention provides a vehicle screen projection system, comprising: a host device, a USB data acquisition card and a vehicle, wherein the audio and video interface of the host device is connected to the USB data acquisition card, and the USB data acquisition card is communicatively connected to the vehicle.

[0017] In a fourth aspect, the present invention provides a computer-readable storage medium having computer instructions stored thereon, the computer instructions being used to cause a computer to execute the method of the first aspect or any corresponding embodiment thereof.

[0018] In a fifth aspect, the present invention provides a vehicle, comprising a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions in the processor, thereby executing the method of the above-mentioned first aspect or any corresponding embodiment thereof.

[0019] The technical solution provided by the present invention has the following advantages:

[0020] (1) According to the above technical means, the present invention collects the audio and video signals of the host device through a USB data acquisition card and transmits them to the vehicle computer. The USB data acquisition card technology is mature and easy to implement, which reduces the difficulty of transmitting the audio and video signals of the host device to the vehicle computer. At the same time, before the vehicle computer projects the collected video data and the collected audio data, the data is also processed to improve the fluency, thereby ensuring that the optimized display screen and audio have the advantages of low latency and high fluency.

[0021] (2) According to the above technical means, the display framework interface is configured to read the latest frame of video data from the user cache each time, thereby ensuring that the displayed frame number is the latest and avoiding the high latency problem caused by the asynchrony between the image and the host device image. In addition, in order to reduce the latency while ensuring that no frames are lost, the number of data frames in the user cache is kept stable by dynamically adjusting the write frame rate, so that the data writing and reading in the user cache are maintained at a uniform and stable state, avoiding a large difference in the data image each time the latest frame of video data is read, thereby improving the smoothness of the screen projection.

[0022] (3) According to the above technical means, the number of data frames in the user cache is stabilized at 1 frame all the time, and the transmission speed of the audio data is maintained stable, which can make the video delay about 120-180ms, and the audio delay is also within tens of milliseconds, so that the delay difference of audio and video data is basically stabilized at about 100ms. For the user's senses, the effect of audio and video synchronization can be achieved, and the synchronization of audio data and video data after screen projection is improved.

[0023] (4) According to the above technical means, on the basis of the libusb protocol, the libuvc library function is used to write the collected video data into the user cache. The libuvc library function supports fine-grained control of USB video devices that export standard USB video class interfaces, enabling developers to write screen projection drivers for more different car models, thereby improving the application scope of the screen projection solution provided by the present invention.

[0024] (5) According to the above technical means, the present invention also decodes, re-encodes, compresses and synchronizes the collected original video data and original audio data through a USB data acquisition card. On the one hand, it matches the data format of the host device and the data format of the vehicle computer. On the other hand, it reduces the space occupied by the collected data and improves the data transmission speed. In addition, the audio and video are preliminarily synchronized to reduce the problem of sound and picture being out of sync after screen projection. BRIEF DESCRIPTION OF THE DRAWINGS

[0025] In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0026] Figure 1 It is a structural schematic diagram of a vehicle screen projection system according to an embodiment of the present invention;

[0027] Figure 2 is a schematic diagram of a program architecture of a vehicle screen projection system according to an embodiment of the present invention;

[0028] Figure 3 It is a flowchart of a method for projecting screen on a vehicle according to an embodiment of the present invention;

[0029] Figure 4 is another schematic diagram of a program architecture of a vehicle screen projection system according to an embodiment of the present invention;

[0030] Figure 5 It is a structural schematic diagram of a vehicle screen projection device according to an embodiment of the present invention;

[0031] Figure 6 is a schematic structural diagram of a vehicle according to an embodiment of the present invention. DETAILED DESCRIPTION

[0032] In order to make the purpose, technical solution and advantages of the embodiments of the present invention clearer, the technical solution in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0033] Related technologies often use the hardware design of a multi-function base and a little software design to project the screen of a game console device to a display. The display needs to support the data format corresponding to the multi-function base in order to project the screen. It is also necessary to implement a car computer application as a carrier to convert the data of the multi-function base and give it to the car computer system.

[0034] According to an embodiment of the present invention, a vehicle screen projection system is provided. Figure 1 As shown, the system includes: a host device, a USB data acquisition card and a vehicle computer, the audio and video interface of the host device is connected to the USB data acquisition card, and the USB data acquisition card is communicatively connected to the vehicle computer.

[0035] Specifically, the technical solution provided by the present invention connects the host device, the USB data acquisition card and the vehicle computer through a wired manner in hardware, or connects the host device to the vehicle computer with a built-in USB data acquisition card. In this embodiment, the host device includes at least one of a handheld game console, a game console, a personal computer, a mobile phone, and a tablet. In terms of software, the USB data acquisition card captures the audio and video data of the host device in real time, and the application of the vehicle computer captures the audio and video data of the USB data acquisition card in real time, and presents the audio and video data in real time on the vehicle in a manner supported by the vehicle computer. The data acquisition of the host by the USB data acquisition card is a known functional design. Since the vehicle computer itself does not directly support audio and video signals such as the Type-C interface of the host device, this means that additional hardware, namely the USB data acquisition card, is required to realize the transmission and display of audio and video signals. The USB data acquisition card plays a role of conversion and adaptation. It receives the digital signal output by the host interface, and then converts these signals into a format and encoding that meets the requirements of the target device through internal hardware and software processing. The above conversion process at least includes signal decoding, re-encoding, compression, synchronization and other processing steps. The necessity of conversion lies in that different devices may use different audio and video codec standards, resolutions, frame rates, color spaces and other parameters. If the signal output from the host interface is directly transmitted to the target device, it may not be played or displayed normally due to format mismatch or incompatible encoding. However, conversion through the USB data acquisition card can ensure the compatibility of the audio and video signals of the host device with the vehicle computer, and realize cross-platform transmission and playback of audio and video content. As for the specific implementation method of the conversion, this is usually achieved through the hardware codec, processor and driver inside the USB data acquisition card. The USB data acquisition card will decode, re-encode and synchronize the output signal of the host device according to the requirements of the vehicle computer, and then transmit the converted signal to the vehicle computer through the USB interface. The process of the USB data acquisition card capturing the audio and video data of the host device is an automatic process based on interface specifications and hardware conversion. The user only needs to ensure that the host device is correctly connected to the USB data acquisition card and install and configure the corresponding software on the vehicle computer. At the same time, the process of USB driver capturing the audio and video data of the host device is also an automatic process based on interface specifications and hardware conversion. Users only need to ensure that the USB data acquisition card is correctly connected to the vehicle computer and wait for the USB driver to load successfully. Then, they can configure and control the USB data acquisition card through the corresponding software, thereby ultimately achieving the purpose of capturing the audio and video data of the host device.

[0036] When the USB data acquisition card is built into the vehicle computer, the vehicle computer is directly connected to the host device, and the built-in audio and video processing hardware modules are connected to each other through the internal bus or dedicated interface of the vehicle computer to form a complete audio and video processing system. The software architecture of the system is as follows: Figure 2As shown. The car computer can be directly connected to host devices such as Switch and Steam. The underlying kernel driver of the car computer receives audio and video data through USB or other interfaces, and decodes, optimizes, and finally provides audio and video capture services for upper-level applications. For the specific implementation details of the data capture of the above-mentioned USB data acquisition card, reference can be made to the prior art, and the present invention will not go into details. The car computer screen projection system provided by the embodiment of the present invention does not require the additional design of a special multi-functional base. It uses a mature USB data acquisition card and configures the car computer with an application that can communicate with the USB data acquisition card to realize data collection and transmission, thereby realizing the projection of the car computer, and providing a lower-cost and easier-to-implement hardware solution for the projection of the host device such as the game console to the car computer.

[0037] According to an embodiment of the present invention, an embodiment of a method for projecting screen on a vehicle is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0038] In this embodiment, a vehicle screen projection method is provided, which can be used in the above-mentioned vehicle screen projection system. Figure 3 The flowchart of a method for projecting screen on a vehicle according to an embodiment of the present invention is specifically the execution steps of the vehicle. The process includes the following steps:

[0039] Step S301, receiving collected video data and collected audio data sent by a USB data acquisition card, the USB data acquisition card is used to collect original video data and original audio data of a host device, and pre-process the original video data and original audio data to obtain collected video data and collected audio data respectively;

[0040] Step S302, performing a process to improve the fluency of the collected video data;

[0041] Step S303, playing the collected audio data and displaying the processed collected video data.

[0042] Specifically, the vehicle screen projection method provided by the embodiment of the present invention first captures the original video data and original audio data of the host device such as the game console through a USB data acquisition card, and then pre-processes the original video data and original audio data through the USB data acquisition card. In an optional implementation, the pre-processing mainly includes decoding, re-encoding, compressing and synchronizing the original video data and the original audio data through the USB data acquisition card. On the one hand, the data format of the host device and the vehicle data format are matched, and on the other hand, the space occupied by the collected data is reduced, and the data transmission speed is improved. In addition, the audio and video tracks are preliminarily synchronized to reduce the problem of asynchrony between the sound and the picture after the screen is projected.

[0043] Afterwards, if Figure 4 As shown, taking an external USB data acquisition card as an example, the USB data acquisition card sends the collected video data and collected audio data obtained by preprocessing to the vehicle computer, and the vehicle computer captures the collected video data and collected audio data of the USB data acquisition card through the USB driver. Although the collected data can be played normally, it cannot be displayed with low latency and high fluency. Based on this, the embodiment of the present invention optimizes the collected data.

[0044] In this embodiment, the vehicle computer mainly processes the collected video data to improve the fluency and reduce the occurrence of screen tearing, jumps, delays, etc. Since the audio data is small in size and has a fast transmission speed, there is often no problem of data missing or data errors, so no special optimization processing is required. Figure 4 The audio driver plays the collected audio data, and the display driver synchronously displays the processed collected video data. The present invention collects the audio and video signals of the host device through the USB data acquisition card and transmits them to the vehicle computer. The USB data acquisition card technology is mature and easy to implement, which reduces the difficulty of transmitting the audio and video signals of the host device to the vehicle computer. At the same time, before the vehicle computer projects the collected video data and the collected audio data, the data is also processed to improve the fluency, thereby ensuring that the optimized display screen and audio have the advantages of low latency and high fluency.

[0045] In this embodiment, the video data transmission speed can be increased by reducing the video clarity, thereby reducing the video delay. The video delay can also be reduced by controlling the sampling frequency to downsample the collected video data. Alternatively, the above two processing schemes can be dynamically adjusted to achieve a balance between the video delay and clarity, thereby improving the video smoothness and reducing the delay while maintaining the video clarity as much as possible.

[0046] In some optional implementations, the above step S302 includes:

[0047] Step a1, when the display framework interface reads the collected video data from the user cache each time, read the latest frame of data; the display framework interface is a service called by the display driver for reading the collected video data, and the display driver is used to display the read collected video data. The user cache is a space in the vehicle computer for storing the collected video data sent by the USB data acquisition card;

[0048] Step a2, initializing the maximum frame rate and writing frame rate;

[0049] Step a3, dynamically adjusting the write frame rate without exceeding the maximum frame rate, so that when the collected video data of the USB data acquisition card is written into the user cache according to the write frame rate, the number of collected video data frames in the user cache remains stable.

[0050] Specifically, the data transmission scheme provided by the present invention is to write the collected video data and collected audio data sent by the USB data acquisition card into the user cache. For example, the collected video data can be written to the USBCamera position, and the collected audio data can be written to the USBAudio position. The vehicle computer takes out the collected video data from the user cache through the display framework interface, and displays the collected video data on the screen through the display driver. Similarly, the vehicle computer takes out the collected audio data from the user cache through the audio framework interface, and enables the speaker to play the collected audio data through the audio driver. Among them, the display framework interface is a part of the display driver's service, which is used to read the collected video data, and the audio framework interface is the same. Based on this, the embodiment of the present invention optimizes this data transmission process, thereby achieving improved fluency processing of the collected video data.

[0051] In particular, for the display framework interface and the audio framework interface, whether it is the Android system, the Hongmeng system or any other car system, these two interfaces are usually provided for the upper-level applications. The only difference is that the rules or methods of use are slightly different, but there is no difference in essence, that is, they are known conditions, and their specific implementation logic can refer to the existing technology, and this embodiment will not be repeated.

[0052] In this embodiment, the user cache is a kernel cache space of a specified size in the vehicle computer. One or more frames of acquired video data can be stored in the user cache. The acquired video data is sent to the user cache frame by frame in the form of a data stream, and the display framework interface also retrieves data from the user cache in the form of a data stream. Since the speed at which the display framework interface retrieves data may not be the same as the speed at which the USB driver writes data to the user cache, it may result in more than one frame of data being written to the user cache. If the display framework interface also retrieves data from the user cache in the order in which it is written, over time it may cause a more serious delay between the vehicle computer projection screen and the original screen of the host device. Therefore, the embodiment of the present invention defines that each time the display framework interface reads the acquired video data from the user cache, it reads the latest frame of data to ensure that the time difference between the screen read by the display framework interface and the screen sent by the host device is minimized, thereby significantly reducing the screen delay between the host device and the vehicle computer.

[0053] Furthermore, if the number of frames of the collected video data stored in the user cache is unstable, sometimes more and sometimes less, the interval between frames will be unstable each time the display framework interface reads the latest frame of data, resulting in poor picture smoothness. The embodiment of the present invention initializes the maximum frame rate and the write frame rate, wherein the maximum frame rate is used to limit the maximum speed of writing the collected video data into the user cache to avoid data accumulation in the user cache and reduce the generation of delays; wherein the write frame rate is a real-time dynamic change used to limit the real-time speed of writing the collected video data into the user cache. The embodiment of the present invention dynamically adjusts the write frame rate without exceeding the maximum frame rate to find the best balance point, and when the collected video data of the USB data acquisition card is written into the user cache according to the write frame rate, the number of data frames in the user cache reaches a stable state.

[0054] Through the above two types of optimization processing, the delay problem of the projection screen can be significantly reduced and the smoothness of the picture can be significantly improved, based on the user's better viewing experience and gaming experience.

[0055] It should be noted that the audio framework interface used will pass the collected audio data to the audio driver, and then play it through the speaker. This process is similar to the processing method of collecting video data. The audio processing flow also implements two threads. One thread is the USB driver, which writes the collected audio data to the user cache, and the other thread is the audio framework interface, which reads the collected audio data to the audio driver. Finally, the audio driver gives the data to the speaker or horn and plays it out. Since the amount of collected audio data is much smaller than the amount of video data, and the biggest impact on delay and fluency is data acquisition and data copying, there is basically no delay and fluency problem under the hardware conditions of the current environment when the data amount is small, so there is no need to optimize delay and fluency like video data.

[0056] In some optional implementation manners, an in-vehicle screen projection method provided by an embodiment of the present invention further includes:

[0057] Step b1, by dynamically adjusting the writing frame rate, the number of frames of the collected video data in the user cache is stabilized at 1 frame all the time;

[0058] Step b2, maintaining a first speed at which the audio framework interface reads collected audio data from the user cache each time, and maintaining a second speed at which the collected audio data of the USB data acquisition card is written into the user cache.

[0059] Specifically, in actual screen projection applications, there may be a problem of the sound and picture being out of sync, that is, the sound is played faster than the picture, or the sound is played slower than the picture. In order to solve this problem, in addition to synchronizing the two through a USB data acquisition card, the embodiment of the present invention further limits the dynamic adjustment result of the above step a3, stabilizing the number of data frames in the user cache at 1 frame all the time. At the same time, the data transmission speed of the collected audio data needs to be kept stable, thereby defining the first speed of the audio framework interface to read the collected audio data from the user cache each time to maintain stability, and the second speed of writing the collected audio data of the USB data acquisition card to the user cache to maintain stability.

[0060] The purpose of this is to maintain the number of frames of collected video data in the user cache at 1 frame, so that the display framework interface only takes out the latest frame of collected video data each time, and stabilizes the delay of the video picture at about 120-180ms. At the same time, the audio delay is usually within tens of milliseconds. It is only necessary to ensure the stability of the audio delay by fixing the first speed and the second speed to ensure the stability of the delay of the video relative to the audio. According to the above configuration, the time difference between the collected video data and the collected audio data is basically stabilized at about 100ms, and the user's naked eye is difficult to feel the delay of about 100ms, so the effect of audio and video synchronization can be achieved naturally.

[0061] In some optional implementations, the above step S301 includes:

[0062] Step c1, writing the collected video data into the user cache through the libuvc library function;

[0063] Step c2, write the collected audio data into the user buffer through the libusb library function.

[0064] Specifically, the libusb library is an open source, C-based USB driver library for applications to communicate with users' USB devices. It allows users to write USB-based applications without understanding the underlying USB protocol. It is portable, uses a unified API externally, and supports multiple operating systems such as Windows, macOS, Linux, and Android. It can be applied in user mode, and applications do not need the highest permissions to communicate with USB devices. It supports all versions of USB protocols. The libuvc library function is a cross-platform library for USB video devices. It is built on libusb. libuvc supports fine-grained control of USB video devices that export standard USB video class interfaces. It can be said that libuvc is a more advanced library that depends on libusb. It uses the underlying functions of libusb to provide support for USB video devices, that is, applications ultimately obtain and control video data through the function interface provided by libuvc. Because libuvc supports fine-grained control of USB video devices that export standard USB video class (UVC) interfaces, it enables developers to write drivers for previously unsupported devices or simply access UVC devices in a generic way. For example, when two cameras of the same model from the same manufacturer are inserted into the Linux system, the device information of the two cameras obtained through the libusb command is the same and cannot be distinguished. At this time, more detailed information of the camera device can be obtained through libuvc, and from this information, it can be distinguished from which camera the video data is obtained. According to this solution, developers can write screen projection drivers for more different car models, which improves the application scope of the screen projection solution provided by the present invention.

[0065] Through the application of the above library functions, such as Figure 4 As shown, the application in the car computer implements thread 1 in the usbcamera function library, and cyclically calls the function interface provided by libuvc according to the given write frame rate to obtain the collected video data of the USB data acquisition card, and then stores the data in the user cache. At the same time, the usbcamera function library implements thread 2, cyclically calls the display framework interface to take out the collected video data in the user cache, and gives it to the display driver. Finally, the display driver gives the data to the screen and displays it. There are fewer restrictions on the collected audio data. Therefore, in the embodiment of the present invention, the application of the car computer implements calling the function interface provided by libusb in the usbaudio function library to obtain the collected audio data, and the audio framework interface provided by the car computer system for the upper application is used to finally play the data on the speaker or horn. The overall process is similar to the transmission of collected video data, and this embodiment will not be repeated.

[0066] In some optional implementations, the above step S303 includes:

[0067] Step d1, reading the collected video data from the user buffer to the display driver through the display framework interface;

[0068] Step d2, enabling the vehicle display screen to display the read collected video data through the display driver;

[0069] Step d3, reading the collected audio data from the user buffer to the audio driver through the audio framework interface;

[0070] Step d4, enabling the vehicle speaker to play the read collected audio data through the audio driver.

[0071] Specifically, the concepts, principles and usage methods of the display framework interface, display driver, audio framework interface and audio driver in this embodiment have been introduced in the previous embodiments. For explanations of the specific steps, please refer to the relevant descriptions of the previous embodiments and will not be repeated here.

[0072] In this embodiment, a car screen projection device is also provided, which is used to implement the above-mentioned embodiments and preferred implementation modes, and will not be repeated here. As used below, the term "module" can be a combination of software and / or hardware that implements a predetermined function. Although the devices described in the following embodiments are preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceivable.

[0073] This embodiment provides a vehicle screen projection device, such as Figure 5 As shown, including:

[0074] The data receiving module 501 is used to receive the collected video data and the collected audio data sent by the USB data acquisition card, the USB data acquisition card is used to collect the original video data and the original audio data of the host device, and pre-process the original video data and the original audio data to obtain the collected video data and the collected audio data respectively;

[0075] A fluency processing module 502 is used to improve the fluency of the collected video data;

[0076] The screen projection module 503 is used to play the collected audio data and display the processed collected video data.

[0077] In some optional implementations, the smooth processing module 502 includes:

[0078] The latest frame definition unit is used to read the latest frame of data each time the display framework interface reads the collected video data from the user cache; the display framework interface is a service called by the display driver to read the collected video data, and the display driver is used to display the read collected video data. The user cache is a space in the vehicle computer for storing the collected video data sent by the USB data acquisition card;

[0079] Frame rate initialization unit, used to initialize the maximum frame rate and write frame rate;

[0080] The data writing unit is used to dynamically adjust the writing frame rate without exceeding the maximum frame rate, so that when the collected video data of the USB data acquisition card is written into the user cache according to the writing frame rate, the number of collected video data frames in the user cache remains stable.

[0081] In some optional implementations, the data receiving module 501 includes:

[0082] A first writing unit is used to write the collected video data into the user cache through the libuvc library function;

[0083] The second writing unit is used to write the collected audio data into the user buffer through the libusb library function.

[0084] In some optional implementations, the screen projection module 503 includes:

[0085] A video data reading unit, used to read the collected video data from the user buffer to the display driver through the display framework interface;

[0086] A display unit, used to enable the vehicle display screen to display the read collected video data through a display driver;

[0087] An audio data reading unit, used to read the collected audio data from the user buffer to the audio driver through the audio framework interface;

[0088] The playback unit is used to enable the vehicle speaker to play the read collected audio data through the audio driver.

[0089] In some optional implementations, preprocessing the raw video data and the raw audio data by the USB data acquisition card includes:

[0090] The pre-processing unit is used for decoding, re-encoding, compressing and synchronously processing the original video data and the original audio data through the USB data acquisition card.

[0091] The further functional description of each of the above modules and units is the same as that of the above corresponding embodiments and will not be repeated here.

[0092] The car screen projection device in this embodiment is presented in the form of a functional unit, where the unit refers to an ASIC (Application Specific Integrated Circuit) circuit, a processor and memory that executes one or more software or fixed programs, and / or other devices that can provide the above functions.

[0093] The embodiment of the present invention also provides a computer-readable storage medium. The method according to the embodiment of the present invention can be implemented in hardware, firmware, or can be implemented as a computer code that can be recorded in a storage medium, or can be implemented as a computer code that is originally stored in a remote storage medium or a non-temporary machine-readable storage medium and will be stored in a local storage medium through a network download, so that the method described herein can be stored in such software processing on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. Among them, the storage medium can be a magnetic disk, an optical disk, a read-only storage memory, a random access memory, a flash memory, a hard disk or a solid-state hard disk, etc.; further, the storage medium can also include a combination of the above types of memories. It can be understood that a computer, a processor, a microprocessor controller, or programmable hardware includes a storage component that can store or receive software or computer code. When the software or computer code is accessed and executed by a computer, a processor, or hardware, the method shown in the above embodiment is implemented.

[0094] The embodiment of the present invention further provides a vehicle having the above Figure 5 The vehicle screen projection device shown in FIG. Figure 6 As shown, the vehicle includes: one or more processors 10, a memory 20, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are connected to each other using different buses for communication, and can be installed on a common mainboard or installed in other ways as needed. The processor can process instructions executed in the vehicle, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In some optional embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple vehicles can be connected, and each device provides some necessary operations (for example, as a server array, a group of blade servers, or a multi-processor system). Figure 6 A processor 10 is taken as an example.

[0095] The processor 10 may be a central processing unit, a network processor or a combination thereof. The processor 10 may further include a hardware chip. The hardware chip may be a dedicated integrated circuit, a programmable logic device or a combination thereof. The programmable logic device may be a complex programmable logic device, a field programmable gate array, a general purpose array logic or any combination thereof.

[0096] The memory 20 stores instructions executable by at least one processor 10, so that at least one processor 10 executes the method shown in the above embodiment.

[0097] The memory 20 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and applications required for at least one function; the data storage area may store data created according to the use of the vehicle, etc. In addition, the memory 20 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some optional embodiments, the memory 20 may optionally include a memory remotely arranged relative to the processor 10, and these remote memories may be connected to the vehicle via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0098] The memory 20 may include a volatile memory, such as a random access memory; the memory may also include a non-volatile memory, such as a flash memory, a hard disk or a solid state drive; the memory 20 may also include a combination of the above types of memory.

[0099] The vehicle further includes a communication interface 30 for the vehicle to communicate with other devices or a communication network.

[0100] Although the embodiments of the present invention have been described in conjunction with the accompanying drawings, those skilled in the art may make various modifications and variations without departing from the spirit and scope of the present invention, and such modifications and variations are all within the scope defined by the appended claims.

Claims

1. A method for projecting screen on a vehicle, characterized in that: Applied to a vehicle computer, the method includes: Receiving collected video data and collected audio data sent by a USB data acquisition card, wherein the USB data acquisition card is used to collect original video data and original audio data of a host device, and preprocessing the original video data and the original audio data to obtain the collected video data and the collected audio data respectively; Performing a process of improving the fluency of the collected video data; The collected audio data is played and the processed collected video data is displayed.

2. The method according to claim 1, characterized in that The processing of improving the fluency of the collected video data includes: When the display framework interface reads the collected video data from the user cache each time, the latest frame of data is read; the display framework interface is a service called by the display driver for reading the collected video data, the display driver is used to display the read collected video data, and the user cache is a space in the vehicle computer for storing the collected video data sent by the USB data acquisition card; Initialize the maximum frame rate and write frame rate; Under the condition of not exceeding the maximum frame rate, the write frame rate is dynamically adjusted so that when the collected video data of the USB data acquisition card is written into the user cache according to the write frame rate, the number of collected video data frames in the user cache remains stable.

3. The method according to claim 2, characterized in that The method further comprises: By dynamically adjusting the write frame rate, the number of frames of the collected video data in the user cache is stabilized to 1 frame at all times; The first speed at which the audio framework interface reads the collected audio data from the user buffer each time is maintained, and the second speed at which the collected audio data of the USB data acquisition card is written into the user buffer is maintained.

4. The method according to claim 1, characterized in that: The receiving of the collected video data and the collected audio data sent by the USB data collection card comprises: The collected video data is written into the user cache through the libuvc library function; The collected audio data is written into the user buffer through the libusb library function.

5. The method according to claim 3, characterized in that: The playing of the collected audio data and displaying the processed collected video data includes: Reading the collected video data from the user buffer to the display driver through the display framework interface; The vehicle display screen is enabled to display the read collected video data through the display driver; Reading the collected audio data from the user buffer to the audio driver through the audio framework interface; The audio driver enables the vehicle speaker to play the read collected audio data.

6. The method according to claim 1, characterized in that The step of preprocessing the original video data and the original audio data by the USB data acquisition card comprises: The original video data and the original audio data are decoded, re-encoded, compressed and synchronously processed by the USB data acquisition card.

7. A car screen projection device, characterized in that: Applied to a vehicle computer, the device comprises: A data receiving module, used for receiving the collected video data and the collected audio data sent by the USB data acquisition card, wherein the USB data acquisition card is used for collecting the original video data and the original audio data of the host device, and pre-processing the original video data and the original audio data to obtain the collected video data and the collected audio data respectively; A fluency processing module, used for processing the collected video data to improve fluency; The screen projection module is used to play the collected audio data and display the processed collected video data.

8. A car screen projection system, characterized in that: include: A host device, a USB data acquisition card and a vehicle computer, wherein the audio and video interface of the host device is connected to the USB data acquisition card, and the USB data acquisition card is communicatively connected to the vehicle computer.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a computer to execute the method according to any one of claims 1 to 6.

10. A vehicle, characterized in that: The invention comprises a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions in the processor, thereby executing the method according to any one of claims 1 to 6.