Vehicle-mounted screen projection method and device, terminal equipment and storage medium

Abstract

The invention discloses a vehicle-mounted screen projection method and device, terminal equipment and a storage medium. The method comprises the following steps: establishing local area network connection with a vehicle instrument; when a screen projection confirmation instruction sent by the vehicle instrument is received, obtaining a current video to be subjected to screen projection; converting the video to be projected into a multi-frame video stream object; and transmitting each frame of video stream object to a vehicle instrument, so that the vehicle instrument displays the video stream object on an instrument display screen. According to the invention, the local area network connection is established between the client and the vehicle instrument, after the video to be projected is converted into the multiple frames of video stream objects, each frame of video stream object is transmitted to the vehicle instrument for display, screen projection can be realized without the help of third-party interconnection screen projection software, the situation of unstable connection is avoided, and the user experience is improved. Therefore, the vehicle-mounted screen projection effect can be effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of video processing, and more particularly to a vehicle screen mirroring method, device, terminal device and storage medium. Background Art

[0002] Vehicle screen mirroring technology is a technology that projects the screen content of a smart phone or other mobile device onto the central control display screen of a vehicle by wireless or wired means. Projecting the video picture onto the vehicle instrument can enhance the riding pleasure and interactivity.

[0003] Currently, existing vehicle screen mirroring methods usually rely on third-party interconnection screen mirroring software and use Bluetooth pairing and enabling the USB debugging function of the client to achieve screen mirroring interconnection between the client and the vehicle. However, when using third-party interconnection screen mirroring software to achieve screen mirroring, connection instability is likely to occur, resulting in poor vehicle screen mirroring effects. Summary of the Invention

[0004] The present invention provides a vehicle screen mirroring method, device, terminal device and storage medium to solve the technical problem that when using existing vehicle screen mirroring methods based on third-party interconnection screen mirroring software to achieve screen mirroring, connection instability is likely to occur, resulting in poor vehicle screen mirroring effects.

[0005] The present invention provides a vehicle screen mirroring method, including:

[0006] Establish a local area network connection with the vehicle instrument;

[0007] When receiving a screen mirroring confirmation instruction sent by the vehicle instrument, obtain the current video to be screen mirrored;

[0008] Convert the video to be screen mirrored into multiple video stream objects;

[0009] Transmit each frame of the video stream object to the vehicle instrument, so that the vehicle instrument displays the video stream object on the instrument display screen.

[0010] Further, the conversion of the video to be screen mirrored into multiple video stream objects includes:

[0011] Capture each frame of the video to be screen mirrored and convert each frame of the picture into a corresponding video stream object.

[0012] Further, the transmission of each frame of the video stream object to the vehicle instrument includes:

[0013] Encode the video object stream to obtain a corresponding data stream;

[0014] When the size of the data stream is less than a preset threshold, divide the data stream into multiple data packets, where the data packets include a start data packet, intermediate data packets, and an end data packet;

[0015] According to the identification code of each data packet, transmit all the data packets to the vehicle instrument in sequence.

[0016] Further, the transmitting all the data packets to the vehicle instrument in sequence further includes:

[0017] When the data packet received by the vehicle instrument is abnormal, determine the data packet to be retransmitted according to the identification code of the abnormal data packet; wherein, when a data packet is lost or the vehicle instrument does not receive a data packet within a preset time period, it is determined that the data packet received by the vehicle instrument is abnormal;

[0018] Transmit the data packet to be retransmitted to the vehicle instrument again.

[0019] Further, the displaying the video stream object on the instrument display screen includes:

[0020] After receiving all the data packets, combine all the data packets into a complete data stream according to the identification codes of all the data packets;

[0021] After encoding the complete data stream, display it on the instrument display screen.

[0022] The present invention further provides an in-vehicle screen mirroring device, including:

[0023] A local area network connection establishment module for establishing a local area network connection with the vehicle instrument;

[0024] A video to be screen mirrored acquisition module for acquiring the current video to be screen mirrored when receiving the screen mirroring confirmation instruction sent by the vehicle instrument;

[0025] A video stream object conversion module for converting the video to be screen mirrored into multiple frames of video stream objects;

[0026] A video stream object display module for transmitting each frame of the video stream object to the vehicle instrument, so that the vehicle instrument displays the video stream object on the instrument display screen.

[0027] Further, the video stream object display module is further configured to:

[0028] Perform encoding processing on the video object stream to obtain a corresponding data stream;

[0029] When the size of the data stream is less than a preset threshold, divide the data stream into multiple data packets, where the data packets include a start data packet, intermediate data packets, and an end data packet;

[0030] According to the identification code of each data packet, all the data packets are transmitted to the vehicle instrument in sequence.

[0031] Furthermore, the step of transmitting all the data packets to the vehicle instrument in sequence further includes:

[0032] When the data packet received by the vehicle instrument is abnormal, determine the data packet to be retransmitted according to the identification code of the abnormal data packet; wherein, when a data packet is lost or the vehicle instrument does not receive a data packet within a preset time period, it is determined that the data packet received by the vehicle instrument is abnormal.

[0033] Transmit the data packet to be retransmitted to the vehicle instrument again.

[0034] The present invention also provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the vehicle-mounted screen projection method as described above is implemented.

[0035] The present invention also provides a computer-readable storage medium, which includes a stored computer program; wherein, when the computer program runs, it controls the device where the computer-readable storage medium is located to execute the vehicle-mounted screen projection method as described above.

[0036] By establishing a local area network connection between the client and the vehicle instrument, converting the video to be projected into multiple video stream objects, and transmitting each frame of the video stream object to the vehicle instrument for display, the present invention can achieve screen projection without relying on a third-party interconnection screen projection software, avoiding unstable connection situations, and thus effectively improving the vehicle-mounted screen projection effect.

[0037] Furthermore, when a data packet is abnormal, determining the data packet that needs to be retransmitted according to the identification code of the abnormal data packet, without retransmitting all data packets, can effectively improve the data packet transmission efficiency, and further effectively improve the vehicle-mounted screen projection efficiency, and can avoid data packet delay and data packet loss, which is beneficial to improving the vehicle-mounted screen projection effect. Description of the Drawings

[0038] Figure 1 is a schematic flowchart of the vehicle-mounted screen projection method provided by an embodiment of the present invention;

[0039] Figure 2 is a schematic timing flowchart of the vehicle-mounted screen projection provided by an embodiment of the present invention;

[0040] Figure 3 is another schematic flowchart of the vehicle-mounted screen projection method provided by an embodiment of the present invention;

[0041] Figure 4 It is a schematic diagram of the screen mirroring test result of a certain client provided by an embodiment of the present invention;

[0042] Figure 5 It is a schematic structural diagram of the vehicle-mounted screen mirroring device provided by an embodiment of the present invention. Specific embodiments

[0043] Next, the technical solutions in the embodiments of the present application will be clearly and completely described with reference to the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present application.

[0044] In the description of the present application, it should be understood that the terms "first" and "second" are only used for descriptive purposes and cannot be construed as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present application, unless otherwise stated, the meaning of "a plurality" is two or more.

[0045] In the description of the present application, it should be noted that unless otherwise clearly defined and limited, the terms "installation", "connection", and "connection" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present application can be understood according to specific situations.

[0046] Please refer to Figure 1 , the present invention provides a vehicle-mounted screen mirroring method, including:

[0047] S1. Establish a local area network connection with the vehicle instrument;

[0048] In the embodiment of the present invention, the execution subject of the vehicle-mounted screen mirroring method can be a client, where the client can be a mobile device. The way for the client to establish a local area network connection with the vehicle instrument can be: receiving the control clicked by the user on the mobile client, turning on the camera, and scanning the two-dimensional code of the vehicle instrument through the camera of the client to establish a local area network connection with the vehicle instrument.

[0049] S2. When receiving the screen mirroring confirmation instruction sent by the vehicle instrument, obtain the current video to be screen mirrored;

[0050] In an embodiment of the present invention, the vehicle instrument receives a screen mirroring instruction initiated by the user clicking the screen mirroring control on the client, and returns the screen mirroring confirmation instruction to the client after receiving it. When the client receives the screen mirroring confirmation instruction, it obtains the video in the current client as the video to be screen mirrored.

[0051] S3. Convert the video to be screen mirrored into multiple video stream objects;

[0052] In an embodiment of the present invention, the video to be screen mirrored can be intercepted for its picture, and then converted into a video stream object.

[0053] S4. Transmit each video stream object to the vehicle instrument, so that the vehicle instrument displays the video stream object on the instrument display screen.

[0054] In an embodiment of the present invention, by establishing a local area network connection between the client and the vehicle instrument, after converting the video to be screen mirrored into multiple video stream objects, each video stream object is transmitted to the vehicle instrument for display, without the need to rely on a third-party interconnection screen mirroring software to achieve screen mirroring, avoiding the situation of unstable connection, thereby effectively improving the in-vehicle screen mirroring effect.

[0055] Please refer to Figure 2 , in an embodiment, a timing process schematic diagram of in-vehicle screen mirroring is provided.

[0056] In an embodiment, step S3, converting the video to be screen mirrored into multiple video stream objects, includes:

[0057] Intercept each frame of the picture of the video to be screen mirrored, and convert each frame of the picture into a corresponding video stream object.

[0058] In an embodiment of the present invention, each frame of the picture of the video to be screen mirrored can be intercepted by means of screen capture.

[0059] In an embodiment, step S4, transmitting each video stream object to the vehicle instrument, includes:

[0060] S41. Perform encoding processing on the video object stream to obtain a corresponding data stream;

[0061] In an embodiment of the present invention, the video stream object can be compressed and encoded in the H264 format to obtain a corresponding data stream.

[0062] S42. When the size of the data stream is less than a preset threshold, divide the data stream into multiple data packets, and the data packets include a start data packet, intermediate data packets, and an end data packet;

[0063] In an embodiment of the present invention, the preset threshold can be set according to actual needs. For example, the preset threshold can be set to 500K, that is, the maximum value of each transmission is 500K.

[0064] In an embodiment of the present invention, a data stream is divided into multiple data packets. For example, if the total size of the data stream is 1300K, the data stream can be divided into three data packets, namely a start data packet of 500K, a middle data packet of 500K, and an end data packet of 300K.

[0065] S43. Transmit all the data packets to the vehicle instrument in sequence according to the identification code of each data packet.

[0066] In an embodiment of the present invention, a corresponding identification code and timestamp can be set for each data packet to distinguish different data packets and different transmission sequences of different data packets.

[0067] In the embodiment of the present invention, by dividing the data stream into multiple data packets, including a start data packet, a middle data packet, and an end data packet, and transmitting all the data packets to the vehicle instrument in sequence according to the identification code of each data packet, the problem of network congestion caused by an overly large video stream can be effectively avoided, thereby effectively improving the transmission efficiency of the data stream, and further effectively improving the efficiency of in-vehicle screen mirroring.

[0068] In one embodiment, S43. Transmitting all the data packets to the vehicle instrument in sequence further includes:

[0069] S431. When the data packet received by the vehicle instrument is abnormal, determine the data packet to be retransmitted according to the identification code of the abnormal data packet; wherein, when a data packet is lost or the vehicle instrument does not receive a data packet within a preset time period, it is determined that the data packet received by the vehicle instrument is abnormal.

[0070] In the embodiment of the present invention, due to reasons such as unstable network or insufficient client memory resulting in abnormal data packet situations, the situations where the data packet received by the vehicle instrument is abnormal include:

[0071] 1. Determine that the data packet received by the vehicle instrument is incomplete according to the identification code of the data packet, for example, the end data packet is missing.

[0072] 2. Determine that the data packet received by the vehicle instrument is delayed according to the identification code of the data packet, for example, the end data packet is not received within a preset time period.

[0073] S432. Retransmit the data packet to be retransmitted to the vehicle instrument.

[0074] In the embodiment of the present invention, when an abnormal data packet occurs, determine the data packet that needs to be retransmitted according to the identification code of the abnormal data packet, without retransmitting all the data packets, which can effectively improve the transmission efficiency of the data packets, and further effectively improve the efficiency of in-vehicle screen mirroring, and can avoid the occurrence of data packet delay and data packet loss, which is beneficial to improving the effect of in-vehicle screen mirroring.

[0075] In one embodiment, step S4, displaying the video stream object on the instrument display screen, includes:

[0076] After receiving all the data packets, combining all the data packets into a complete data stream according to the identification codes of all the data packets;

[0077] After encoding the data stream, display it on the instrument display screen.

[0078] In the embodiment of the present invention, compare the identification code of all the received data packets with the identification code of the data packets before sending. If they are the same, it is determined that all the data packets have been received. At this time, combine all the data packets into a complete data stream according to the identification codes of all the data packets, and perform H264 decoding processing on the complete data stream to display the decoded video stream on the instrument display screen.

[0079] In the embodiment of the present invention, after completing the screen projection of one frame of image, repeat steps S3 - S4 to perform the transmission of the next frame of image until a screen projection closing instruction sent by the client is received.

[0080] Please refer to Figure 3 , which is another process schematic diagram of a vehicle screen projection method provided for an embodiment.

[0081] Please refer to Figure 4 , which is a schematic diagram of the screen projection test results for a certain client in an embodiment.

[0082] Implementing the embodiment of the present invention has the following beneficial effects:

[0083] In the embodiment of the present invention, by establishing a local area network connection between the client and the vehicle instrument, after converting the video to be screen - projected into multiple video stream objects, each video stream object is transmitted to the vehicle instrument for display. There is no need to rely on a third - party inter - connected screen projection software to achieve screen projection, avoiding unstable connection situations, and thus effectively improving the vehicle screen projection effect.

[0084] Furthermore, when data packet anomalies occur, determine the data packets that need to be re - transmitted according to the identification codes of the abnormal data packets. There is no need to re - transmit all the data packets, which can effectively improve the efficiency of data packet transmission, and then effectively improve the efficiency of vehicle screen projection. Moreover, it can avoid data packet delay and data packet loss, which is beneficial to improving the vehicle screen projection effect.

[0085] Please refer to Figure 5 , based on the same inventive concept as the above - mentioned embodiment, the present invention also provides a vehicle screen projection device, including:

[0086] A local area network connection establishment module 10, used to establish a local area network connection with the vehicle instrument;

[0087] The video to be screen-cast acquisition module 20 is configured to acquire the current video to be screen-cast when receiving a screen-cast confirmation instruction sent by the vehicle instrument;

[0088] The video stream object conversion module 30 is configured to convert the video to be screen-cast into multiple frames of video stream objects;

[0089] The video stream object display module 40 is configured to transmit each frame of the video stream object to the vehicle instrument, so that the vehicle instrument displays the video stream object on the instrument display screen.

[0090] In one embodiment, the video stream object conversion module 30 is further configured to:

[0091] Capture each frame of the video to be screen-cast and convert each frame of the picture into a corresponding video stream object.

[0092] In one embodiment, the video stream object display module 40 is further configured to:

[0093] Perform encoding processing on the video object stream to obtain a corresponding data stream;

[0094] When the size of the data stream is less than a preset threshold, divide the data stream into multiple data packets, where the data packets include a start data packet, intermediate data packets, and an end data packet;

[0095] According to the identification code of each data packet, transmit all the data packets to the vehicle instrument in sequence.

[0096] In one embodiment, transmitting all the data packets to the vehicle instrument in sequence further includes:

[0097] When the data packet received by the vehicle instrument is abnormal, determine the data packet to be retransmitted according to the identification code of the abnormal data packet; wherein, when the data packet is lost or when the vehicle instrument does not receive a data packet within a preset time period, it is determined that the data packet received by the vehicle instrument is abnormal;

[0098] Transmit the data packet to be retransmitted to the vehicle instrument again.

[0099] Correspondingly, an embodiment of the present invention further provides a terminal device, including: a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, the vehicle-mounted screen-casting method of any one of the above embodiments is implemented.

[0100] The terminal device of this embodiment includes: a processor, a memory, and a computer program and computer instructions stored in the memory and executable on the processor. When the processor executes the computer program, the steps in the above-mentioned first embodiment are implemented, for example Figure 1Steps S1 to S4 shown above. Alternatively, when the processor executes the computer program, it implements the functions of each module / unit in the above device embodiment, such as the video stream object conversion module 30.

[0101] Exemplarily, the computer program can be segmented into one or more modules / units. One or more modules / units are stored in the memory and executed by the processor to complete the present invention. One or more modules / units can be a series of computer program instruction segments capable of performing specific functions, and these instruction segments are used to describe the execution process of the computer program in the terminal device. For example, the video stream object conversion module 30 is used to convert the video to be cast onto the screen into multiple video stream objects.

[0102] The terminal device can be a computing device such as a desktop computer, a notebook, a palm computer, and a cloud server, etc. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art can understand that the schematic diagram is only an example of the terminal device and does not constitute a limitation on the terminal device. It may include more or fewer components than shown in the figure, or combine certain components, or different components. For example, the terminal device may further include input / output devices, network access devices, a bus, etc.

[0103] The so-called processor may be a Central Processing Unit (CPU), or may also be other general-purpose processors, Digital Signal Processors (DSPs), Application Specific Integrated Circuits (ASICs), Field-Programmable Gate Arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The processor is the control center of the terminal device and connects various parts of the entire terminal device through various interfaces and lines.

[0104] The memory can be used to store computer programs and / or modules. By running or executing the computer programs and / or modules stored in the memory, and calling the data stored in the memory, the processor can implement various functions of the terminal device. The memory mainly includes a program storage area and a data storage area. Among them, the program storage area can store an operating system, application programs required for at least one function, etc.; the data storage area can store data created according to the use of the mobile terminal, etc. In addition, the memory can include high-speed random access memory, and can also include non-volatile memory, such as a hard disk, memory, plug-in hard disk, Smart Media Card (SMC), Secure Digital (SD) card, Flash Card, at least one magnetic disk storage device, flash memory device, or other volatile solid-state storage devices.

[0105] Among them, if the modules / units integrated in the terminal device are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on such an understanding, to implement all or part of the processes in the above-mentioned embodiment methods of the present invention, it can also be completed by a computer program instructing related hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned various method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice in the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0106] Correspondingly, an embodiment of the present invention also provides a computer-readable storage medium. The computer-readable storage medium includes a stored computer program. Among them, when the computer program runs, it controls the device where the computer-readable storage medium is located to execute the vehicle-mounted screen mirroring method in any one of the above embodiments.

[0107] In the above specific embodiments, the object, technical solution and beneficial effects of the present invention have been further described in detail. It should be understood that the above are only specific embodiments of the present invention and are not used to limit the protection scope of the present invention. In particular, it is pointed out that for those skilled in the art, any modifications, equivalent replacements, improvements, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A vehicle-mounted screen projection method, characterized in that, including: Establish a local area network connection with the vehicle instrument; When receiving the screen mirroring confirmation instruction sent by the vehicle instrument, obtain the current video to be screen mirrored; Convert the video to be screen mirrored into multiple frames of video stream objects; Transmit each frame of the video stream object to the vehicle instrument, so that the vehicle instrument displays the video stream object on the instrument display screen.

2. The vehicle-mounted screen projection method according to claim 1, wherein Converting the video to be screen mirrored into multiple frames of video stream objects includes: Capture each frame of the video to be screen mirrored, and convert each frame of the picture into a corresponding video stream object.

3. The vehicle-mounted screen projection method according to claim 1, characterized in that, Transmitting each frame of the video stream object to the vehicle instrument includes: Perform encoding processing on the video object stream to obtain a corresponding data stream; When the size of the data stream is less than a preset threshold, divide the data stream into multiple data packets, and the data packets include start data packets, intermediate data packets, and end data packets; According to the identification code of each data packet, transmit all the data packets to the vehicle instrument in sequence.

4. The vehicle-mounted screen projection method according to claim 3, wherein, Transmitting all the data packets to the vehicle instrument in sequence further includes: When the data packet received by the vehicle instrument is abnormal, determine the data packet to be retransmitted according to the identification code of the abnormal data packet; wherein, when the data packet is lost or the vehicle instrument does not receive a data packet within a preset time period, it is determined that the data packet received by the vehicle instrument is abnormal; Transmit the data packet to be retransmitted to the vehicle instrument again.

5. The vehicle-mounted screen projection method according to claim 3, characterized in that, Displaying the video stream object on the instrument display screen includes: After receiving all the data packets, combine all the data packets into a complete data stream according to the identification codes of all the data packets; After performing encoding processing on the complete data stream, display it on the instrument display screen.

6. A vehicle-mounted screen projection device, characterized in that, including; A local area network connection establishment module for establishing a local area network connection with the vehicle instrument; A video to be screen mirrored acquisition module for obtaining the current video to be screen mirrored when receiving the screen mirroring confirmation instruction sent by the vehicle instrument; A video stream object conversion module for converting the video to be screen mirrored into multiple frames of video stream objects; A video stream object display module for transmitting each frame of the video stream object to the vehicle instrument, so that the vehicle instrument displays the video stream object on the instrument display screen.

7. The vehicle-mounted screen projection device according to claim 6, wherein, The video stream object display module is further configured to: Perform encoding processing on the video object stream to obtain a corresponding data stream; When the size of the data stream is less than a preset threshold, divide the data stream into multiple data packets, and the data packets include start data packets, intermediate data packets, and end data packets; According to the identification code of each data packet, transmit all the data packets to the vehicle instrument in sequence.

8. The vehicle-mounted screen projection device according to claim 7, wherein, Transmitting all the data packets to the vehicle instrument in sequence further includes: When the data packet received by the vehicle instrument is abnormal, determine the data packet to be retransmitted according to the identification code of the abnormal data packet; wherein, when the data packet is lost or the vehicle instrument does not receive a data packet within a preset time period, it is determined that the data packet received by the vehicle instrument is abnormal; Transmit the data packet to be retransmitted to the vehicle instrument again.

9. A terminal device, characterized in that, including: A processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein when the processor executes the computer program, the vehicle screen mirroring method according to any one of claims 1-5 is implemented.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes a stored computer program; wherein, when the computer program runs, it controls the device where the computer-readable storage medium is located to execute the vehicle screen mirroring method according to any one of claims 1-5.

Images