On-board interconnection method and apparatus, and electronic device and storage medium

Abstract

Disclosed in the present application are an on-board interconnection method and apparatus, and an electronic device and a storage medium. The method is applied to a terminal side, and comprises: in response to a first user interconnection operation, establishing a basic interconnection with a head unit side; in response to a screen-mirroring instruction for a target application, determining corresponding interconnection scenario data on the basis of an application interconnection manner, and an application interface of the target application; and on the basis of WebRTC, establishing a real-time transport protocol channel with the head unit side, and sending the interconnection scenario data to the head unit side by means of the real-time transport protocol channel, and displaying the interconnection scenario data. By means of the technical solutions of the embodiments of the present application, a vehicle factory only needs to develop an on-board interconnection scheme to meet market and user requirements, thereby achieving high compatibility; and developers can realize control across all layers from upper-layer application to underlying implementation, such that upon encountering a problem, the solving efficiency is high, and problem localization is clear, thereby reducing the access difficulty of an on-board interconnection scenario and improving the development efficiency.

Description

A vehicle connectivity method, apparatus, electronic device, and storage medium

[0001] This application claims priority to Chinese Patent Application No. 202411544681.8, filed with the Chinese Patent Office on October 31, 2024, the entire contents of which are incorporated herein by reference. Technical Field

[0002] This application relates to the field of vehicle connectivity technology, and more particularly to a vehicle connectivity method, apparatus, electronic device, and storage medium. Background Technology

[0003] With the continuous improvement of people's living standards, the penetration rate of automobiles in my country has increased significantly. Although the automotive ecosystem is constantly developing, it is still limited by hardware devices and related ecosystem constraints. This has led to the emergence of various in-vehicle connectivity technologies that integrate mobile terminal (such as smartphones) ecosystems with the vehicle ecosystem. Currently, mature in-vehicle connectivity technologies include HUAWEI HiCar, CarPlay, Baidu Carlife, Android Auto, and CarLink. However, these technologies suffer from limited application support, high integration difficulty, cumbersome certification processes, and high costs. Therefore, how to enable automakers to develop only one in-vehicle connectivity solution to meet market and user needs has become a pressing issue. Summary of the Invention

[0004] This application provides a vehicle connectivity method, device, electronic device, and storage medium, enabling car manufacturers to meet market and user needs by developing only one vehicle connectivity solution, thereby reducing the access difficulty and certification cost of vehicle connectivity scenarios.

[0005] According to one aspect of this application, a vehicle connectivity method is provided, the method being applied to a terminal side, comprising:

[0006] In response to the first user interconnection operation, a basic interconnection connection is established with the vehicle's infotainment system;

[0007] In response to the screen mirroring command for the target application, the corresponding interconnection scenario data is determined based on the application interconnection method and the application interface of the target application;

[0008] A real-time transmission protocol channel is established between WebRTC and the vehicle's infotainment system, and interconnected scenario data is sent to the vehicle's infotainment system for display through the real-time transmission protocol channel.

[0009] According to another aspect of this application, a vehicle connectivity method is provided, which is applied to the vehicle's infotainment system and includes:

[0010] In response to a second user interconnection operation, a basic interconnection connection is established with the terminal side;

[0011] A real-time transmission protocol channel is established between WebRTC and the terminal, and interconnection scenario data sent by the terminal is obtained through the real-time transmission protocol channel;

[0012] Display connected scene data on the target in-vehicle display device.

[0013] According to another aspect of this application, an in-vehicle connectivity device is provided, which is applied to a terminal side and includes:

[0014] The basic interconnect module is used to establish a basic interconnect connection with the vehicle's infotainment system in response to the first user interconnect operation.

[0015] The data determination module is used to respond to the screen projection command of the target application and determine the corresponding interconnection scenario data based on the application interconnection method and the application interface of the target application.

[0016] The data transmission module is used to establish a real-time transmission protocol channel between WebRTC and the vehicle-mounted system, and to send interconnected scene data to the vehicle-mounted system for display through the real-time transmission protocol channel.

[0017] According to another aspect of this application, an in-vehicle connectivity device is provided, which is applied to the vehicle's infotainment system and includes:

[0018] The basic interconnect module is used to establish a basic interconnect connection with the terminal side in response to the second user interconnect operation.

[0019] The data acquisition module is used to establish a real-time transmission protocol channel between WebRTC and the terminal side, and to acquire interconnection scenario data sent by the terminal side through the real-time transmission protocol channel;

[0020] The data display module is used to display connected scene data on the target in-vehicle display device.

[0021] According to another aspect of this application, an electronic device is provided, the electronic device comprising:

[0022] At least one processor; and

[0023] A memory communicatively connected to the at least one processor; wherein,

[0024] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the vehicle connectivity method described in any embodiment of this application.

[0025] According to another aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the vehicle connectivity method according to any embodiment of this application.

[0026] The technical solution of this application embodiment is applied to the terminal side. In response to a first user interconnection operation, a basic interconnection connection is established with the vehicle-mounted system. In response to a screen projection command for a target application, corresponding interconnection scenario data is determined based on the application interconnection method and the application interface of the target application. A real-time transmission protocol channel is established between the terminal and the vehicle-mounted system based on WebRTC, and the interconnection scenario data is sent to the vehicle-mounted system for display through this channel. In this solution, after establishing a basic interconnection connection with the vehicle-mounted system, the terminal determines the corresponding interconnection scenario data based on the application interconnection method and the application interface of the target application to be projected. The interconnection scenario data is then transmitted to the vehicle-mounted system for display through the real-time transmission protocol channel established between the terminal and the vehicle-mounted system based on WebRTC. This allows car manufacturers to develop only one in-vehicle interconnection solution to meet market and user needs, resulting in high compatibility. Simultaneously, developers can achieve layer-by-layer control from the upper application layer to the lower layer, leading to high efficiency in problem-solving and clear problem localization. This reduces the difficulty of accessing in-vehicle interconnection scenarios and improves development efficiency.

[0027] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0028] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0029] Figure 1 is a flowchart of a vehicle connectivity method according to Embodiment 1 of this application;

[0030] Figure 2 is a flowchart of a vehicle connectivity method according to Embodiment 2 of this application;

[0031] Figure 3 is an interactive flowchart of terminal-side device discovery according to Embodiment 2 of this application;

[0032] Figure 4 is an interactive flowchart of a terminal-side / vehicle-side device connection according to Embodiment 2 of this application;

[0033] Figure 5 is a flowchart of the data acquisition process for an interconnected scenario according to Embodiment 2 of this application;

[0034] Figure 6 is a flowchart of the construction of an interconnected scenario according to Embodiment 2 of this application;

[0035] Figure 7 is a flowchart of the data transmission process in an interconnected scenario according to Embodiment 2 of this application;

[0036] Figure 8 is a flowchart of a vehicle connectivity method according to Embodiment 3 of this application;

[0037] Figure 9 is a schematic diagram of a TLV-encoded data frame format provided according to Embodiment 3 of this application;

[0038] Figure 10 is a flowchart of a counter-control process for a target application according to Embodiment 3 of this application;

[0039] Figure 11 is a structural schematic diagram of an in-vehicle connectivity device according to Embodiment 4 of this application;

[0040] Figure 12 is a structural schematic diagram of an in-vehicle connectivity device according to Embodiment 5 of this application;

[0041] Figure 13 is a schematic diagram of the structure of an electronic device that implements the vehicle connectivity method of the present application. Detailed Implementation

[0042] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0043] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0044] Example 1

[0045] Existing in-vehicle connectivity solutions all have certain limitations. The following analysis examines several mainstream in-vehicle connectivity solutions currently on the market:

[0046] HUAWEI HiCar can currently only be used in some Huawei HarmonyOS and EMUI systems and some Honor Magic systems. It is difficult to integrate, takes a long time to authenticate, and all of them depend on the terminal device (such as mobile phone) system application.

[0047] Apple CarPlay can only be used on iOS devices, and the authentication process is cumbersome, integration is difficult, and it supports few applications.

[0048] Android Auto is only available in overseas markets. It is difficult to integrate, has a complicated certification process, supports fewer applications, and relies on reserved permissions from Google Play.

[0049] Carlink, although supported by OPPO, Xiaomi and VIVO, has inconsistent integration effects among manufacturers, limited application support, significant defects in some manufacturers, high integration difficulty and cumbersome certification.

[0050] Baidu Carlife is supported by almost all Android and Apple phones, but it only supports Baidu's own ecosystem content, making integration difficult and authentication cumbersome.

[0051] In summary, existing vehicle connectivity solutions suffer from limited application support, high integration difficulty, and cumbersome and costly authentication processes. To address these issues, Figure 1 is a flowchart of a vehicle connectivity method provided in Embodiment 1 of this application. This embodiment is applicable to vehicle connectivity between the terminal side and the vehicle's infotainment system. This method can be executed by a vehicle connectivity device, which can be implemented in hardware and / or software and can be configured in an electronic device. As shown in Figure 1, the vehicle connectivity method provided in Embodiment 1, applied to the terminal side, specifically includes the following steps:

[0052] S110, in response to the first user interconnection operation, establishes a basic interconnection connection with the vehicle's infotainment system.

[0053] The "terminal side" can refer to the user's mobile terminal device, such as, but not limited to, mobile phones, tablets, and smart wearable devices (such as extended reality XR devices). The "vehicle-mounted side" can refer to the in-vehicle terminal (i.e., the vehicle-mounted system).

[0054] The first user interconnection operation can be understood as a series of user operations performed by the user on the terminal side in order to achieve basic interconnection between the terminal side and the vehicle system side. These operations may include, but are not limited to: using a data cable to connect the terminal side and the vehicle system side via wired connection, turning on the Bluetooth function of the terminal side, and turning on the WiFi connection function of the terminal side.

[0055] Basic interconnectivity can include wired and wireless connections, such as but not limited to: USB (Universal Serial Bus) connection, Bluetooth communication connection, and peer-to-peer (P2P) local area network communication connection. It can be understood that the basic interconnectivity between the terminal side and the vehicle side is the connection foundation for subsequent vehicle interconnectivity, and can establish a basic data communication channel between the two, such as a P2P channel.

[0056] In this embodiment of the application, when a user needs to perform in-vehicle interconnection between the terminal side and the vehicle-mounted system, a series of related first user interconnection operations can be performed on the terminal side. In response to the first user interconnection operations, the terminal side can establish a basic interconnection connection with the vehicle-mounted system, such as a wired connection or a wireless connection, thereby providing a data communication foundation for subsequent in-vehicle interconnection.

[0057] S120: In response to the screen mirroring command for the target application, determine the corresponding interconnection scenario data based on the application interconnection method and the application interface of the target application.

[0058] The target application can refer to various applications (APPs) running on the terminal side that can be projected onto the vehicle's infotainment system, such as navigation applications, music applications, and video applications.

[0059] Application interconnection can be understood as the multi-screen display method that users expect. Application interconnection can include two types: simultaneous multi-screen display and different multi-screen display. Simultaneous multi-screen display means that the display screen on the terminal side and the display screen on the vehicle side display the same screen content, i.e., screen mirroring; while different multi-screen display means that the display screen on the terminal side and the display screen on the vehicle side display different screen content.

[0060] Interconnected scenario data can be understood as media stream data obtained after performing a series of operations such as screen capture, recording, compression, and encoding on the application interconnected scenario (such as map navigation, multimedia playback, widget components, etc.) built on the terminal side.

[0061] In this embodiment, the user can trigger the corresponding screen casting operation for the target application according to their desired application interconnection method, such as multi-screen simultaneous display or multi-screen different display. The terminal side can respond to the screen casting command of the target application, capture and record the application interface of the target application during operation, convert it into media stream data, and then obtain the interconnection scenario data to be transmitted to the vehicle-mounted system after data processing operations such as compression and encoding of the media stream data.

[0062] S130: Establish a real-time transmission protocol channel between WebRTC and the vehicle-mounted system, and send interconnected scenario data to the vehicle-mounted system for display through the real-time transmission protocol channel.

[0063] WebRTC (Web Real-Time Communication) is a real-time communication technology used to enable end-to-end audio, video, and data sharing between browsers. It consists of a set of standards, protocols, and a JavaScript API (Application Programming Interface), which enables P2P communication between two devices.

[0064] Real-time Transport Protocol (RTP) is a real-time transport protocol based on UDP, which can be used to transmit multimedia data such as audio and video in real time.

[0065] In this embodiment, the terminal and the vehicle-mounted system can establish a channel for real-time transmission of media stream data, namely the RTP channel, through the exchange and negotiation of WebRTC Session Description Protocol (SDP) information. The generated interconnected scene data is then continuously transmitted to the vehicle-mounted system through the RTP channel, so that the vehicle-mounted system can decode and display the interconnected scene data, thereby projecting the target application on the terminal to the vehicle-mounted display device on the vehicle-mounted system, thus realizing vehicle interconnection.

[0066] The technical solution of this application embodiment is applied to the terminal side. In response to a first user interconnection operation, a basic interconnection connection is established with the vehicle-mounted system. In response to a screen projection command for a target application, corresponding interconnection scenario data is determined based on the application interconnection method and the application interface of the target application. A real-time transmission protocol channel is established between the terminal and the vehicle-mounted system based on WebRTC, and the interconnection scenario data is sent to the vehicle-mounted system for display through this channel. In this solution, after establishing a basic interconnection connection with the vehicle-mounted system, the terminal determines the corresponding interconnection scenario data based on the application interconnection method and the application interface of the target application to be projected. The interconnection scenario data is then transmitted to the vehicle-mounted system for display through the real-time transmission protocol channel established between the terminal and the vehicle-mounted system based on WebRTC. This allows car manufacturers to develop only one in-vehicle interconnection solution to meet market and user needs, resulting in high compatibility. Simultaneously, developers can achieve layer-by-layer control from the upper application layer to the lower layer, leading to high efficiency in problem-solving and clear problem localization. This reduces the difficulty of accessing in-vehicle interconnection scenarios and improves development efficiency.

[0067] Example 2

[0068] Figure 2 is a flowchart of a vehicle connectivity method provided in Embodiment 2 of this application. It is further optimized and extended based on the above embodiments and can be combined with various optional technical solutions in the above embodiments. As shown in Figure 2, the vehicle connectivity method provided in Embodiment 2 is applied to the terminal side and specifically includes the following steps:

[0069] S210, in response to the first user interconnection operation, sequentially establishes a Bluetooth communication connection and a point-to-point local area network communication connection with the vehicle's infotainment system.

[0070] In this embodiment, before the terminal and the vehicle infotainment system can connect in-vehicle, they need to establish a basic interconnection connection, namely a Bluetooth communication connection and a P2P-based local area network (LAN) communication connection. Specifically, the basic interconnection connection includes two parts: terminal-side device discovery and terminal-side / vehicle infotainment system device connection. The interaction process for terminal-side device discovery is shown in Figure 3. In this embodiment, terminal-side device discovery is mainly achieved based on general Bluetooth BLE (Bluetooth Low Energy) near-field communication. The interaction process for terminal-side / vehicle infotainment system device connection is shown in Figure 4. In this embodiment, device connection between the terminal and the vehicle infotainment system is achieved based on P2P LAN communication.

[0071] S220: In response to the screen mirroring command for the target application, call the preset interface capture function to capture the application interface of the target application and obtain the screen content data.

[0072] The preset interface capture function can refer to a pre-configured API for implementing screen capture and screen recording. This preset interface capture function can include at least the MediaProjection function, which allows the application to capture the device screen content with user authorization. Screen content data can refer to data obtained by capturing the target application's interface; this data contains the displayed content of the target application's interface to be projected.

[0073] In this embodiment, when the application interconnection method is multi-screen simultaneous display, the terminal side can execute steps S220 to S230 to generate interconnection scene data. That is, the terminal side can respond to the screen projection command of the target application and capture the application interface of the target application by calling preset interface capture functions such as MediaProjection, thereby obtaining the corresponding screen content data. Specifically, the process of the terminal side generating screen content data is as follows:

[0074] 1) Call the getSystemService method to obtain a MediaProjectionManager instance, which is used to manage activities and requests related to MediaProjection;

[0075] 2) Call the `createScreenCaptureIntent` method of `MediaProjectionManager` to create an Intent to trigger the system's screen recording authorization interface. The user will receive a system-level permission request to allow the application to capture screen content.

[0076] 3) Once the user grants authorization, a MediaProjection instance can be obtained by calling the getMediaProjection method of MediaProjectionManager, which is used to retrieve the screen content;

[0077] 4) Create a VirtualDisplay object by calling the createVirtualDisplay method of MediaProjection. This object is used to capture screen content and display it on the Surface. You can specify parameters such as the width, height, and pixel density of the display.

[0078] 5) Call the MediaRecorder start method to begin recording screen content in order to obtain screen content data.

[0079] S230: Call the preset media encoder to encode the screen content data to obtain the corresponding interconnected scene data.

[0080] The preset media encoder can refer to a pre-configured encoder component for implementing multimedia data encoding. The preset media encoder can include at least the MediaCodec audio and video codec class, which allows applications to encode and decode audio and video data.

[0081] In this embodiment of the application, as shown in Figure 5, taking the MediaCodeC encoder as an example, the process of obtaining interconnected scene data is as follows:

[0082] 1) Create an encoder by calling the MediaCodec's createEncoderByType method and passing in the encoding type;

[0083] 2) Configure the encoder by calling the configure method, passing in encoding parameters such as bitrate, frame rate, and encoding format;

[0084] 3) Prepare input and output surfaces for the encoder, where the input surface is used to transmit the data to be encoded (i.e., screen content data), and the output surface is used to receive the encoded data (i.e., interconnected scene data);

[0085] 4) Call the start method to start the encoder and send the screen content data to be encoded to the encoder's input queue through the write method;

[0086] 5) Monitor the output queue and obtain the encoded data through the dequeueOutputBuffer method to get the corresponding interconnected scenario data.

[0087] S240. In response to the screen casting command for the target application, create a media display component and a virtual display object, and associate the virtual display object with the media display component.

[0088] Among them, the media presentation component can refer to the component that can display the application interface content on an external display screen, and the media presentation component can include at least the Presentation component.

[0089] A virtual display object can refer to a created virtual display screen in which the application interface of the target application can be displayed. It is the basis for the subsequent implementation of multi-screen display function. For example, a virtual display object can include at least a VirtualDisplay object.

[0090] In this embodiment, when the application interconnection method is multi-screen display, the terminal can execute steps S240 to S260 to generate interconnection scene data. Specifically, the terminal can respond to the screen casting command for the target application, create corresponding media display components and virtual display objects, and associate the two. In a specific embodiment, as shown in Figure 6, taking the Presentation component and VirtualDisplay object as an example, the terminal can respond to the screen casting command for the target application, first create a MediaProjection instance and a VirtualDisplay object according to the method in S220, and create a Presentation component by overriding the constructor. Then, the VirtualDisplay object is associated with the Presentation component through the DisplayManager class callback interface.

[0091] S250: Call the preset interface capture function to capture the application interface of the target application, so that the application interface is projected onto the media display component.

[0092] In this embodiment of the application, taking the Presentation component as the media display component as an example, the Presentation component can be configured with a SurfaceView component. After obtaining screen recording permission, the Surface object is obtained from the SurfaceView component and assigned to the MediaProjection instance. Then, the application interface of the target application can be captured by calling the MediaProjection instance so that the application interface is projected onto the Presentation component. At this time, the view displayed in the Presentation component is the page to be projected.

[0093] S260. Call the preset media encoder to encode the view displayed in the media display component to obtain the corresponding interconnected scene data.

[0094] In this embodiment of the application, the encoding process in S230 can be referred to to create a preset media encoder, such as the MediaCodeC encoder, and configure it. Then, the MediaCodeC encoder can be used to encode the view displayed in the Presentation component, i.e. the page to be projected, so as to obtain the corresponding interconnected scene data.

[0095] S270. Based on the point-to-point communication channel with the vehicle-mounted system, WebRTC session description protocol information is exchanged and negotiated to establish a real-time transmission protocol channel.

[0096] The Session Description Protocol (SDP) is a format used to describe multimedia sessions. It provides information about the session, including the session name, purpose, start and end times, media type, transport protocol, encoding method, etc.

[0097] In this embodiment of the application, as shown in Figure 7, the terminal side can forward the created offer SDP (a set of SDP information describing local multimedia capabilities) to the vehicle-mounted system through the P2P channel already established with the vehicle-mounted system. After receiving the offer SDP, the vehicle-mounted system responds according to its own support capabilities, and at the same time creates an answer SDP and returns it to the terminal side, thereby completing the exchange and negotiation of WebRTC SDP information between the terminal side and the vehicle-mounted system, and establishing an RTP channel between the two.

[0098] S280: Transmit the connected scene data to the vehicle's infotainment system via a real-time transmission protocol channel so that the vehicle's infotainment system can decode and display the connected scene data.

[0099] In this embodiment, the terminal can transmit the interconnected scene data media stream to the vehicle-mounted unit via the RTP channel, so that the vehicle-mounted unit can decode and restore the interconnected scene data and render and display the recovered image frames on the display device to achieve in-vehicle interconnection.

[0100] The technical solution of this application embodiment is applied to the terminal side. In response to the first user interconnection operation, it sequentially establishes a Bluetooth communication connection and a point-to-point local area network communication connection with the vehicle-mounted system. Then, according to different application interconnection methods, it captures and encodes the application interface of the target application into corresponding interconnection scene data. Finally, it uses the real-time transmission protocol channel established with the vehicle-mounted system to transmit the interconnection scene data to the vehicle-mounted system for decoding and display. This allows car manufacturers to develop only one in-vehicle interconnection solution to meet market and user needs, with high compatibility. At the same time, developers can achieve layer-by-layer control from the upper layer application to the lower layer, resulting in high efficiency in problem solving and clear problem localization, thereby reducing the difficulty of accessing in-vehicle interconnection scenarios and improving development efficiency. In addition, this in-vehicle interconnection solution can also enrich user usage scenarios, improve user experience, and can be customized for user needs, with higher scalability.

[0101] Example 3

[0102] Figure 8 is a flowchart of a vehicle connectivity method provided in Embodiment 3 of this application. This embodiment is applicable to vehicle connectivity between the vehicle-mounted system and the terminal side. The method can be executed by a vehicle connectivity device, which can be implemented in hardware and / or software and can be configured in an electronic device. As shown in Figure 8, the vehicle connectivity method provided in Embodiment 3, applied to the vehicle-mounted system side, specifically includes the following steps:

[0103] S310, in response to the second user interconnection operation, establishes a basic interconnection connection with the terminal side.

[0104] The second user interconnection operation can be understood as a series of user operations performed by the user on the vehicle-mounted system in order to achieve basic interconnection between the vehicle-mounted system and the terminal. These operations may include, but are not limited to: using a data cable to connect the vehicle-mounted system and the terminal, turning on the Bluetooth function of the vehicle-mounted system, and turning on the WiFi connection function of the vehicle-mounted system.

[0105] In this embodiment of the application, referring to Figures 3 and 4, the vehicle-mounted system can sequentially establish a Bluetooth communication connection and a P2P-based local area network communication connection with the terminal, that is, establish a basic interconnection connection with the terminal. The specific connection process can be referred to the above embodiments, and will not be repeated here.

[0106] S320: Establishes a real-time transmission protocol channel between WebRTC and the terminal side, and obtains interconnection scenario data sent by the terminal side through the real-time transmission protocol channel.

[0107] In this embodiment of the application, referring to FIG7, the vehicle-mounted system can establish an RTP channel between the two based on the P2P channel already established with the terminal side, through the exchange and negotiation of WebRTC SDP information, and continuously receive interconnection scenario data sent by the terminal side through the RTP channel.

[0108] S330 displays connected scene data on the target in-vehicle display device.

[0109] In this embodiment, after receiving the interconnection scene data, the vehicle-mounted system can call a preset media decoder to decode and restore it, and then render and display the decoded interconnection scene data on the target vehicle display device, such as the vehicle central control screen, thereby projecting the target application on the terminal side to the vehicle-mounted system to achieve vehicle interconnection.

[0110] Furthermore, based on the above-mentioned embodiments, the in-vehicle connectivity method provided in this embodiment further includes:

[0111] The system acquires user touch events targeting the in-vehicle display device and encapsulates these touch events into a reverse control data packet according to the pre-configured TLV protocol.

[0112] The counter-control data packet is sent to the terminal side so that the terminal side can control the target application according to the counter-control data packet.

[0113] A touch event (MotionEvent) can refer to a MotionEvent object generated by the vehicle system when a user touches the screen of the target in-vehicle display device. This MotionEvent object contains relevant information about the user's touch action, such as the coordinate position information, time information, and pressure information of the touch.

[0114] The TLV (Tag-Length-Value) protocol is a communication protocol that uses a triplet of Tag, Length, and Value to encode data, as shown in Figure 9. A TLV unit consists of three parts: Tag, Length, and Value, which are described below:

[0115] A tag is used to identify the type or meaning of data, essentially labeling the Value field. It can be a number, a character, or other predefined identifier, indicating the specific meaning of the data that follows. In some specifications, a tag can be a single byte or multiple bytes, depending on the number and range of identifiers required.

[0116] Length is used to identify the length of the Value field. It can be one byte, two bytes, or more, depending on the maximum possible length of the Value field. The Length field usually uses variable-length encoding. For shorter Value fields, fewer bits are used to represent their length, thus saving space.

[0117] The Value is used to identify the actual data content. Its type and length are determined by the preceding Tag and Length fields, and can be an integer, string, binary data block, etc. Because the TLV format allows for variable Value lengths, it can flexibly handle data of different types and lengths.

[0118] This application uses a TLV-type protocol format to ensure data integrity, which has at least the following advantages:

[0119] ① Scalability: New types can be easily added without affecting existing data parsing;

[0120] ② Flexibility: Value can be any type of data, from simple numerical values ​​to complex structures;

[0121] ③ Robustness: Even if some data cannot be parsed, the parser can skip this part of the data and continue processing subsequent TLV units;

[0122] ④ Self-descriptive: Each TLV unit contains all the information needed for processing, without requiring external context or metadata.

[0123] In this embodiment of the application, as shown in Figure 10, after the application interface of the target application on the terminal side is projected onto the vehicle-mounted system, the vehicle-mounted system can realize reverse control of the target application on the terminal side based on the user's touch operation on the target vehicle-mounted display device. Specifically, the vehicle-mounted system can obtain the user's touch event MotionEvent on the target vehicle-mounted display device, such as the vehicle's central control screen, and then encapsulate it into a reverse control data packet using the TLV protocol; then, the reverse control data packet is sent to the terminal side through a pre-established P2P channel with the terminal side. Upon receiving the reverse control data packet, the terminal side parses and obtains the touch event MotionEvent, and replays it to simulate the user's operation, thereby realizing reverse control of the target application. That is, the user's operation on the vehicle-mounted system can be responded to in real time on the terminal side.

[0124] It is understandable that the hardware environment of in-vehicle terminals is fixed after the vehicle leaves the factory and cannot be upgraded. However, with the upgrading of in-vehicle software, the computing power requirements are increasing, which will cause the in-vehicle system to run slower and slower. The in-vehicle interconnection method provided in this application embodiment can project the target application on the terminal side to the in-vehicle system. Users can then reverse-operate the target application on the terminal side through the in-vehicle display device on the in-vehicle system, saving computing power of the in-vehicle terminal. At the same time, displaying the target application on the in-vehicle system can also bring a better viewing experience to users.

[0125] The technical solution of this application embodiment is applied to the vehicle-mounted system. It establishes a basic interconnection connection with the terminal side in response to a second user interconnection operation; establishes a real-time transmission protocol channel between the vehicle and the terminal side based on WebRTC, and obtains interconnection scene data sent by the terminal side through the real-time transmission protocol channel; and displays the interconnection scene data on the target in-vehicle display device. Using this technical solution, car manufacturers only need to develop one in-vehicle interconnection solution to meet market and user needs, with high compatibility; at the same time, developers can achieve layer-by-layer control from the upper application layer to the lower layer, resulting in high efficiency in problem solving and clear problem localization, thereby reducing the difficulty of accessing in-vehicle interconnection scenarios and improving development efficiency.

[0126] Example 4

[0127] Figure 11 is a structural schematic diagram of an in-vehicle connectivity device provided in Embodiment 4 of this application. As shown in Figure 11, the device is applied to the terminal side and includes:

[0128] Basic interconnect module 41 is used to establish a basic interconnect connection with the vehicle-mounted system in response to the first user interconnect operation;

[0129] The data determination module 42 is used to respond to the screen projection command of the target application and determine the corresponding interconnection scenario data according to the application interconnection method and the application interface of the target application.

[0130] The data transmission module 43 is used to establish a real-time transmission protocol channel between WebRTC and the vehicle-mounted system, and to send interconnected scene data to the vehicle-mounted system for display through the real-time transmission protocol channel.

[0131] The technical solution of this application embodiment is applied to the terminal side. A basic interconnection module responds to the first user interconnection operation and establishes a basic interconnection connection with the vehicle-mounted system. A data determination module responds to the screen projection command for the target application and determines the corresponding interconnection scenario data based on the application interconnection method and the application interface of the target application. A data transmission module establishes a real-time transmission protocol channel with the vehicle-mounted system based on WebRTC and sends the interconnection scenario data to the vehicle-mounted system for display through this real-time transmission protocol channel. In this solution, after establishing a basic interconnection connection with the vehicle-mounted system, the terminal side determines the corresponding interconnection scenario data based on the application interconnection method and the application interface of the target application to be projected. The interconnection scenario data is then transmitted to the vehicle-mounted system for display through the real-time transmission protocol channel established between the terminal side and the vehicle-mounted system based on WebRTC. This allows car manufacturers to develop only one in-vehicle interconnection solution to meet market and user needs, resulting in high compatibility. Simultaneously, developers can achieve layer-by-layer control from the upper application layer to the lower layer, leading to high efficiency in problem solving and clear problem localization. This reduces the difficulty of accessing in-vehicle interconnection scenarios and improves development efficiency.

[0132] Furthermore, based on the above-described embodiments, the basic interconnection module 41 includes:

[0133] The basic interconnection unit is used to respond to the first user interconnection operation and establish a Bluetooth communication connection and a point-to-point local area network communication connection with the vehicle's infotainment system in sequence.

[0134] Furthermore, based on the above-mentioned application embodiments, when the application interconnection method is multi-screen simultaneous display, the data determination module 42 includes:

[0135] The screen content data acquisition unit is used to respond to the screen casting command of the target application, call the preset interface capture function to capture the application interface of the target application, and obtain screen content data.

[0136] The first interconnected scene data acquisition unit is used to call a preset media encoder to encode the screen content data and obtain the corresponding interconnected scene data.

[0137] Furthermore, based on the above-described embodiments, when the application interconnection method is multi-screen display, the data determination module 42 includes:

[0138] The component association unit is used to respond to the screen casting command of the target application, create a media display component and a virtual display object, and associate the virtual display object with the media display component;

[0139] The application interface capture unit is used to call a preset interface capture function to capture the application interface of the target application, so that the application interface is projected onto the media display component.

[0140] The second interconnected scene data acquisition unit is used to call a preset media encoder to encode the view displayed in the media display component to obtain the corresponding interconnected scene data.

[0141] Furthermore, based on the above-described embodiments, the data transmission module 43 includes:

[0142] The real-time transmission protocol channel establishment unit is used to exchange and negotiate WebRTC session description protocol information based on the point-to-point communication channel with the vehicle-mounted device, so as to establish a real-time transmission protocol channel.

[0143] The data transmission unit is used to transmit connected scenario data to the vehicle-mounted system via a real-time transmission protocol channel, so that the vehicle-mounted system can decode and display the connected scenario data.

[0144] The vehicle connectivity device provided in this application embodiment can execute the vehicle connectivity method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of executing the method.

[0145] Example 5

[0146] Figure 12 is a structural schematic diagram of a vehicle connectivity device provided in Embodiment 5 of this application. As shown in Figure 12, applied to the vehicle's infotainment system, the device includes:

[0147] The basic interconnection module 51 is used to establish a basic interconnection connection with the terminal side in response to the second user interconnection operation;

[0148] Data acquisition module 52 is used to establish a real-time transmission protocol channel between WebRTC and the terminal side, and to acquire interconnection scenario data sent by the terminal side through the real-time transmission protocol channel;

[0149] The data display module 53 is used to display interconnected scene data on the target in-vehicle display device.

[0150] The technical solution of this application embodiment is applied to the vehicle-mounted system. A basic interconnection module responds to a second user interconnection operation and establishes a basic interconnection connection with the terminal. A data acquisition module establishes a real-time transmission protocol channel with the terminal based on WebRTC and acquires interconnection scene data sent by the terminal through this channel. A data display module displays the interconnection scene data on the target in-vehicle display device. This technical solution allows car manufacturers to develop only one in-vehicle interconnection solution to meet market and user needs, offering high compatibility. Simultaneously, developers can achieve layer-by-layer control from the upper application layer to the lower layer, resulting in high efficiency in problem-solving and clear problem localization, thereby reducing the difficulty of integrating in-vehicle interconnection scenarios and improving development efficiency.

[0151] Furthermore, based on the above-described embodiments, the data display module 53 includes:

[0152] The data display unit is used to call a preset media decoder to decode the interconnected scene data and render and display the decoded interconnected scene data on the target in-vehicle display device.

[0153] Furthermore, based on the above-described embodiments, the in-vehicle connectivity device also includes:

[0154] The touch event encapsulation module is used to acquire user touch events on the target vehicle display device and encapsulate the touch events into reverse control data packets according to the pre-configured TLV protocol.

[0155] The counter-control module is used to send counter-control data packets to the terminal side, so that the terminal side can control the target application according to the counter-control data packets.

[0156] The vehicle connectivity device provided in this application embodiment can execute the vehicle connectivity method provided in any embodiment of this application, and has the corresponding functional modules and beneficial effects of executing the method.

[0157] Example 6

[0158] Figure 13 illustrates a schematic diagram of an electronic device 60 that can be used to implement embodiments of this application. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the application described and / or claimed herein.

[0159] As shown in Figure 13, the electronic device 60 includes at least one processor 61 and a memory, such as a read-only memory (ROM) 62 or a random access memory (RAM) 63, communicatively connected to the at least one processor 61. The memory stores computer programs executable by the at least one processor. The processor 61 can perform various appropriate actions and processes based on the computer program stored in the ROM 62 or loaded from storage unit 68 into the RAM 63. The RAM 63 can also store various programs and data required for the operation of the electronic device 60. The processor 61, ROM 62, and RAM 63 are interconnected via a bus 64. An input / output (I / O) interface 65 is also connected to the bus 64.

[0160] Multiple components in electronic device 60 are connected to I / O interface 65, including: input unit 66, such as keyboard, mouse, etc.; output unit 67, such as various types of monitors, speakers, etc.; storage unit 68, such as disk, optical disk, etc.; and communication unit 69, such as network card, modem, wireless transceiver, etc. Communication unit 69 allows electronic device 60 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0161] Processor 61 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 61 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 61 performs the various methods and processes described above, such as in-vehicle connectivity methods.

[0162] In some embodiments, the vehicle connectivity method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 68. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 60 via ROM 62 and / or communication unit 69. When the computer program is loaded into RAM 63 and executed by processor 61, one or more steps of the vehicle connectivity method described above may be performed. Alternatively, in other embodiments, processor 61 may be configured to perform the vehicle connectivity method by any other suitable means (e.g., by means of firmware).

[0163] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), payload-programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0164] Computer programs used to implement the methods of this application may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0165] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0166] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0167] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0168] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0169] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this application can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this application can be achieved, and this is not limited herein.

[0170] The specific embodiments described above do not constitute a limitation on the scope of protection of this application. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application should be included within the scope of protection of this application.

Claims

1. A vehicle connectivity method, applied to a terminal side, the method comprising: In response to the first user interconnection operation, a basic interconnection connection is established with the vehicle's infotainment system; In response to a screen mirroring command for a target application, the corresponding interconnection scenario data is determined based on the application interconnection method and the application interface of the target application. A real-time transmission protocol channel is established between WebRTC and the vehicle-mounted system, and the interconnected scene data is sent to the vehicle-mounted system for display through the real-time transmission protocol channel.

2. The method according to claim 1, wherein, The step of establishing a basic interconnection connection with the vehicle-mounted system in response to the first user interconnection operation includes: In response to the first user interconnection operation, a Bluetooth communication connection and a point-to-point local area network communication connection are established sequentially with the vehicle-mounted system.

3. The method according to claim 1, wherein, When the application interconnection method is multi-screen simultaneous display, the step of responding to the screen casting command of the target application and determining the corresponding interconnection scenario data according to the application interconnection method and the application interface of the target application includes: In response to the screen mirroring command for the target application, a preset interface capture function is invoked to capture the application interface of the target application and obtain screen content data; The screen content data is encoded by calling a preset media encoder to obtain the corresponding interconnected scene data.

4. The method according to claim 1, wherein, When the application interconnection method is multi-screen display, the step of responding to the screen casting command for the target application and determining the corresponding interconnection scenario data based on the application interconnection method and the application interface of the target application includes: In response to a screen mirroring command for the target application, a media display component and a virtual display object are created, and the virtual display object is associated with the media display component; A preset interface capture function is invoked to capture the application interface of the target application, so that the application interface is projected onto the media display component; The preset media encoder is invoked to encode the view displayed in the media display component to obtain the corresponding interconnected scene data.

5. The method according to claim 1, wherein, The process of establishing a real-time transmission protocol channel between WebRTC and the vehicle-mounted system, and sending the interconnected scenario data to the vehicle-mounted system for display via the real-time transmission protocol channel, includes: The WebRTC session description protocol information is exchanged and negotiated based on the point-to-point communication channel with the vehicle-mounted system to establish the real-time transmission protocol channel. The interconnected scene data is transmitted to the vehicle-mounted system via the real-time transmission protocol channel, so that the vehicle-mounted system can decode and display the interconnected scene data.

6. A vehicle connectivity method, applied to the vehicle's infotainment system, the method comprising: In response to a second user interconnection operation, a basic interconnection connection is established with the terminal side; A real-time transmission protocol channel is established between WebRTC and the terminal side, and interconnection scenario data sent by the terminal side is obtained through the real-time transmission protocol channel; The interconnected scene data is displayed on the target in-vehicle display device.

7. The method according to claim 6, wherein, The step of displaying the interconnected scene data on the target in-vehicle display device includes: The preset media decoder is invoked to decode the interconnected scene data, and the decoded interconnected scene data is rendered and displayed on the target in-vehicle display device.

8. The method according to claim 6, further comprising: The system acquires user touch events on the target vehicle display device and encapsulates the touch events into a reverse control data packet according to the pre-configured TLV protocol. The counter-control data packet is sent to the terminal side so that the terminal side controls the target application according to the counter-control data packet.

9. A vehicle-mounted connectivity device, applied on a terminal side, the device comprising: The basic interconnect module is used to establish a basic interconnect connection with the vehicle's infotainment system in response to the first user interconnect operation. The data determination module is used to determine the corresponding interconnection scenario data based on the application interconnection method and the application interface of the target application in response to the screen projection command of the target application. The data transmission module is used to establish a real-time transmission protocol channel between WebRTC and the vehicle-mounted system, and to send the interconnected scene data to the vehicle-mounted system for display through the real-time transmission protocol channel.

10. A vehicle-mounted connectivity device, applied to the vehicle's infotainment system, the device comprising: The basic interconnect module is used to establish a basic interconnect connection with the terminal side in response to the second user interconnect operation. The data acquisition module is used to establish a real-time transmission protocol channel between WebRTC and the terminal side, and to acquire interconnection scenario data sent by the terminal side through the real-time transmission protocol channel; The data display module is used to display the interconnected scene data on the target in-vehicle display device.

11. An electronic device, the electronic device comprising: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the vehicle connectivity method according to any one of claims 1-8.

12. A computer-readable storage medium storing computer instructions that, when executed by a processor, implement the vehicle connectivity method of any one of claims 1-8.

Images