Vehicle-mounted interconnection method and device, electronic equipment and storage medium

Abstract

The invention discloses a vehicle-mounted interconnection method and device, electronic equipment and a storage medium. The method is applied to a terminal side and comprises the following steps: in response to a first user interconnection operation, establishing a basic interconnection connection with a vehicle machine side; in response to a screen projection instruction for the target application, determining corresponding interconnection scene data according to the application interconnection mode and an application interface of the target application; and establishing a real-time transmission protocol channel with the vehicle machine side based on WebRTC, and sending the interconnection scene data to the vehicle machine side for display through the real-time transmission protocol channel. According to the technical scheme of the embodiment of the invention, a vehicle factory only needs to develop a set of vehicle-mounted interconnection scheme to meet market and user requirements, and the compatibility is high; and meanwhile, developers can realize layer-by-layer control from upper-layer application to bottom-layer application, the problem solving efficiency is high, and the problem positioning is clear, so that the access difficulty of a vehicle-mounted interconnection scene is reduced, and the development efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of vehicle-mounted interconnection technology, and in particular to a vehicle-mounted interconnection method, device, electronic equipment and storage medium. Background Art

[0002] With the continuous improvement of people's living standards, the penetration rate of automobiles in my country has increased significantly. Although the automobile ecosystem continues to develop, it is still limited by hardware equipment and related ecological restrictions. As a result, various in-vehicle interconnection technologies that integrate the ecology of mobile terminals (such as mobile phones, etc.) and the ecology of car machines have been born. At present, the mature in-vehicle interconnection technologies mainly include: HUAWEI HiCar, CarPlay, Baidu Carlife, Android Auto, CarLink, etc. However, the above-mentioned interconnection technologies have the problems of few supported applications, difficult integration, cumbersome certification and high cost. Therefore, how to enable car manufacturers to develop only one set of in-vehicle interconnection solutions to meet market and user needs has become an urgent problem to be solved. Summary of the invention

[0003] The present invention provides a vehicle-mounted interconnection method, device, electronic device and storage medium, so that car manufacturers only need to develop one set of vehicle-mounted interconnection solutions to meet market and user needs, thereby reducing the access difficulty and authentication cost of vehicle-mounted interconnection scenarios.

[0004] According to one aspect of the present invention, a vehicle-mounted interconnection method is provided, which is applied to a terminal side and includes:

[0005] In response to the first user interconnection operation, establishing a basic interconnection connection with the vehicle computer side;

[0006] In response to a screen projection instruction for a target application, corresponding interconnection scene data is determined according to the application interconnection mode and the application interface of the target application;

[0007] A real-time transport protocol channel is established between WebRTC and the vehicle computer side, and the interconnected scene data is sent to the vehicle computer side for display through the real-time transport protocol channel.

[0008] According to another aspect of the present invention, a vehicle-mounted interconnection method is provided, which is applied to a vehicle computer side and includes:

[0009] In response to the second user interconnection operation, establishing a basic interconnection connection with the terminal side;

[0010] A real-time transport protocol channel is established between WebRTC and the terminal side, and the interconnected scene data sent by the terminal side is obtained through the real-time transport protocol channel;

[0011] Display the interconnected scene data on the target vehicle display device.

[0012] According to another aspect of the present invention, a vehicle-mounted interconnection device is provided, which is applied to a terminal side and includes:

[0013] A basic interconnection module, configured to establish a basic interconnection connection with the vehicle computer side in response to a first user interconnection operation;

[0014] A data determination module, for responding to a screen projection instruction for a target application, and determining corresponding interconnection scene data according to an application interconnection mode and an application interface of the target application;

[0015] The data transmission module is used to establish a real-time transmission protocol channel between the vehicle side and the vehicle side based on WebRTC, and send the interconnected scene data to the vehicle side for display through the real-time transmission protocol channel.

[0016] According to another aspect of the present invention, a vehicle-mounted interconnection device is provided, which is applied to a vehicle computer side and includes:

[0017] A basic interconnection module, configured to establish a basic interconnection connection with the terminal side in response to an interconnection operation of a second user;

[0018] The data acquisition module is used to establish a real-time transport protocol channel between the terminal side and WebRTC, and obtain the interconnected scene data sent by the terminal side through the real-time transport protocol channel;

[0019] The data display module is used to display the interconnected scene data on the target vehicle display device.

[0020] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising:

[0021] at least one processor; and

[0022] a memory communicatively connected to the at least one processor; wherein,

[0023] The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the in-vehicle interconnection method described in any embodiment of the present invention.

[0024] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the vehicle-mounted interconnection method described in any embodiment of the present invention when executed.

[0025] The technical solution of the embodiment of the present invention is applied to the terminal side, and establishes a basic interconnection connection with the vehicle side by responding to the first user interconnection operation; in response to the screen projection instruction for the target application, the corresponding interconnection scene data is determined according to the application interconnection method and the application interface of the target application; a real-time transmission protocol channel is established between WebRTC and the vehicle side, and the interconnection scene data is sent to the vehicle side for display through the real-time transmission protocol channel. After the terminal side in this solution establishes a basic interconnection connection with the vehicle side, it determines the corresponding interconnection scene data according to the application interconnection method and the application interface of the target application to be projected, and transmits the interconnection scene data to the vehicle side for display through the real-time transmission protocol channel established between WebRTC and the vehicle side, so that the car manufacturer only needs to develop one set of in-vehicle interconnection solutions to meet the market and user needs, and has high compatibility; at the same time, developers can implement layer-by-layer control from upper-layer applications to the bottom layer, and have high efficiency in solving problems and clear problem location, thereby reducing the difficulty of accessing in-vehicle interconnection scenarios and improving development efficiency.

[0026] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0028] Figure 1 is a flow chart of a vehicle interconnection method provided according to Embodiment 1 of the present invention;

[0029] Figure 2 is a flow chart of a vehicle interconnection method provided according to Embodiment 2 of the present invention;

[0030] Figure 3 This is an interactive flow chart of a terminal side device discovery provided according to Embodiment 2 of the present invention;

[0031] Figure 4 This is an interactive flow chart of a terminal side / vehicle side device connection provided according to the second embodiment of the present invention;

[0032] Figure 5 is a flow chart of a collection process of interconnected scene data provided according to the second embodiment of the present invention;

[0033] Figure 6is a flow chart for constructing an interconnection scenario provided according to Embodiment 2 of the present invention;

[0034] Figure 7 is a flow chart of a transmission process of interconnected scene data provided according to Embodiment 2 of the present invention;

[0035] Figure 8 is a flow chart of a vehicle interconnection method provided according to Embodiment 3 of the present invention;

[0036] Fig. 9 This is a schematic diagram of a TLV-encoded data frame format provided according to Embodiment 3 of the present invention;

[0037] Fig.10 is a flow chart of a reverse control process of a target application provided according to Embodiment 3 of the present invention;

[0038] Fig.11 is a structural schematic diagram of a vehicle-mounted interconnection device provided according to a fourth embodiment of the present invention;

[0039] Fig.12 is a structural schematic diagram of a vehicle-mounted interconnection device provided according to Embodiment 5 of the present invention;

[0040] Fig.13 It is a structural schematic diagram of an electronic device for implementing the vehicle-mounted interconnection method according to an embodiment of the present invention. DETAILED DESCRIPTION

[0041] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0042] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0043] Embodiment 1

[0044] Existing in-vehicle interconnection solutions all have certain limitations. The following is an analysis of several mainstream in-vehicle interconnection solutions currently on the market:

[0045] HUAWEI HiCar can currently only be applied to some Huawei Hongmeng, EMUI systems and some Honor Magic systems. It is difficult to integrate, takes a long time to authenticate, and depends on the system application of the terminal device (such as a mobile phone);

[0046] Apple CarPlay can only be used on iOS devices, and has a complicated certification process, high integration difficulty, and few supported applications.

[0047] Android Auto is only available in overseas markets, has high integration difficulty, a cumbersome certification process, supports fewer applications, and relies on Google Play reserved permissions;

[0048] Carlink: Although OPPO / Xiaomi / VIVO all support it, the integration effects of different manufacturers are inconsistent, and there are fewer supported applications. Some manufacturers have too many defects, making integration difficult and certification cumbersome.

[0049] Baidu Carlife, although supported by almost all Android+Apple phones, only supports Baidu's own ecological content, which makes integration difficult and authentication cumbersome.

[0050] In summary, the existing in-vehicle interconnection solutions have the problems of few supported applications, difficult integration, cumbersome certification and high cost. In order to solve the above problems, Figure 1 This is a flow chart of a vehicle interconnection method provided in the first embodiment of the present invention. This embodiment is applicable to the case where vehicle interconnection is performed between the terminal side and the vehicle side. The method can be executed by a vehicle interconnection device. The vehicle interconnection device can be implemented in the form of hardware and / or software. The vehicle interconnection device can be configured in an electronic device. Figure 1 As shown, a vehicle interconnection method provided in the first embodiment is applied to the terminal side and specifically includes the following steps:

[0051] S110: In response to the first user interconnection operation, establish a basic interconnection connection with the vehicle computer side.

[0052] The terminal side may refer to the user's mobile terminal device, for example, including but not limited to: mobile phones, tablet computers, smart wearable devices (such as extended reality XR devices), etc. The vehicle side may refer to the vehicle terminal (i.e., the vehicle system).

[0053] The first user interconnection operation can be understood as a series of user operations performed on the terminal side in order to achieve a basic interconnection connection between the terminal side and the vehicle side, which may include but is not limited to: using a data cable to connect the terminal side to the vehicle side, turning on the Bluetooth function of the terminal side, turning on the WiFi connection function of the terminal side, etc.

[0054] Basic interconnection connections may include wired connection methods and wireless connection methods, such as but not limited to: USB (Universal Serial Bus) connection, Bluetooth communication connection, local area network communication connection based on peer-to-peer (P2P), etc. It can be understood that the basic interconnection connection between the terminal side and the vehicle side is the connection basis for subsequent vehicle-mounted interconnection, and can establish a basic data communication channel between the two, such as a P2P channel.

[0055] In an embodiment of the present invention, when the user needs to perform in-vehicle interconnection between the terminal side and the vehicle side, a series of related first-user interconnection operations can be performed on the terminal side, so that the terminal side can respond to the first-user interconnection operation and establish a basic interconnection connection with the vehicle side, such as a wired connection or a wireless connection, thereby providing a data communication basis for subsequent in-vehicle interconnection.

[0056] S120. In response to a screen projection instruction for a target application, corresponding interconnection scenario data is determined according to an application interconnection method and an application interface of the target application.

[0057] Among them, the target application can refer to various applications (Application, APP) running on the terminal side that can be projected and displayed on the car side, such as navigation applications, music applications, video applications, etc.

[0058] The application interconnection method can be understood as the multi-screen display method of the application expected by the user. The application interconnection method can include two types: multi-screen same display and multi-screen different display. Among them, multi-screen same display means that the display screen on the terminal side and the display screen on the vehicle side display the same screen content, that is, screen mirroring; while multi-screen different display means that the display screen on the terminal side and the display screen on the vehicle side respectively display different screen contents.

[0059] The interconnected scene data can be understood as the media stream data obtained after a series of operations such as screen capture, recording, compression, and encoding for the application interconnected scenes built on the terminal side (such as map navigation, multimedia playback, widget components, etc.).

[0060] In an embodiment of the present invention, a user can trigger a corresponding screen projection operation on a target application according to the desired application interconnection method, such as multi-screen same display or multi-screen different display, so that the terminal side can respond to the screen projection instruction of the target application, capture and record the application interface of the target application when it is running, convert it into media stream data, and then obtain the interconnection scene data to be transmitted to the vehicle side after compressing, encoding and other data processing operations on the media stream data.

[0061] S130, establishing a real-time transport protocol channel between the vehicle computer side and the vehicle computer side based on WebRTC, and sending the interconnected scene data to the vehicle computer side for display through the real-time transport protocol channel.

[0062] Among them, WebRTC (Web Real-Time Communication) is a real-time communication technology used to realize (end-to-end) audio, video and data sharing between browsers. It consists of a set of standards, protocols and JavaScript API (Application Programming Interface), which can realize P2P communication between two devices.

[0063] Real-time Transport Protocol (RTP) is a real-time transport protocol based on UDP protocol, which can be used for real-time transmission of multimedia data such as audio and video.

[0064] In an embodiment of the present invention, the terminal side and the vehicle computer side can establish a channel for real-time transmission of media stream data between the two through the exchange and negotiation of WebRTC Session Description Protocol (SDP) information, namely, an RTP channel, and then continuously transmit the generated interconnected scene data to the vehicle computer side through the RTP channel, so that the vehicle computer side decodes and displays the interconnected scene data, thereby projecting the target application on the terminal side onto the on-board display device on the vehicle computer side to realize in-vehicle interconnection.

[0065] The technical solution of the embodiment of the present invention is applied to the terminal side, and establishes a basic interconnection connection with the vehicle side by responding to the first user interconnection operation; in response to the screen projection instruction for the target application, the corresponding interconnection scene data is determined according to the application interconnection method and the application interface of the target application; a real-time transmission protocol channel is established between WebRTC and the vehicle side, and the interconnection scene data is sent to the vehicle side for display through the real-time transmission protocol channel. After the terminal side in this solution establishes a basic interconnection connection with the vehicle side, it determines the corresponding interconnection scene data according to the application interconnection method and the application interface of the target application to be projected, and transmits the interconnection scene data to the vehicle side for display through the real-time transmission protocol channel established between WebRTC and the vehicle side, so that the car manufacturer only needs to develop one set of in-vehicle interconnection solutions to meet the market and user needs, and has high compatibility; at the same time, developers can implement layer-by-layer control from upper-layer applications to the bottom layer, and have high efficiency in solving problems and clear problem location, thereby reducing the difficulty of accessing in-vehicle interconnection scenarios and improving development efficiency.

[0066] Embodiment 2

[0067] Figure 2 This is a flow chart of a vehicle interconnection method provided in the second embodiment of the present invention, which is further optimized and expanded based on the above implementation, and can be combined with various optional technical solutions in the above implementation. Figure 2 As shown, a vehicle interconnection method provided in the second embodiment is applied to the terminal side and specifically includes the following steps:

[0068] S210: In response to the first user interconnection operation, a Bluetooth communication connection and a point-to-point local area network communication connection are established in sequence with the vehicle computer side.

[0069] In the embodiment of the present invention, before the terminal side and the vehicle side are interconnected, the two sides need to establish a basic interconnection connection, namely, a Bluetooth communication connection and a P2P-based LAN communication connection. Specifically, the basic interconnection connection includes two parts: terminal side device discovery and terminal side / vehicle side device connection. Among them, the interactive process of terminal side device discovery is as follows: Figure 3 As shown, the embodiment of the present invention mainly realizes terminal-side device discovery based on universal Bluetooth BLE (Bluetooth Low Energy, low power consumption) near field communication; the interactive process of terminal-side / vehicle-side device connection is as follows Figure 4 As shown, the embodiment of the present invention realizes the device connection between the terminal side and the vehicle side based on P2P local area network communication.

[0070] S220. In response to a screen projection instruction for a target application, a preset interface capture function is called to capture an application interface of the target application to obtain screen content data.

[0071] The preset interface capture function may refer to a pre-configured API for implementing screen capture and screen recording, and the preset interface capture function may include at least a MediaProjection function, which allows an application to capture the content of a device screen with user authorization. The screen content data may refer to data obtained by screen capturing an application interface of a target application, and the screen content data includes the display content of the application interface of the target application to be projected.

[0072] In an embodiment of the present invention, when the application interconnection mode is multi-screen co-display, the terminal side can execute steps S220 to S230 to generate interconnection scene data, that is, the terminal side can respond to the screen projection instruction of the target application, and realize the capture of the application interface of the target application by calling the preset interface capture function such as MediaProjection, so as to obtain the corresponding screen content data. Specifically, the process of generating screen content data on the terminal side is as follows:

[0073] 1) Call the getSystemService method to obtain a MediaProjectionManager instance to manage activities and requests related to MediaProjection;

[0074] 2) Call the createScreenCaptureIntent method of MediaProjectionManager to create an Intent to trigger the system's screen recording authorization interface. The user will receive a system-level permission request to allow the application to capture the screen content;

[0075] 3) After the user agrees to the authorization, a MediaProjection instance can be obtained by calling the getMediaProjection method of MediaProjectionManager to obtain the screen content;

[0076] 4) Create a virtual display VirtualDisplay object by calling the createVirtualDisplay method of MediaProjection to capture the screen content and display it on the Surface, and specify the display width, height, pixel density and other parameters;

[0077] 5) Call the start method of MediaRecorder to start recording the screen content to obtain the screen content data.

[0078] S230: Call a preset media encoder to encode the screen content data to obtain corresponding interconnected scene data.

[0079] Among them, the preset media encoder may refer to a pre-configured encoder component for implementing multimedia data encoding, and the preset media encoder may at least include a MediaCodec audio and video codec class, which allows an application to encode and decode audio and video data.

[0080] In the embodiment of the present invention, Figure 5 As shown in the figure, taking the MediaCodeC encoder as an example, the process of obtaining interconnected scene data is as follows:

[0081] 1) Create an encoder by calling the createEncoderByType method of MediaCodec and passing in the encoding type;

[0082] 2) Configure the encoder by calling the configure method and passing in encoding parameters such as bit rate, frame rate, encoding format, etc.

[0083] 3) Prepare input and output Surface for the encoder, where the input Surface is used to transmit the data to be encoded (i.e., screen content data), and the output Surface is used to receive the encoded data (i.e., interconnected scene data);

[0084] 4) Call the start method to start the encoder, and send the screen content data to be encoded to the input queue of the encoder through the write method;

[0085] 5) Monitor the output queue and obtain the encoded data through the dequeueOutputBuffer method to obtain the corresponding interconnected scene data.

[0086] S240: In response to a screen projection instruction for a target application, create a media presentation component and a virtual display screen object, and associate the virtual display screen object with the media presentation component.

[0087] The media display component may refer to a component capable of displaying application interface content on an external display screen, and the media display component may include at least a Presentation component.

[0088] A virtual display screen object may refer to a created virtual display screen, in which the application interface of a target application may be displayed, and is the basis for the subsequent implementation of a multi-screen display function. Exemplarily, the virtual display screen object may include at least a VirtualDisplay object.

[0089] In an embodiment of the present invention, when the application interconnection mode is multi-screen heterogeneous display, the terminal side can execute steps S240 to S260 to generate interconnection scene data. Specifically, the terminal side can respond to the screen projection instruction of the target application, create a corresponding media display component and a virtual display screen object, and associate the two. Figure 6 As shown, taking the Presentation component and the VirtualDisplay object as an example, the terminal side can respond to the screen projection instruction of the target application, first create a MediaProjection instance and a VirtualDisplay object according to the method in S220, and create a Presentation component by overriding the constructor, and then associate the VirtualDisplay object with the Presentation component through the DisplayManager class callback interface.

[0090] S250 , calling a preset interface capture function to capture an application interface of a target application, so that the application interface is projected to a media display component.

[0091] In an embodiment of the present invention, taking the media display component as a Presentation component as an example, a SurfaceView component can be configured in the Presentation component. After obtaining the screen recording permission, the Surface object is obtained from the SurfaceView component, and the Surface object is assigned to the MediaProjection instance. Then, the application interface of the target application during runtime can be captured by calling the MediaProjection instance, so that the application interface is projected to the Presentation component. At this time, the view displayed in the Presentation component is the page to be projected.

[0092] S260: Call a preset media encoder to encode the view displayed in the media presentation component to obtain corresponding interconnected scene data.

[0093] In an embodiment of the present invention, a preset media encoder such as a MediaCodeC encoder can be created and configured with reference to the encoding process in S230, and then the MediaCodeC encoder is used to encode the view displayed in the Presentation component, that is, the page to be projected, to obtain the corresponding interconnected scene data.

[0094] S270: Exchange and negotiate WebRTC session description protocol information based on the point-to-point communication channel with the vehicle computer side to establish a real-time transport protocol channel.

[0095] Among them, the Session Description Protocol (SDP) is a format used to describe multimedia sessions. It provides information about the session, including the name, purpose, start and end time, media type, transmission protocol, encoding method, etc. of the session.

[0096] In the embodiment of the present invention, Figure 7 As shown, the terminal side can forward the created offer SDP (a set of SDP information describing local multimedia capabilities) to the vehicle side through the P2P channel based on the P2P channel that has been established between the terminal side and the vehicle side. After receiving the offer SDP, the vehicle side responds according to its own support capabilities, and creates an answer SDP and returns it to the terminal side, thereby completing the exchange and negotiation of WebRTC SDP information between the terminal side and the vehicle side, and establishing an RTP channel between the two.

[0097] S280: Transmit the interconnected scene data to the vehicle computer side through the real-time transmission protocol channel, so that the vehicle computer side decodes and displays the interconnected scene data.

[0098] In an embodiment of the present invention, the terminal side can transmit the interconnected scene data media stream to the vehicle side through the RTP channel, so that the vehicle side can decode and restore the interconnected scene data, and render and display the restored image frame on the display device to realize in-vehicle interconnection.

[0099] The technical solution of the embodiment of the present invention is applied to the terminal side. By responding to the first user interconnection operation, a Bluetooth communication connection and a point-to-point local area network communication connection are established with the vehicle side in sequence. Then, according to different application interconnection methods, the application interface of the target application is captured and encoded into corresponding interconnection scene data. Finally, the real-time transmission protocol channel established with the vehicle side is used to transmit the interconnection scene data to the vehicle side for decoding and display, so that the car manufacturer only needs to develop one set of in-vehicle interconnection solutions to meet the market and user needs, and the compatibility is high. At the same time, the developers can achieve layer-by-layer control from the upper application to the bottom layer, and the problem solving efficiency is high and the problem location is clear, thereby reducing the difficulty of accessing the in-vehicle interconnection scene and improving the development efficiency. In addition, the in-vehicle interconnection solution can also enrich the user's usage scenarios and improve the user's usage experience. At the same time, customized access can be performed according to user needs, and the scalability is higher.

[0100] Embodiment 3

[0101] Figure 8 A flowchart of a vehicle interconnection method is provided for the third embodiment of the present invention. This embodiment is applicable to the case where vehicle interconnection is performed between the vehicle side and the terminal side. The method can be executed by a vehicle interconnection device, which can be implemented in the form of hardware and / or software, and the vehicle interconnection device can be configured in an electronic device. Figure 8As shown, a vehicle-mounted interconnection method provided in the third embodiment is applied to the vehicle computer side and specifically includes the following steps:

[0102] S310: In response to the second user interconnection operation, establish a basic interconnection connection with the terminal side.

[0103] Among them, the second user interconnection operation can be understood as a series of user operations performed on the vehicle side in order to achieve basic interconnection between the vehicle side and the terminal side. For example, it may include but is not limited to: using a data cable to connect the vehicle side and the terminal side, turning on the Bluetooth function on the vehicle side, turning on the WiFi connection function on the vehicle side, etc.

[0104] In the embodiment of the present invention, refer to Figure 3 and Figure 4 The vehicle side can establish a Bluetooth communication connection and a P2P-based LAN communication connection with the terminal side in sequence, that is, establish a basic interconnection connection with the terminal side. The specific connection process can refer to the above embodiment and will not be repeated here.

[0105] S320, establishing a real-time transport protocol channel between the terminal side and WebRTC, and obtaining the interconnected scene data sent by the terminal side through the real-time transport protocol channel.

[0106] In the embodiment of the present invention, refer to Figure 7 Based on the P2P channel that has been established between the vehicle side and the terminal side, the vehicle side can establish an RTP channel between the two through the exchange and negotiation of WebRTC SDP information, and continuously receive the interconnected scene data sent by the terminal side through the RTP channel.

[0107] S330: Display the interconnected scene data on the target vehicle-mounted display device.

[0108] In an embodiment of the present invention, after receiving the interconnected scene data, the vehicle computer side can call a preset media decoder to decode and restore it, and render and display the decoded interconnected scene data on a target vehicle display device, such as a vehicle-mounted central control screen, thereby projecting the target application on the terminal side to the vehicle computer side to achieve vehicle-mounted interconnection.

[0109] Furthermore, based on the above-mentioned embodiments of the invention, this embodiment provides a vehicle interconnection method, further comprising:

[0110] Acquire the user's touch event on the target vehicle display device, and encapsulate the touch event into a reverse control data packet according to the pre-configured TLV protocol;

[0111] The reverse control data packet is sent to the terminal side, so that the terminal side controls the target application according to the reverse control data packet.

[0112] Among them, a touch event (MotionEvent) may refer to a MotionEvent object generated by the vehicle system when the user touches the screen of the target vehicle display device and captures the user's action. The MotionEvent object contains relevant information about the user's touch action, such as the coordinate position information, time information, pressure information, etc. of the touch.

[0113] TLV (Tag-Length-Value) protocol is a communication protocol that uses Tag, Length and Value triples to encode data, such as Fig. 9 As shown in the figure, a TLV unit consists of three parts: Tag, Length, and Value, which are described as follows:

[0114] Tag is used to identify the type or meaning of data, which is equivalent to "labeling" the Value field. It can be a numeric value, character, or other predefined identifier to indicate the specific meaning of the data that follows. In some specifications, a tag can be a single byte or multiple bytes, depending on the number and range of required identifiers.

[0115] Length is used to identify the length of the Value field. It can be one byte, two bytes or more, depending on the maximum possible length of the Value field. The Length field usually uses variable-length encoding. For a shorter Value field, fewer bits are used to represent its length, thus saving space.

[0116] Value is used to identify the actual data content. Its type and length are determined by the previous Tag field and Length field. It can be an integer, a string, a binary data block, etc. Since the TLV format allows the Value length to be variable, it can flexibly handle data of different types and lengths.

[0117] The embodiment of the present invention adopts a TLV type protocol format to ensure data integrity, which has at least the following advantages:

[0118] ① Extensibility: New types can be easily added without affecting existing data parsing;

[0119] ②Flexibility: Value can be any type of data, from simple values ​​to complex structures;

[0120] ③ Robustness: Even if some data cannot be parsed, the parser can skip this part of the data and continue to process the subsequent TLV units;

[0121] ④ Self-describing: Each TLV unit contains all the information required for processing and does not require external context or metadata.

[0122] In the embodiment of the present invention, Fig.10 As shown, after the application interface of the target application on the terminal side is projected to the vehicle side, the vehicle side can realize reverse control of the target application on the terminal side according to the user's touch operation on the target vehicle display device. Specifically, the vehicle side can obtain the user's touch event MotionEvent for the target vehicle display device, such as the vehicle central control screen, and then encapsulate it into a reverse control data packet through the TLV protocol; then send the reverse control data packet to the terminal side through the P2P channel pre-established with the terminal side. After receiving the reverse control data packet, the terminal side parses the touch event MotionEvent and replays it to simulate the user operation, thereby realizing reverse control of the target application, that is, the user's operation on the vehicle side can be responded to in real time on the terminal side.

[0123] It is understandable that after the vehicle leaves the factory, the hardware environment of the vehicle terminal is fixed and can no longer be upgraded. However, with the upgrading of vehicle-mounted software, the computing power requirements are getting higher and higher, which will cause the vehicle computer to run slower and slower. The vehicle-mounted interconnection method provided by the embodiment of the present invention can project the target application on the terminal side to the vehicle computer side, and the user can reversely operate the target application on the terminal side through the vehicle-mounted display device on the vehicle computer side, which saves the computing power of the vehicle terminal. At the same time, displaying the target application on the vehicle computer side can also bring a better viewing experience to the user.

[0124] The technical solution of the embodiment of the present invention is applied to the vehicle side, and establishes a basic interconnection connection with the terminal side by responding to the second user's interconnection operation; establishes a real-time transmission protocol channel between the terminal side based on WebRTC, and obtains the interconnection scene data sent by the terminal side through the real-time transmission protocol channel; and displays the interconnection scene data on the target vehicle display device. Utilizing this technical solution, car manufacturers only need to develop one set of vehicle interconnection solutions to meet market and user needs, with high compatibility; at the same time, developers can achieve layer-by-layer control from upper-layer applications to the bottom layer, with high efficiency in problem solving and clear problem location, thereby reducing the difficulty of accessing vehicle interconnection scenarios and improving development efficiency.

[0125] Embodiment 4

[0126] Fig.11 This is a schematic diagram of the structure of a vehicle-mounted interconnection device provided in Embodiment 4 of the present invention. Fig.11 As shown, the device is applied to the terminal side and includes:

[0127] The basic interconnection module 41 is used to establish a basic interconnection connection with the vehicle side in response to the first user interconnection operation;

[0128] A data determination module 42, for responding to a screen projection instruction for a target application, and determining corresponding interconnection scene data according to an application interconnection mode and an application interface of the target application;

[0129] The data transmission module 43 is used to establish a real-time transmission protocol channel between the vehicle computer side based on WebRTC, and send the interconnected scene data to the vehicle computer side for display through the real-time transmission protocol channel.

[0130] The technical solution of the embodiment of the present invention is applied to the terminal side, and responds to the first user interconnection operation through the basic interconnection module to establish a basic interconnection connection with the vehicle side; the data determination module responds to the screen projection instruction for the target application, and determines the corresponding interconnection scene data according to the application interconnection method and the application interface of the target application; the data transmission module establishes a real-time transmission protocol channel between the vehicle side based on WebRTC, and sends the interconnection scene data to the vehicle side for display through the real-time transmission protocol channel. After the terminal side in this solution establishes a basic interconnection connection with the vehicle side, it determines the corresponding interconnection scene data according to the application interconnection method and the application interface of the target application to be projected, and transmits the interconnection scene data to the vehicle side for display through the real-time transmission protocol channel established between WebRTC and the vehicle side, so that the car manufacturer only needs to develop a set of vehicle-mounted interconnection solutions to meet the market and user needs, and has high compatibility; at the same time, developers can realize layer-by-layer control from upper-layer applications to the bottom layer, and have high efficiency in solving problems and clear problem positioning, thereby reducing the difficulty of accessing vehicle-mounted interconnection scenarios and improving development efficiency.

[0131] Further, based on the above-mentioned embodiment of the invention, the basic interconnection module 41 includes:

[0132] The basic interconnection unit is used to establish a Bluetooth communication connection and a point-to-point local area network communication connection with the vehicle side in response to the first user interconnection operation.

[0133] Further, based on the above-mentioned embodiment of the invention, when the application interconnection mode is multi-screen simultaneous display, the data determination module 42 includes:

[0134] A screen content data acquisition unit, configured to, in response to a screen projection instruction for a target application, call a preset interface capture function to capture an application interface of the target application to obtain screen content data;

[0135] The first interconnected scene data acquisition unit is used to call a preset media encoder to encode the screen content data to obtain corresponding interconnected scene data.

[0136] Further, based on the above-mentioned embodiment of the invention, when the application interconnection mode is multi-screen different display, the data determination module 42 includes:

[0137] A component association unit, configured to create a media presentation component and a virtual display screen object in response to a screen projection instruction for a target application, and associate the virtual display screen object with the media presentation component;

[0138] An application interface capturing unit, used for calling a preset interface capturing function to capture the application interface of a target application, so that the application interface is projected to a media display component;

[0139] The second interconnected scene data acquisition unit is used to call a preset media encoder to encode the view displayed in the media presentation component to obtain corresponding interconnected scene data.

[0140] Further, based on the above-mentioned embodiment of the invention, the data transmission module 43 includes:

[0141] A real-time transport protocol channel establishment unit, used to exchange and negotiate WebRTC session description protocol information based on a point-to-point communication channel with the vehicle side to establish a real-time transport protocol channel;

[0142] The data transmission unit is used to transmit the interconnected scene data to the vehicle computer side through the real-time transmission protocol channel, so that the vehicle computer side decodes and displays the interconnected scene data.

[0143] The in-vehicle interconnection device provided in the embodiment of the present invention can execute the in-vehicle interconnection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0144] Embodiment 5

[0145] Fig.12 This is a schematic diagram of the structure of a vehicle-mounted interconnection device provided in Embodiment 5 of the present invention. Fig.12 As shown, the device is applied to the vehicle side and includes:

[0146] A basic interconnection module 51, configured to establish a basic interconnection connection with the terminal side in response to a second user interconnection operation;

[0147] The data acquisition module 52 is used to establish a real-time transport protocol channel with the terminal side based on WebRTC, and obtain the interconnection scene data sent by the terminal side through the real-time transport protocol channel;

[0148] The data display module 53 is used to display the interconnected scene data on the target vehicle display device.

[0149] The technical solution of the embodiment of the present invention is applied to the vehicle computer side, and responds to the second user's interconnection operation through the basic interconnection module to establish a basic interconnection connection with the terminal side; the data acquisition module establishes a real-time transmission protocol channel with the terminal side based on WebRTC, and obtains the interconnection scene data sent by the terminal side through the real-time transmission protocol channel; the data display module displays the interconnection scene data on the target vehicle display device. Utilizing this technical solution, car manufacturers only need to develop one set of vehicle interconnection solutions to meet market and user needs, with high compatibility; at the same time, developers can achieve layer-by-layer control from upper-layer applications to the bottom layer, with high efficiency in problem solving and clear problem location, thereby reducing the difficulty of accessing vehicle interconnection scenarios and improving development efficiency.

[0150] Further, based on the above-mentioned embodiment of the invention, the data display module 53 includes:

[0151] The data display unit is used to call a preset media decoder to decode the interconnected scene data, and render and display the decoded interconnected scene data on a target vehicle-mounted display device.

[0152] Furthermore, based on the above-mentioned embodiments of the invention, the vehicle-mounted interconnection device further includes:

[0153] A touch event encapsulation module is used to obtain a user's touch event on a target vehicle-mounted display device and encapsulate the touch event into a reverse control data packet according to a pre-configured TLV protocol;

[0154] The reverse control module is used to send the reverse control data packet to the terminal side so that the terminal side controls the target application according to the reverse control data packet.

[0155] The in-vehicle interconnection device provided in the embodiment of the present invention can execute the in-vehicle interconnection method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0156] Embodiment 6

[0157] Fig.13 A schematic diagram of the structure of an electronic device 60 that can be used to implement an embodiment of the present invention is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.) and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or required herein.

[0158] like Fig.13 As shown, the electronic device 60 includes at least one processor 61, and a memory connected to the at least one processor 61, such as a read-only memory (ROM) 62, a random access memory (RAM) 63, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 61 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 62 or the computer program loaded from the storage unit 68 to the random access memory (RAM) 63. In the RAM 63, various programs and data required for the operation of the electronic device 60 can also be stored. The processor 61, the ROM 62, and the RAM 63 are connected to each other via a bus 64. An input / output (I / O) interface 65 is also connected to the bus 64.

[0159] A number of components in the electronic device 60 are connected to the I / O interface 65, including: an input unit 66, such as a keyboard, a mouse, etc.; an output unit 67, such as various types of displays, speakers, etc.; a storage unit 68, such as a disk, an optical disk, etc.; and a communication unit 69, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 69 allows the electronic device 60 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0160] The processor 61 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the processor 61 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 61 executes the various methods and processes described above, such as the vehicle interconnection method.

[0161] In some embodiments, the in-vehicle interconnection method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 68. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 60 via the ROM 62 and / or the communication unit 69. When the computer program is loaded into the RAM 63 and executed by the processor 61, one or more steps of the in-vehicle interconnection method described above may be performed. Alternatively, in other embodiments, the processor 61 may be configured to execute the in-vehicle interconnection method in any other appropriate manner (e.g., by means of firmware).

[0162] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0163] Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions / operations specified in the flow chart and / or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0164] In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in combination with an instruction execution system, device or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0165] To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

[0166] The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

[0167] A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

[0168] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

[0169] The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A vehicle interconnection method, characterized in that: Applied to the terminal side, the method includes: In response to the first user interconnection operation, establishing a basic interconnection connection with the vehicle computer side; In response to a screen projection instruction for a target application, determining corresponding interconnection scene data according to an application interconnection mode and an application interface of the target application; A real-time transport protocol channel is established between WebRTC and the vehicle computer side, and the interconnected scene data is sent to the vehicle computer side for display through the real-time transport protocol channel.

2. The method according to claim 1, characterized in that The step of establishing a basic interconnection connection with the vehicle computer side in response to the first user interconnection operation includes: In response to the first user interconnection operation, a Bluetooth communication connection and a point-to-point local area network communication connection are established with the vehicle computer side in sequence.

3. The method according to claim 1, characterized in that When the application interconnection mode is multi-screen co-display, the response to the screen projection instruction for the target application is to determine the corresponding interconnection scene data according to the application interconnection mode and the application interface of the target application, including: In response to a screen projection instruction for the target application, calling a preset interface capture function to capture an application interface of the target application to obtain screen content data; A preset media encoder is called to encode the screen content data to obtain the corresponding interconnected scene data.

4. The method according to claim 1, characterized in that: When the application interconnection mode is multi-screen different display, the response to the screen projection instruction of the target application, determining the corresponding interconnection scene data according to the application interconnection mode and the application interface of the target application, includes: In response to a screen projection instruction for the target application, create a media presentation component and a virtual display screen object, and associate the virtual display screen object with the media presentation component; Calling a preset interface capture function to capture the application interface of the target application, so that the application interface is projected to the media display component; A preset media encoder is called to encode the view displayed in the media presentation component to obtain the corresponding interconnected scene data.

5. The method according to claim 1, characterized in that The establishing of a real-time transport protocol channel between the WebRTC and the vehicle computer side, and sending the interconnected scene data to the vehicle computer side for display through the real-time transport protocol channel, includes: Exchanging and negotiating WebRTC session description protocol information based on the point-to-point communication channel between the vehicle computer side to establish the real-time transport protocol channel; The interconnected scene data is transmitted to the vehicle computer side through the real-time transport protocol channel, so that the vehicle computer side decodes and displays the interconnected scene data.

6. A vehicle-mounted interconnection method, characterized in that: Applied to the vehicle computer side, the method includes: In response to the second user interconnection operation, establishing a basic interconnection connection with the terminal side; Establishing a real-time transport protocol channel between the terminal side and WebRTC, and obtaining the interconnection scene data sent by the terminal side through the real-time transport protocol channel; The interconnected scene data is displayed on a target vehicle-mounted display device.

7. The method according to claim 6, characterized in that The displaying of the interconnected scene data on the target vehicle-mounted display device includes: A preset media decoder is called to decode the interconnected scene data, and the decoded interconnected scene data is rendered and displayed on the target vehicle-mounted display device.

8. The method according to claim 6, characterized in that Also includes: Acquire a touch event of a user on the target vehicle-mounted display device, and encapsulate the touch event into a reverse control data packet according to a pre-configured TLV protocol; The reverse control data packet is sent to the terminal side, so that the terminal side controls the target application according to the reverse control data packet.

9. A vehicle-mounted interconnection device, characterized in that: Applied to the terminal side, the device includes: A basic interconnection module, configured to establish a basic interconnection connection with the vehicle computer side in response to a first user interconnection operation; A data determination module, for responding to a screen projection instruction for a target application, and determining corresponding interconnection scene data according to an application interconnection mode and an application interface of the target application; The data transmission module is used to establish a real-time transport protocol channel with the vehicle side based on WebRTC, and send the interconnected scene data to the vehicle side for display through the real-time transport protocol channel.

10. A vehicle-mounted interconnection device, characterized in that: Applied to the vehicle side, the device includes: A basic interconnection module, configured to establish a basic interconnection connection with the terminal side in response to an interconnection operation of a second user; A data acquisition module, used to establish a real-time transport protocol channel with the terminal side based on WebRTC, and acquire the interconnected scene data sent by the terminal side through the real-time transport protocol channel; The data display module is used to display the interconnected scene data on a target vehicle display device.

11. An electronic device, characterized in that: The electronic device comprises: at least one processor; and a memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the in-vehicle interconnection method described in any one of claims 1-8.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the in-vehicle interconnection method according to any one of claims 1 to 8 when executed.

Images