Vehicle control method and device, equipment, storage medium and product

Abstract

The invention discloses a vehicle control method, device and equipment, a storage medium and a product, which are applied to the technical field of vehicle control, and the method comprises the following steps: if a short-range data transmission channel and a screen projection transmission channel are established between a mobile terminal and a vehicle terminal integrated with interconnection software, obtaining first vehicle data transmitted by the vehicle terminal through the short-range data transmission channel; based on the first vehicle data, generating a handcart interconnection page which is adaptive to a vehicle machine display screen of the vehicle machine end and has a super space in a super space module of the mobile end, and projecting the handcart interconnection page to the vehicle machine display screen through a screen projection transmission channel; obtaining a touch event which is reported by the vehicle terminal through the short-range data transmission channel and aims at the handcart interconnection page; and determining a vehicle control instruction corresponding to the touch event, and sending the vehicle control instruction to the vehicle terminal through the short-range data transmission channel by using the target message transmission protocol, so that the vehicle terminal transmits the vehicle control instruction to an execution unit corresponding to the vehicle control instruction for instruction execution after receiving the vehicle control instruction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of vehicle control, and in particular to a vehicle control method, device, equipment, storage medium and product. BACKGROUND

[0002] With the iteration and upgrading of multi-terminal interconnection technology in the intelligent field, current intelligent vehicles generally carry hand-car interconnection functions, and have realized seamless connection, video stream conversion and partial data and computing power sharing between mobile terminals such as mobile phones and vehicle machines. From the market status, the hand-car interconnection solution presents a diversified development trend, mainly covering three types of connection methods based on mobile phones as the core, communication links and account systems.

[0003] Due to the barriers between mobile terminal manufacturers and automobile manufacturers, the current interconnection between the vehicle machine end and the mobile terminal only shares the display screen of the vehicle machine end to the mobile terminal, and the mobile terminal operates the vehicle machine end to realize remote control of the vehicle machine end. Or the navigation interface and video playback interface of the mobile terminal can be projected to the display screen of the vehicle machine end, and the navigation interface of the mobile terminal is displayed through the display screen for the convenience of the driver to view and operate, or the video playback interface of the mobile terminal is displayed through the display screen for the convenience of the rear seat or co-driver to watch videos.

[0004] However, the application of interconnection between the vehicle machine end and the mobile terminal also has limitations. SUMMARY

[0005] One of the purposes of the present application is to provide a vehicle control method, device, equipment, storage medium and product.

[0006] In order to achieve the above-mentioned purpose, the technical scheme of the embodiments of the present application is as follows:

[0007] In a first aspect, the present application provides a vehicle control method applied to a mobile terminal, the method comprising:

[0008] If the mobile terminal and the vehicle machine end integrated with the interconnection software establish a short-range data transmission channel and a screen projection transmission channel, the first vehicle data transmitted by the vehicle machine end through the short-range data transmission channel is obtained; wherein the short-range data transmission channel and the screen projection transmission channel are two different transmission channels established using the same communication module, the message transmission protocol of the short-range data transmission channel is a target message transmission protocol, and the target message transmission protocol is a message queue telemetry transmission protocol;

[0009] Based on the first vehicle data, a hand-car interconnection page with a super space adapted to the vehicle machine display screen of the vehicle machine end is generated in the super space module of the mobile terminal, and the hand-car interconnection page is projected to the vehicle machine display screen through the screen projection transmission channel, wherein the super space module is a functional module capable of generating vehicle state information and vehicle functions for display and controlling the vehicle functions.

[0010] obtaining a touch event for a hand-vehicle interconnection page reported by the vehicle terminal through the short-range data transmission channel;

[0011] determining a vehicle control instruction corresponding to the touch event, and sending the vehicle control instruction to the vehicle terminal through the short-range data transmission channel using a target message transmission protocol, so that the vehicle terminal receives the vehicle control instruction and transmits it to an execution unit corresponding to the vehicle control instruction for instruction execution.

[0012] According to the above technical means, due to the interconnection of the vehicle terminal and the mobile terminal, the short-range data transmission channel and the screen projection transmission channel are established between the vehicle terminal and the mobile terminal, and data and screen projection are transmitted in separate channels. Further, the mobile terminal sets the core control logic software of the vehicle in the mobile terminal. After the mobile terminal obtains the first vehicle data through the short-range data transmission channel, the mobile terminal generates a hand-vehicle interconnection page with a super space based on the first vehicle data, and the page is adapted to the vehicle terminal display screen. The hand-vehicle interconnection page with the super space function is projected to the display screen of the vehicle terminal for display and touch control. The mobile terminal analyzes and generates a vehicle control instruction based on the touch instruction, and sends it to the vehicle terminal through the short-range data transmission channel to control the corresponding function. In this way, the deployment of the intelligent control function application of the vehicle terminal can be saved, and the user can be provided with an immersive touch experience of the vehicle terminal in the super space. The intelligent control function of the vehicle terminal gradually increases, and does not affect the load of the vehicle terminal. The influence of the function and performance of the vehicle terminal can be avoided, the function development is not limited, the development of the vehicle terminal software is reduced, the vehicle terminal computing power resources are saved, the resource sharing of the mobile terminal and the vehicle terminal is realized, and the software of the function is iterated quickly and efficiently in the mobile terminal.

[0013] In a second aspect, the present application provides a vehicle control device, which comprises:

[0014] The obtaining module is configured to, if the mobile terminal and the vehicle terminal integrated with the interconnection software establish a short-range data transmission channel and a screen projection transmission channel, obtain first vehicle data transmitted by the vehicle terminal through the short-range data transmission channel. The short-range data transmission channel and the screen projection transmission channel are two different transmission channels established using the same communication module. The message transmission protocol of the short-range data transmission channel is a target message transmission protocol, and the target message transmission protocol is a message queue telemetry transmission protocol.

[0015] The processing module is configured to, based on the first vehicle data, generate a hand-vehicle interconnection page with a super space in a super space module of the mobile terminal, which is adapted to the vehicle terminal display screen, and project the hand-vehicle interconnection page to the vehicle terminal display screen through the screen projection transmission channel. The super space module is a function module capable of generating vehicle state information and vehicle functions for display and control.

[0016] The obtaining module is further configured to obtain a touch event for the vehicle-mobile interconnection page reported by the vehicle terminal through the short-range data transmission channel.

[0017] The determining module is configured to determine a vehicle control instruction corresponding to the touch event.

[0018] The sending module is configured to send the vehicle control instruction to the vehicle terminal through the short-range data transmission channel by using the target message transmission protocol, so that the vehicle terminal receives the vehicle control instruction and delivers the vehicle control instruction to an execution unit corresponding to the vehicle control instruction for instruction execution.

[0019] In a third aspect, the present application provides a computer device, comprising a transceiver, a memory and a processor,

[0020] The memory stores a computer program capable of running on the processor.

[0021] The transceiver is configured to perform the operation of receiving data or sending data under the control of the processor.

[0022] The processor implements part or all steps of the vehicle control method according to the first aspect when executing the computer program.

[0023] In a fourth aspect, the present application provides a computer readable storage medium, which stores one or more computer programs, and the one or more computer programs can be executed by one or more processors to implement part or all steps of the vehicle control method according to the first aspect.

[0024] In a fifth aspect, the present application provides a computer program product, comprising a computer program or instructions, which are executed by a processor to implement part or all steps of the vehicle control method according to the first aspect.

[0025] The beneficial effects of the embodiments of the present application are as follows:

[0026] (1) The data transmission channel is established independently, and is decoupled from the mobile phone manufacturer, having strong independence and expansibility.

[0027] (2) The control link can be intelligently selected according to WIFI and 4G / 5G network information, to complete long-range and short-range fusion, and the long-range and short-range protocols both use the MQTT standardized interface, which can be reused.

[0028] (3) The vehicle terminal receives the control instruction sent by the mobile terminal, delivers the control instruction to the vehicle semantic AI module, can complete intelligent semantic analysis, and reuses the voice execution link to reduce the occupation of vehicle resources. BRIEF DESCRIPTION OF DRAWINGS

[0029] The drawings incorporated into the specification and forming a part of the specification, illustrate embodiments consistent with the present application and together with the description, serve to explain the principles of the present application.

[0030] Figure 1 A network architecture schematic diagram of an optional vehicle control method provided for the embodiments of the present application;

[0031] Figure 2 A structural block diagram of an optional vehicle control system provided for the embodiments of the present application;

[0032] Figure 3 An implementation flow schematic diagram of an optional vehicle control method provided for the embodiments of the present application Figure 1 ;

[0033] Figure 4 A page schematic diagram of an optional hand vehicle interconnection interface provided for the embodiments of the present application;

[0034] Figure 5 An implementation flow schematic diagram of an optional vehicle control method provided for the embodiments of the present application Figure 2 ;

[0035] Figure 6 An implementation flow schematic diagram of an optional vehicle control method provided for the embodiments of the present application Figure 3 ;

[0036] Figure 7 A structural schematic diagram of an optional vehicle control device provided for the embodiments of the present application;

[0037] Figure 8 A hardware structure schematic diagram of an optional computer device provided for the embodiments of the present application. DETAILED DESCRIPTION

[0038] In order to make the purposes, technical solutions and advantages of the present application clearer, the technical solutions of the present application are further described in detail below in combination with the drawings and embodiments, and the described embodiments should not be regarded as limiting the present application, and all other embodiments obtained by those of ordinary skill in the art without making creative efforts fall within the scope of protection of the present application.

[0039] In the following description, "some embodiments" are referred, which describe a subset of all possible embodiments, but it can be understood that "some embodiments" can be the same subset or different subset of all possible embodiments, and can be combined with each other without conflict. The term "first / second / third" referred to only distinguishes similar objects, and does not represent a specific order for the objects. Understandably, "first / second / third" can be interchanged with a specific order or sequence as allowed, so that the embodiments of the present application described herein can be implemented in an order other than that illustrated or described herein.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this application belongs. The terminology used herein is for the purpose of describing the present application only and is not intended to be limiting of the present application.

[0041] With the iteration and upgrading of multi-terminal interconnection technology in the field of intelligence, current intelligent vehicles generally carry hand-car interconnection functions, and have realized the seamless connection of mobile terminals such as mobile phones and vehicle machines, video stream conversion, and partial data and computing power sharing. From the market status, the hand-car interconnection solution presents a diversified development trend, mainly covering three types of connection methods based on mobile phones as the core, communication links, and account systems.

[0042] Due to the barriers between mobile terminal manufacturers and automobile manufacturers, the current interconnection between the vehicle machine end and the mobile terminal only shares the display screen of the vehicle machine end to the mobile terminal, and operates the controls displayed on the display screen of the vehicle machine end through the mobile terminal to realize remote control of the vehicle machine end. Or the navigation interface, video playing interface, etc. of the mobile terminal can be projected to the display screen of the vehicle machine end, and the navigation interface of the mobile terminal is displayed through the display screen for the convenience of the driver to view and operate, or the video playing interface of the mobile terminal is displayed through the display screen for the convenience of the rear seat or co-driver to watch videos.

[0043] The application between the vehicle machine and the mobile terminal is also limited. For example, at least the following problems exist: first, due to the large number of mobile terminal brands and the uneven function gradient, effective platformization cannot be achieved. At present, it is impossible to realize the immersive space experience and intelligent control of the vehicle end through the hand-vehicle interconnection system. The mainstream interconnection methods include HiCar, Carlink, HONOR CarConnect, etc. This interconnection method is usually limited to mobile phones and cars under the same brand, and the authorization of the car manufacturer is required to realize the transmission of key data between the vehicle machine and the mobile terminal. This method at least has the problems of low device compatibility and complex user operation. Second, although the vehicle machine end has realized the basic vehicle control function, the interactive interface design is single, and the user's immersive experience is poor; and in the hand-vehicle interconnection scene, the user needs to slide the hand-vehicle interconnection interface to the negative screen or minimize the hand-vehicle interconnection interface to enter the vehicle control interface of the vehicle machine, so as to realize the control of the vehicle control function. This method at least has the problems of operation load and low efficiency. Third, the vehicle machine end needs to carry the human-computer interaction front-end interface, and with the continuous expansion and development of functions, the operation load of the vehicle machine end gradually approaches the limit, resulting in slow function response speed and limited overall performance of the vehicle machine end, which further affects the user experience.

[0044] To solve one or more of the above problems, the embodiments of the present application propose a vehicle control method, which establishes a double transmission channel architecture of a projection transmission channel and a short-range data transmission channel between the mobile terminal and the vehicle machine, and realizes data and projection transmission in separate channels. Further, after the mobile terminal obtains vehicle data through the short-range data transmission channel, the mobile terminal generates a hand-vehicle interconnection page with a super space based on the vehicle data, and the page is adapted to the vehicle machine display screen, and the hand-vehicle interconnection page with the super space function is projected to the display screen of the vehicle machine end for display and touch control. The mobile terminal analyzes and generates a vehicle control instruction based on the touch instruction, and sends it to the vehicle machine end through the short-range data transmission channel to control the corresponding function; in this way, the core control logic software of the vehicle is transferred to the mobile terminal, the deployment of the intelligent control function application of the vehicle machine end is saved, and the user can also be provided with an immersive touch experience under the super space of the vehicle machine end. The intelligent control function of the vehicle end gradually increases.

[0045] Figure 1 Fig. 1 is a network architecture schematic diagram of a vehicle control method provided in the embodiments of the present application. The network architecture includes a mobile terminal 100, a vehicle machine 200 and a cloud end 300; wherein the mobile terminal 100 and the vehicle machine 200 can be connected through a network such as a local area network, and the mobile terminal 100 and the vehicle machine 200 are connected with the cloud end 300 through a network such as a wide area network.

[0046] Here, the local area network can be a Wireless Fidelity (WIFI) network, and the wide area network can be a 4th Generation Mobile Communication Technology (4G) network, a 5th Generation Mobile Communication Technology (5G) network, a 6th Generation Mobile Communication Technology (6G) network, or a new mobile communication technology network to appear in the future, and the present application does not make specific limitations.

[0047] Referring to Figure 2 Fig. 1 shows a structural block diagram of a vehicle control system provided by an embodiment of the present application, wherein the mobile terminal 100 can be referred to as a mobile device connected with the vehicle terminal 200, wherein the mobile terminal 100 can support an application (APP), a long-range communication module, a short-range communication module, and a hand-to-car interconnection APP containing a long-and-short-range fusion module, wherein the application includes but is not limited to a navigation APP and a music APP, the long-range communication module includes but is not limited to a 4G / 5G module, and the short-range communication module includes but is not limited to a Bluetooth Low Energy (BLE) module and a WIFI module. The mobile terminal 100 includes but is not limited to a smart phone, a tablet computer, a Personal Digital Assistant (PDA), a camera, a wearable device, a smart television, a smart camera, a smart projector, a laptop computer, and a desktop computer, etc.

[0048] Here, continuing to refer to Figure 2 , the vehicle terminal 200 can be an in-vehicle infotainment system installed in the interior of a car, and can realize information communication between a person and a car and between a car and the outside world. The vehicle terminal 200 can include a vehicle application, a display screen, support for screen projection input, a long-range communication module, a short-range communication module, an audio module, an image module, and a display device of a hand-to-car interconnection APP. Of course, the vehicle terminal 200 can also include a core service, a Bluetooth phone, a vehicle desktop launcher (Launcher), a system user interface component (System UI), a business component, etc. The vehicle application includes but is not limited to a music APP, a voice / semantic analysis module, a navigation APP, and a video APP, the long-range communication module includes but is not limited to a 4G / 5G module, and the short-range communication module includes but is not limited to a Bluetooth module and a WIFI module.

[0049] Here, the cloud 300 can refer to a terminal capable of completing remote control logic, data transmission and storage. The cloud 300 can be deployed with a server, which can be a single server or a server cluster composed of multiple servers, a cloud computing center, etc.

[0050] Here, continuing to refer to Figure 2 , the car machine end 200 can also be in communication connection with the vehicle end execution device 400, wherein the vehicle end execution device 400 includes a vehicle body controller 410 and an execution unit 420. The car machine end 200 can be connected with the vehicle end execution device 400, so that the car machine end transmits the vehicle control instruction to the vehicle body controller 410. The vehicle body controller 410 can refer to a domain controller that completes the vehicle control execution unit control logic. The vehicle body controller 410 is used to control the execution unit 420 to perform the corresponding action according to the vehicle control instruction. The execution unit 420 can be an actuator such as an air conditioner, a window, a door, an atmosphere lamp, a lock, a sunroof, and a rearview mirror connected with the vehicle body controller 410.

[0051] Referring to Figure 3 , Figure 3 , an implementation flow diagram of a vehicle control method provided by the embodiment of the present application is shown. The method can be executed by the processor of the mobile terminal in Figure 1 and Figure 2 . Here, the steps shown in Figure 3 will be described,

[0052] Step 301, if the mobile terminal and the car machine end integrated with the interconnection software establish a short-range data transmission channel and a screen projection transmission channel, obtain the first vehicle data transmitted by the car machine end through the short-range data transmission channel.

[0053] Among them, the short-range data transmission channel and the screen projection transmission channel are two different transmission channels established by using the same communication module. The message transmission protocol of the short-range data transmission channel is a target message transmission protocol, and the target message transmission protocol is a message queue telemetry transmission protocol.

[0054] In the embodiment of the present application, the car machine end is installed and integrated with the interconnection software, also known as the interconnection APP. The mobile terminal and the car machine end can establish a screen projection transmission channel by using the interconnection software for the car machine end by the automobile manufacturer, and the message transmission protocol and the data transmission format used by the BLE and WIFI communication modules to transmit messages, realize the interconnection between the mobile terminal and the car machine end, and then realize the screen projection of the interface of the mobile terminal to the car machine end through the screen projection transmission channel. Optionally, the interconnection software includes but is not limited to Hicar, Carlink, and HONOR CarConnect.

[0055] In the embodiments of the present application, the short-range data transmission channel can be a data transmission channel established by the mobile terminal and the vehicle terminal based on a target message transmission protocol through the WiFi module. The target message transmission protocol can be a Message Queuing Telemetry Transport (MQTT) protocol. For example, with reference to the above example Figure 1 The short-range data transmission channel can be a short-range WIFI transmission channel established based on the MQTT protocol through the WiFi module.

[0056] In the embodiments of the present application, the first vehicle data includes vehicle body component state data and vehicle detection data. The vehicle body component state data is collected or calculated by the vehicle through sensors and controllers in real time, and is used to reflect the quantitative information of the running state, health degree and function of the key components of the vehicle body. It is the core basis for vehicle diagnosis, safety warning and intelligent control. It should be noted that the vehicle body component state data is the data of the vehicle body components that can be actively operated by the user. The vehicle detection data is a quantitative index or key information that directly reflects the basic running ability, safety state and key function performance of the vehicle, and is the core basis for vehicle state monitoring and user's understanding of the core situation of the vehicle. This data is usually collected or calculated by the core assemblies of the vehicle power system, chassis system and energy system in real time, and the user cannot directly change its value through simple operation. It needs to be adjusted indirectly through refueling / charging, maintenance, etc. Its role is to help the user judge whether the vehicle has normal driving conditions, whether there is a safety risk, and plan a travel strategy.

[0057] In the embodiments of the present application, the first vehicle data can be obtained by collecting data through sensors, cameras, radars (such as laser radars, ultrasonic radars, etc.) installed on the vehicle. These data can include images, videos, radar scanning data, inertial measurement unit (IMU) data, temperature, humidity, vehicle speed, etc. Then the above data is transmitted to the vehicle terminal or the engine control unit (ECU) through the vehicle bus, or the vehicle can be sent to the mobile terminal through cellular network or other network communication (such as satellite communication). The obtained data information can be stored in the internal storage medium of the vehicle, or stored in the cloud or mobile terminal connected to the vehicle through network connection.

[0058] In the embodiments of the present application, the data transmission format corresponding to the target message transmission protocol is a data format agreed between the mobile terminal and the vehicle terminal. The target message transmission protocol can be an MQTT protocol, which is a lightweight Internet of Things communication protocol based on a publish / subscribe mode. The MQTT protocol format is a JSON (JavaScript Object Notation) serialized MQTT message topic cluster. By way of example, the mobile terminal is a client, the vehicle terminal is a server, and the topic cluster of the JSON serialized MQTT message is shown in Table 1.

[0059] Table 1

[0060]

[0061]

[0062] In the embodiments of the present application, the JSON message structure corresponding to the MQTT protocol includes header (Header) information and payload (payload) information, also known as data (data) information, wherein the data information uses a JSON string. Here, the message fields involved in the JSON message structure corresponding to the MQTT protocol and the corresponding descriptions are shown in Table 2.

[0063] Table 2

[0064]

[0065] In step 302, based on the first vehicle data, a hand-vehicle interconnection page with a super space adapted to the vehicle display screen of the vehicle terminal is generated in the super space module of the mobile terminal, and the hand-vehicle interconnection page is projected to the vehicle display screen through a projection transmission channel.

[0066] Among them, the super space module is a functional module that can generate a function module for displaying vehicle state information and vehicle functions, and controlling the vehicle functions.

[0067] In the embodiments of the present application, the mobile terminal is installed with an application program of the super space module, and the super space module includes one or more of the following elements: wallpaper, 3D (Three-Dimensional) car model, vehicle condition card and car control card.

[0068] In the embodiment of the present application, after the mobile terminal obtains the first vehicle data of the vehicle terminal through the target data transmission channel, the mobile terminal can generate a hand-in-vehicle interconnection page with a super space in the super space module of the mobile terminal based on the first vehicle data, adjust the hand-in-vehicle interconnection page according to the display ratio of the vehicle display screen of the vehicle terminal, generate a hand-in-vehicle interconnection page adapted to the vehicle display screen of the vehicle terminal, and project the hand-in-vehicle interconnection page with a super space to the vehicle display screen through the projection transmission channel. For example, as shown in Figure 4 Figure 4 The hand-in-vehicle interconnection page with a super space in the mobile terminal and vehicle terminal interconnection mode is a page schematic diagram, which includes a wallpaper (not shown in the figure), a three-dimensional vehicle model 41, a vehicle condition card 42, and a vehicle control card 43. In this way, the super space content of the mobile terminal is synchronously displayed on the vehicle display screen of the vehicle terminal, which is convenient for display and user operation, and the user can enjoy an immersive experience on the projection interface.

[0069] Step 303: Obtain the touch event of the hand-in-vehicle interconnection page reported by the vehicle terminal through the short-range data transmission channel.

[0070] In the embodiment of the present application, the vehicle terminal listens to the touch instruction of the user for the hand-in-vehicle interconnection page displayed on the vehicle display screen, such as the touch event of clicking the opening rearview mirror folding button. The vehicle terminal sends it to the mobile terminal through the short-range data transmission channel, which is analyzed and processed by the mobile terminal, and then determines the control of the vehicle.

[0071] Step 304: Determine the vehicle control instruction corresponding to the touch event, and send the vehicle control instruction to the vehicle terminal through the short-range data transmission channel by using the target message transmission protocol, so that the vehicle terminal receives the vehicle control instruction and transmits it to the execution unit corresponding to the vehicle control instruction for instruction execution.

[0072] In the embodiment of the present application, the control logic software of the super space of the vehicle terminal is set in the mobile terminal, and then the mobile terminal and the vehicle terminal can be interconnected. The mobile terminal can project the hand-in-vehicle interconnection page with a super space to the vehicle terminal through the projection transmission channel, and display and touch based on the vehicle display screen of the vehicle terminal. The super space is run by the mobile terminal, and the touch event of the hand-in-vehicle interconnection page of the vehicle display screen is transmitted to the mobile terminal through the short-range data transmission channel. The mobile terminal analyzes and processes the touch event, generates a vehicle control instruction corresponding to the touch event, and sends the vehicle control instruction to the vehicle terminal through the short-range data transmission channel by using the target message transmission protocol. After receiving the vehicle control instruction, the vehicle terminal transmits the vehicle control instruction to the vehicle body controller and the corresponding execution unit for instruction execution. After the execution unit finishes execution, the execution result is fed back to the vehicle terminal, and the vehicle terminal feeds back the execution result to the mobile terminal.

[0073] ​From the above, the embodiment of the application can realize touch on the display screen of the vehicle machine end, the mobile terminal identifies and processes the touch event, generates corresponding vehicle control instructions, and then re-sends the vehicle control instructions to the execution component of the vehicle machine end for response. Based on this, the mobile terminal as the control core subject, compared with the current control mode taking the vehicle machine end as the subject, can not only save the deployment of the intelligent control function application of the vehicle machine end, but also provide the user with an immersive touch experience under the super space of the vehicle machine end, the intelligent control function of the vehicle end gradually increases, and the load of the vehicle machine end is not affected, the influence of the function and performance of the vehicle machine end can be avoided, and the function development is not limited; in addition, the development of the vehicle machine software is reduced, the vehicle machine computing resource is saved, the resource sharing of the mobile terminal and the vehicle machine end is realized, and the rapid and efficient iteration of the software of the function on the mobile terminal is realized. At the same time, based on the interconnection of the vehicle machine end and the mobile terminal, other function applications in the intelligent cabin can also be set on the mobile terminal, and the resources of the mobile terminal are used to run these function applications. In addition, the mobile terminal can also realize real-time connection with the cloud end, and the cloud end resources are used to process data, which can reduce the hardware cost of the vehicle end. The embodiment of the application increases the application scene of the interconnection of the mobile terminal and the vehicle machine end, and increases the universality. In addition, the application program is deployed on the mobile terminal, the performance requirement of the vehicle machine end is reduced, as long as the vehicle has a responding vehicle body controller and an execution unit, even if the performance of the vehicle machine end is low, the intelligent control function can also be realized through the mobile terminal, therefore, the embodiment of the application can be applied to high-end / low-end vehicles (low-cost vehicles), and high-end services are provided for low-end vehicle users, and the use experience of the vehicle is improved.

[0074] In some embodiments, the establishment process of the screen projection transmission channel and the short-range data transmission channel in step 301 includes:

[0075] Step 401, in response to the mobile terminal scanning the Bluetooth device in the vehicle machine end, displaying a Bluetooth pairing code, and sending a Bluetooth pairing request carrying the Bluetooth pairing code to the vehicle machine end, so that the vehicle machine end automatically verifies according to the Bluetooth pairing code to complete pairing and opens a WiFi hotspot; in response to the mobile terminal connecting to the WiFi hotspot of the vehicle machine end, determining that the mobile terminal establishes a screen projection transmission channel with the vehicle machine end through the WiFi hotspot.

[0076] In the embodiment of the application, the screen projection transmission channel can be a WiFi transmission channel established by the mobile terminal and the vehicle machine end based on the message transmission protocol and the corresponding data transmission format used by the WIFI communication module predefined by the automobile manufacturer. The screen projection transmission channel can be used for screen projection and data transmission. It should be noted that when data is transmitted using the screen projection transmission channel, authorization of the automobile manufacturer is required to obtain the message transmission protocol and encryption and decryption methods.

[0077] In the embodiment of the present application, the user is not aware of the whole process of establishing the screen projection transmission channel, thereby bringing convenience and better experience to the user. In a kind of realizable scene, the establishment process of the screen projection transmission channel is as follows: in the case that the user opens Bluetooth in mobile terminal and vehicle terminal, the vehicle terminal calls Bluetooth module through interconnection software, so that the Bluetooth module continuously sends Bluetooth broadcast, the mobile terminal scans Bluetooth device, after discovering connectable Bluetooth device, the mobile terminal pops up Bluetooth Personal Identification Number (PIN), and sends Bluetooth pairing request carrying Bluetooth Personal Identification Number to the vehicle terminal, after receiving the Bluetooth pairing request, the vehicle terminal automatically checks according to Bluetooth Personal Identification Number to complete pairing, and opens WiFi hotspot. The mobile terminal can connect to the WiFi hotspot, thereby realizing the establishment of the screen projection transmission channel between the mobile terminal and the vehicle terminal. In this way, the user's unaware connection between the mobile terminal and the vehicle terminal is realized.

[0078] Step 402, a channel establishment request of the short-range data transmission channel is sent to the vehicle terminal by calling a system interface; feedback information sent by the vehicle terminal through the system interface is received, and based on the feedback information, a short-range data transmission channel between the mobile terminal and the vehicle terminal is established through the WiFi hotspot.

[0079] In the embodiment of the present application, the feedback information can include one or more of the following: Internet Protocol (IP), communication port and Vehicle Identification Number (VIN).

[0080] The Internet Protocol address can be the IP address of the vehicle terminal, the communication port can be the communication port not occupied by the mobile terminal and the vehicle terminal. The Vehicle Identification Number can be the unique identification code of the vehicle terminal. After the user registers the vehicle, the Vehicle Identification Number corresponds to the vehicle owner. After the user binds the mobile terminal and the vehicle, the mobile terminal user corresponds to the Vehicle Identification Number.

[0081] Exemplarily, the feedback information can include the IP address of the vehicle terminal and the communication port, so that the mobile terminal can establish a short-range data transmission channel between the vehicle terminal through the WiFi hotspot based on the IP address of the vehicle terminal, the communication port and the target message transmission protocol.

[0082] Exemplarily, the feedback information can also include the IP address of the vehicle terminal, the communication port and the VIN code, so that the mobile terminal can verify the identity of the vehicle terminal user and the mobile terminal user based on the VIN code, and after verification, the mobile terminal can establish a short-range data transmission channel between the vehicle terminal through the WiFi hotspot based on the IP address of the vehicle terminal, the communication port and the target message transmission protocol.

[0083] In an implementation, in a case that the mobile terminal and the vehicle terminal establish the screen projection transmission channel, the mobile terminal can send a channel establishment request of the short-range data transmission channel to the vehicle terminal by calling a system interface. The vehicle terminal obtains its own internet protocol address, vehicle identification code, and determines a communication port that is not occupied according to the channel establishment request, and sends feedback information carrying the internet protocol address, the communication port, and one or two of the internet protocol address and the vehicle identification code to the mobile terminal. Further, the mobile terminal can perform consistency verification on the vehicle terminal user and the mobile terminal user based on the vehicle identification code, and obtain a verification result. If the verification result indicates that the vehicle terminal user and the mobile terminal user are the same, the mobile terminal and the vehicle terminal can establish the short-range data transmission channel between them through a WiFi hotspot based on the internet protocol address, the target message transmission protocol, and the communication port. In this way, the establishment process of the short-range data transmission channel between the mobile terminal and the vehicle terminal is efficient and non-intrusive, and the user is also non-intrusive throughout, thereby bringing convenience and good experience to the user.

[0084] It should be noted that, on the basis of the dual-channel architecture (i.e., the screen projection transmission channel and the short-range data transmission channel are established in parallel), after the vehicle terminal displays the vehicle-interactive interface integrated with the super space by the screen projection transmission channel, the mobile terminal can capture the touch events of the user on the interface in real time by means of the short-range data transmission channel, generate a vehicle control instruction accordingly, and transmit the instruction to the vehicle terminal efficiently and stably through the same channel, so as to finally realize precise control of the vehicle function. It should also be noted that, if the mobile terminal does not install the corresponding application program integrated with the super space module, even if the screen projection transmission channel can project the interactive interface to the vehicle terminal and capture the interface touch events, the mobile terminal still cannot generate an effective vehicle control instruction, thereby causing a lack of vehicle control function. Therefore, it is particularly important to independently build the short-range data transmission channel between the super space module application program and the vehicle terminal. Through this special link, the mobile terminal can analyze the touch events and convert the instruction through the super space module after receiving the touch events, and then reliably send the vehicle control instruction to the vehicle terminal through the short-range data transmission channel. Without this independent short-range data transmission channel, even if the screen projection and interface rendering are successful, the entire system will still be unable to realize any vehicle control function due to the lack of instruction generation and transmission capability, which seriously affects the core interactive experience and functional integrity of the vehicle-interactive system.

[0085] In some embodiments, the computer device can further perform the following steps.

[0086] In step 311, if the mobile terminal and the vehicle terminal establish the remote data transmission channel, the first network connection information of the short-range data transmission channel and the second network connection information of the remote data transmission channel are obtained. The message transmission protocols of the short-range data transmission channel and the remote data transmission channel are the same.

[0087] In the embodiments of the present application, the first network connection information is used to represent the network connection state of the short-range data transmission channel, such as the WiFi connection state, and the first network connection information includes one or more of the WiFi network signal strength, the data transmission delay, the data packet loss rate, and the number of disconnections within a preset time.

[0088] In the embodiments of the present application, the second network connection information is used to represent the network connection state of the long-range data transmission channel, such as the network state of 4G / 5G, and the second network connection information can be whether the mobile terminal receives information that the network of the vehicle terminal is in an unstable state.

[0089] In one implementation manner, continuing to refer to Figure 1 and Figure 2 , the mobile terminal and the vehicle terminal establish a screen projection transmission channel through the software development kit (SDK) of the interconnection APP and the vehicle display screen of the vehicle terminal. At the same time, the short-range data transmission channel and the long-range data transmission channel are established between the mobile terminal and the vehicle terminal, and further, the first network connection information of the short-range data transmission channel and the second network connection information of the long-range data transmission channel are obtained.

[0090] Step 312, based on the first network connection information and / or the second network connection information, selecting the short-range data transmission channel and the long-range data transmission channel to determine the target data transmission channel, wherein the target data transmission channel is used to transmit the first vehicle data and / or the vehicle control instruction.

[0091] In one implementation manner, continuing to refer to Figure 2 , the mobile terminal arbitrates the short-range data transmission channel and the long-range data transmission channel based on the first network connection information and / or the second network connection information through the short-range and long-range fusion module to determine the stable target data transmission channel, so as to be able to transmit the first vehicle data and the vehicle control instruction.

[0092] It should be noted that step 312 can be executed before any step, after any step, or synchronously with any step, and the present application does not make specific limitations thereto.

[0093] In some embodiments, step 312 selects the short-range data transmission channel and the long-range data transmission channel based on the first network connection information and / or the second network connection information to determine the target data transmission channel, which can be achieved by the following steps:

[0094] If the first network connection information satisfies the first condition, a short-range data transmission channel is selected; wherein the first condition includes: network signal strength greater than a first strength threshold, no more than a first number of disconnect and reconnect operations within a preset time; if the first network connection information does not satisfy the first condition and the second network connection information satisfies the second condition, switching from the short-range data transmission channel to the long-range data transmission channel; wherein the second condition includes: the mobile terminal does not receive the prompt information that the vehicle terminal is in a network unstable state; if the second network connection information does not satisfy the second condition, switching from the long-range data transmission channel to the short-range data transmission channel.

[0095] In the embodiment of the application, the first strength threshold can be a pre-set signal strength threshold, such as -90 decibels relative to one milliwatt (dBm), and the number of disconnect and reconnect operations within the preset time can be no more than 3 times within a preset time such as 10s, of course, it can also be disconnected within the preset time, and the first number such as 3 times of reconnect failure.

[0096] In one implementation manner, if the network signal strength (such as the WiFi network signal strength) is greater than the first strength threshold, and the number of disconnect and reconnect operations within the preset time is no more than the first number, that is, the first network connection information satisfies the first condition, indicating that the short-range data transmission channel is stable, at this time, the short-range data transmission channel can be preferentially selected, that is, switched to the short-range data transmission channel as the target data transmission channel. If the first network connection information does not satisfy the first condition in the process of transmitting data in the short-range data transmission channel, and the mobile terminal does not receive the prompt information that the vehicle terminal is in a network unstable state, that is, the second network connection information satisfies the second condition, at this time, the short-range data transmission channel can be switched to the long-range data transmission channel. If the prompt information that the vehicle terminal is in a network unstable state or an error code is received in the process of transmitting data in the long-range data transmission channel, that is, the second network connection information does not satisfy the second condition, at this time, the long-range data transmission channel can be switched to the short-range data transmission channel.

[0097] From the above, in the embodiments of the present application, when the short-range channel signal strength is sufficient and the connection is stable (few times of disconnection and reconnection), it is preferred to be selected, making full use of the characteristics of low delay and high bandwidth of short-range transmission, adapting to the real-time interaction scene of vehicle machine and mobile terminal; if the short-range channel does not meet the conditions, if the long-range channel is available (the vehicle machine does not prompt network instability), automatically switch to the long-range channel to avoid communication interruption caused by short-range connection problems and ensure the continuous availability of core functions. When the long-range channel also does not meet the conditions, switch back to the short-range channel to form a two-way switching mechanism to cope with complex network environments (such as temporary recovery of short-range signal fluctuations and sudden congestion of long-range network) and improve connection reliability. In this way, by clear condition judgment, stable data channel is selected to transmit data, ensuring the integrity, security and timeliness of data.

[0098] In other embodiments of the present application, to avoid frequent switching between short-range transmission channel and long-range transmission channel, a switching threshold can be set. For example, when switching from a short-range data transmission channel to a long-range data transmission channel, the short-range data transmission channel indicator needs to continuously meet the first condition for 5 seconds, and the long-range data transmission channel indicator needs to continuously meet the second condition for 5 seconds, to trigger the switching. When switching from a long-range data transmission channel to a short-range data transmission channel, the WiFi of the short-range data transmission channel needs to be successfully reconnected and the stability indicator needs to meet the standard for 3 seconds to trigger the switching. In this way, frequent switching between the two channels is avoided, system resource consumption is reduced, and temporary communication delay caused by switching is also reduced.

[0099] In the prior art, the implementation process of the voice control vehicle function includes: collecting voice signals through the built-in microphone of the vehicle machine or the microphone of the mobile phone, obtaining the original audio data of the voice instruction issued by the user, and sending it to the vehicle machine; the built-in voice module of the vehicle machine receives the original audio data, first performs voice-to-text operation, thereby converting the audio signal into corresponding text content; then the text is processed by generalization (such as correcting accent deviation, supplementing omitted semantics, unifying expression format, etc.); then, the text after generalization is transmitted to the Natural Language Understanding (NLU) module, which accurately identifies the user's intention through semantic analysis and analysis of the control vehicle instruction, and extracts the effective control vehicle instruction from it, to complete the conversion from natural language to machine recognizable instruction. Further, the control vehicle instruction is converted into a signal conforming to the vehicle communication protocol, such as CAN bus signal or DDS data distribution service format, and is sent to the execution unit to execute corresponding operations, thereby realizing voice control of the vehicle.

[0100] However, the above method at least has software logic encapsulated in the software package of the vehicle machine end to issue and analyze the control vehicle instruction, which increases the complexity of the software package and the computing power load of the vehicle machine.

[0101] To reduce the computing power load of the car machine, in some embodiments, the determination of the car control instruction corresponding to the touch event in step 304 can be realized by the following steps:

[0102] According to the touch event, the target application touched is determined; according to the touch event and the function configured in the target application, the generalization information of the target function indicated by the touch event is determined; and the car control instruction of the target function is generated according to the structured data set corresponding to the generalization information of the target function in the preset corpus table, wherein the preset corpus table is determined by the mobile terminal and the car machine terminal together, so that the car control instruction generated based on the preset corpus table can reuse the semantic analysis module of the car machine terminal.

[0103] In the embodiments of the present application, the corpus table generally refers to a corpus presented in the form of a table, which contains generalization information for different functions, operation instructions for the functions, and semantic instruction information with structured data set corresponding to the generalization information, thereby forming a structured data set that can be queried and analyzed.

[0104] In the embodiments of the present application, the semantic analysis module of the car machine terminal has intelligent analysis capability for generalized semantics.

[0105] It should be noted that the preset corpus table can be a corpus table determined by the car machine terminal and the mobile terminal together, and the control instruction generated based on the preset corpus table can reuse the semantic analysis module of the car machine terminal, so that the transmission degree of freedom of the car control instruction is high and the instruction understanding ability is intelligent and accurate, which improves the expansibility, flexibility and understanding of the car control instruction, and reuses the voice execution link of the car machine terminal, reduces the resource occupation of the car machine. Exemplarily, the preset corpus table can be as shown in Table 3,

[0106] Table 3

[0107]

[0108] In the embodiment of the present application, after the vehicle machine end listens to the touch event of the vehicle machine display screen, the vehicle machine display screen identifies the hot area touch event of the control interface and reports it to the mobile end. The mobile end analyzes the touch event and determines the generalization information of the target function indicated by the touch event. Then, through the preset corpus table, the mobile end can directly map the generalization information of the target function to the semantic result with a structured data set, i.e., semantic instruction information. It should be noted that the semantic result exists in the form of a string, including but not limited to fields such as mode, name, insType, etc. The semantic result can be directly used as the control vehicle instruction of the target function. Further, the mobile end sends the control vehicle instruction to the vehicle machine end through a short-range data transmission channel. The software package of the vehicle machine end calls the local semantic analysis module such as the NLU module through the interface. The NLU module analyzes the mode, name, insType, etc. in the control vehicle instruction according to its own logic, and converts it into a DDS / CAN signal on the vehicle machine end and sends it to the corresponding domain controller, thereby controlling the execution unit. For example, if the touch event indicates opening the door, the domain controller can be the body controller, the execution unit is the door, and the control vehicle instruction is to open the door.

[0109] As can be seen from the above, by prepositioning the touch instruction analysis and semantic generalization process to the mobile end, the mobile end can directly analyze the touch instruction and output the generalized semantics. In this way, the vehicle machine end does not need to specially deploy the logic software package for recognizing and converting the control instruction of the display screen to perform additional analysis. Further, the semantic result after generalization is transmitted to the vehicle machine end, and only the existing NLU module logic of the vehicle machine end is needed to complete the instruction processing. In this way, not only the processing flow of the vehicle machine end is simplified, but also the load pressure of the vehicle machine end is effectively reduced, and the existing NLU module capability of the vehicle machine end is fully utilized, achieving the dual goals of function reuse and efficiency improvement.

[0110] It should be noted that the original control logic software of the super space of the vehicle machine end is transferred to the mobile end and run and analyzed by the mobile end, and the vehicle machine end only displays the application and does not process data.

[0111] Taking the 3D car model application as an example, the control interface of the 3D car model application displays a 3D car model. The user can trigger control through the 3D car model, for example, the user clicks the door of the 3D car model, and the touch door is opened. As can be seen, in this example, after the mobile end listens to the touch event, it determines the target function indicated by the touch event based on the trigger position of the touch event on the 3D car model, and then generates the control vehicle instruction of the target function according to the structured data set corresponding to the target function in the preset corpus table.

[0112] The functions in the 3D car model application include one or more of rotating the 3D car model, opening / closing the car door, opening / closing the trunk, rendering the 3D car model, or pasting a map in the triggered position of the 3D car model. The mobile terminal analyzes the corresponding control instruction according to the function triggered by the user, and generates the corresponding control instruction. For example, if the target function is to rotate the 3D car model, the mobile terminal needs to determine the steering direction of the 3D car model according to the direction indicated by the touch event, and then generate 3D car model data after rotating according to the steering direction, and send it to the car terminal for displaying the 3D car model after rotating. Or, if the target function is to render the 3D vehicle, the mobile terminal determines the position to be rendered and the rendering information according to the touch event, calls the rendering tool to render the position to be rendered on the 3D car model, and sends the rendered 3D car model data to the car terminal for displaying the rendered 3D car model. Or, the target function is to open the car door, and the mobile terminal generates a car door opening instruction and sends it to the car body controller, which controls the car door to open. The above examples are used to illustrate that in this embodiment, the control data of the car terminal is analyzed and controlled by the mobile terminal, and the display screen of the car terminal collects data without analyzing and controlling the data. Therefore, the embodiment of the application realizes intelligent control of the vehicle using the resources of the mobile terminal, and reduces the load of the car terminal.

[0113] In some embodiments, the car control instruction is sent to the car terminal through the short-range data transmission channel using the target message transmission protocol in step 304, so that after the car terminal receives the car control instruction, it is passed to the execution unit corresponding to the car control instruction for instruction execution. This can be achieved by the following steps.

[0114] The first asymmetric key pair is generated in advance by the first secure storage module; wherein the first asymmetric key pair includes a first private key and a first public key; the car control instruction is encrypted using the first private key to obtain a ciphertext car control instruction; the ciphertext car control instruction is sent to the car terminal through the short-range data transmission channel using the target message transmission protocol, so that the car terminal decrypts the ciphertext car control instruction using the first public key transmitted in advance to obtain the car control instruction.

[0115] In the embodiment of the application, the first secure storage module can be an Android Keystore in the mobile terminal. The asymmetric key pair can be an RSA (Rivest-Shamir-Adleman) key pair.

[0116] In an implementation, when the mobile terminal sends data such as a vehicle control instruction to the vehicle terminal through the short-range data transmission channel, the mobile terminal can generate a first RSA key pair including a RSA private key (i.e., a first private key) and a RSA public key (i.e., a first public key) in advance through the first secure storage module, store the RSA private key in the first secure storage module, and send the RSA public key to the vehicle terminal. Then, the mobile terminal can encrypt the vehicle control instruction using the RSA private key to obtain a ciphertext vehicle control instruction, store the ciphertext vehicle control instruction in a data information part in a data format corresponding to a target message transmission protocol, and send the ciphertext vehicle control instruction to the vehicle terminal using the target message transmission protocol through the short-range data transmission channel. After receiving the ciphertext vehicle control instruction, the vehicle terminal decrypts the ciphertext vehicle control instruction using the RSA public key sent by the mobile terminal to obtain the vehicle control instruction, and then transmits the vehicle control instruction to a corresponding vehicle-side execution device for execution. In this way, the security of data transmission is ensured.

[0117] In another implementation, when the vehicle terminal sends data such as vehicle data to the mobile terminal through the short-range data transmission channel, the vehicle terminal can generate an RSA key pair including a RSA private key and a RSA public key in advance through its own secure storage module, store the RSA private key in its own secure storage module, and send the RSA public key to the mobile terminal. Then, the vehicle terminal can encrypt the vehicle data using the RSA private key to obtain ciphertext vehicle data, store the ciphertext vehicle data in a data information part in a data format corresponding to a target message transmission protocol, and send the ciphertext vehicle data to the mobile terminal using the target message transmission protocol through the short-range data transmission channel. After receiving the ciphertext vehicle data, the mobile terminal decrypts the ciphertext vehicle data using the RSA public key sent by the vehicle terminal to obtain the vehicle data, and then uses the vehicle data to render and process each element in the super space module. In this way, the security of data transmission is ensured.

[0118] In some embodiments, in step 304, the vehicle control instruction is sent to the vehicle terminal through the short-range data transmission channel using the target message transmission protocol, so that after the vehicle terminal receives the vehicle control instruction, the vehicle control instruction is transmitted to a corresponding execution unit for instruction execution. The sending can also be implemented through the following steps.

[0119] In step 501, a second asymmetric key pair is generated in advance through the first secure storage module, wherein the second asymmetric key pair includes a second private key and a second public key.

[0120] In step 502, the vehicle terminal sends a ciphertext symmetric key and a first signature value through a short-range data transmission channel. The ciphertext symmetric key is a symmetric key dynamically generated by the vehicle terminal through a second secure storage module, and is encrypted using a second public key. The first signature value is obtained by digitally signing the ciphertext symmetric key using a first key Hash Message Authentication Code (HMAC).

[0121] In step 503, the second HMAC stored in the first secure storage module is used to verify the signature of the ciphertext symmetric key. If the verification is passed, the ciphertext symmetric key is decrypted using a second private key to obtain a symmetric key. The first HMAC is the same as the second HMAC.

[0122] In step 504, the symmetric key is used to encrypt the vehicle control instruction to obtain a ciphertext vehicle control instruction, and the second HMAC is used to digitally sign the ciphertext vehicle control instruction to obtain a second signature value.

[0123] In step 505, the ciphertext vehicle control instruction and the second signature value are sent to the vehicle terminal through the short-range data transmission channel using a target message transmission protocol. The vehicle terminal verifies the signature of the second signature value using the first HMAC. If the verification is passed, the ciphertext vehicle control instruction is decrypted using the symmetric key to obtain the vehicle control instruction.

[0124] In the embodiments of the present application, the mobile terminal and the vehicle terminal both pre-store the same Hash-Based Message Authentication Code (HMAC).

[0125] In the embodiments of the present application, the symmetric key can be an Advanced Encryption Standard (AES) key.

[0126] In the embodiments of the present application, the second secure storage module can be a secure storage module in the vehicle terminal.

[0127] In an implementation, when the mobile terminal sends data such as a vehicle control instruction to the vehicle terminal through the short-range data transmission channel, the mobile terminal can pre-generate a second RSA key pair including an RSA private key (i.e., a second private key) and an RSA public key (i.e., a second public key) through the first secure storage module, store the RSA private key in the first secure storage module, and send the RSA public key to the vehicle terminal. Then, the vehicle terminal can pre-generate an AES key through the second secure storage module, encrypt the AES key using the RSA public key transmitted by the mobile terminal to obtain a ciphertext AES key, use the first HMAC to digitally sign the ciphertext AES key to obtain a first signature value, and send the ciphertext AES key and the first signature value to the mobile terminal through the short-range data transmission channel. Further, the mobile terminal can use the second HMAC to verify the signature of the ciphertext AES key to obtain a verification result, and if the verification result indicates that the verification is passed, use the RSA private key to decrypt the ciphertext AES key to obtain the AES key. The mobile terminal encrypts the vehicle control instruction using the AES key to obtain a ciphertext vehicle control instruction, uses the second HMAC to digitally sign the ciphertext vehicle control instruction to obtain a second signature value, and sends the ciphertext vehicle control instruction and the second signature value to the vehicle terminal through the short-range data transmission channel using the target message transmission protocol. The vehicle terminal uses the first HMAC to verify the signature of the second signature value, and if the verification is passed, uses the AES key to decrypt the ciphertext vehicle control instruction to obtain the vehicle control instruction, so as to pass the vehicle control instruction to the corresponding vehicle-side execution device for execution. In this way, while ensuring the performance of both ends, the mobile terminal generates an asymmetric key pair in the secure storage module such as the keystore before communication, and transmits the public key to the vehicle terminal. The vehicle terminal generates a symmetric key, encrypts the symmetric key using the public key, and transmits the symmetric key to the mobile terminal to complete key negotiation. The transmission process uses symmetric key encryption transmission, thereby ensuring the confidentiality of key transmission. In addition, the mobile terminal and the vehicle terminal pre-store the same HMAC algorithm to generate a key, and both use the key to calculate the signature value of the transmission message. When receiving, the signature value is verified to ensure the integrity of the transmission data.

[0128] In some embodiments, when the sender (the mobile terminal or the vehicle terminal) sends information, a timestamp can be added to the information, and the receiver (the vehicle terminal or the mobile terminal) can verify the timestamp to achieve the purpose of tamper resistance.

[0129] It should be noted that the RSA private key needs to ensure confidentiality, the RSA public key needs to prevent tampering, the AES key needs to be stored in a secure module or encrypted and stored in a secure module, and the first HMAC needs to be stored in a secure module or encrypted and stored in a secure module.

[0130] Here, when the vehicle terminal sends data such as vehicle data to the mobile terminal through the short-range data transmission channel, similar to the above method, this will not be repeated.

[0131] In some embodiments, the super space module comprises one or more of the following elements: wallpaper, three-dimensional car model, vehicle condition card and vehicle control card. Based on the first vehicle data, a hand-car interaction page with super space is generated in the super space module of the mobile terminal, which is adapted to the car machine display screen of the car machine terminal. This can be achieved by the following steps.

[0132] Step 601, obtaining the environment information of the location where the car machine terminal is located and the user preference function.

[0133] In the embodiments of the present application, the environment information can at least include weather information and time information. According to the weather information and the time information, the basic form of the wallpaper in the super space module can be rendered. Further, the environment information can also include geographic location information, traffic live, etc. The geographic location information can include latitude and longitude, topography, etc. The traffic live can include the congestion of the current road, the state of the traffic light, whether there is a traffic accident, the number of road intersections, etc. In addition, the environment information can also include local characteristic buildings, cultural landscapes, etc. By collecting rich environment information, rich materials can be provided for later adjusting the wallpaper in the super space module.

[0134] In the embodiments of the present application, the environment information of the location where the car machine terminal is located can be obtained from the network in real time after the vehicle positioning.

[0135] In the embodiments of the present application, the user preference function can be a function designed to improve the convenience and personalized experience based on user preference data. The user preference function can be a single function, such as a rear door, a rearview mirror, etc., such as the quick control card 43 in Figure 4 , or a combination of multiple functions corresponding to a scene mode, such as the quick scene card 44 in Figure 4 , such as the small rest mode, the starry sky mode, the cinema mode, the commuting mode, etc. Taking the small rest mode as an example, the seats, air conditioners, lights and other vehicle functions need to be controlled synchronously. Taking the commuting mode as an example, the preference settings are automatically triggered according to the time and the scene. The commonly used navigation route is automatically opened and the commonly heard music is automatically played during the commuting period. The air conditioner is automatically adjusted to the user preference temperature and the window is automatically closed after getting on the car in summer, etc. The present application does not make specific limitations on this.

[0136] Step 602, dynamically adjusting the form of the wallpaper in the super space module based on the environment information.

[0137] In the embodiments of the present application, the environmental information at least includes weather information and time information. Based on the weather information, a plurality of weather elements can be determined, which can include temperature, humidity, wind power, wind direction, solar radiation intensity, cloud thickness, and precipitation, etc. Based on the plurality of weather elements, the form of the wallpaper in the super space module is dynamically adjusted and rendered. Specifically, the weather elements can be manifested as visible animation effects such as raindrops, snowflakes, and light intensity in the wallpaper of the super space module, for example, the number and size of raindrops in the wallpaper are increased or decreased to represent the size of the current rain, the current environmental temperature and light time are represented by adjusting the light intensity and light color, the current heavy fog weather is represented by rendering the fog effect, and the day and night are distinguished according to the time information and in cooperation with the scene background color.

[0138] In step 603, based on the vehicle body component state data in the first vehicle data, the state of the three-dimensional vehicle model in the super space module is synchronously displayed; wherein the vehicle body component state data is used to indicate the working state of each component of the vehicle body in the vehicle.

[0139] In the embodiments of the present application, the vehicle body component state data includes the working state of each component in the vehicle body, and the components include but are not limited to seat ventilation / heating, air conditioning, four-door window, sunroof, sunshade curtain, four-door, four-door lock, etc.

[0140] In the embodiments of the present application, after the mobile terminal obtains the first vehicle data, based on the vehicle body component state data, the state of the three-dimensional vehicle model in the super space module is synchronously displayed, so that the state of each component of the vehicle body in the three-dimensional vehicle model corresponds to the actual state of the vehicle. For example, as shown in the display of the three-dimensional vehicle model 41 in the super space module 40. Figure 4 In this way, the three-dimensional vehicle model supports user clicking and rotating to view the state of each component, which is more interactive than static cards; at the same time, the actual vehicle state is synchronized to the virtual vehicle model, enhancing the user's "connection" with the vehicle, improving the user's immersive experience.

[0141] In step 604, based on the vehicle detection data in the first vehicle data, the vehicle condition card in the super space module is synchronously displayed; wherein the vehicle detection data is used to indicate the basic running ability, safety state and key function performance of the vehicle.

[0142] In the embodiments of the present application, the vehicle detection data includes but is not limited to four tire pressure, vehicle remaining mileage, remaining oil / charge, etc.

[0143] In the embodiments of the present application, after the mobile terminal obtains the first vehicle data, based on the vehicle detection data, the vehicle condition card in the super space module is synchronously displayed, for example, as shown in the display of the vehicle condition card 42 in the super space module 40. Figure 4The 3D vehicle model 42 is shown in the image. In this way, scattered detection data such as tire pressure and remaining mileage are presented centrally and visually through cards. Users can grasp the basic operation, safety, and functional status of the vehicle without having to switch between multiple pages in a super-space, reducing information acquisition costs.

[0144] Step 605: Based on user preference functions, determine the vehicle control function displayed on the vehicle control card in the super space module; wherein, the vehicle control function is used to instruct the user to perform touch operations.

[0145] In this embodiment, after obtaining user preference functions, the mobile terminal determines the vehicle control functions with the highest usage frequency (TOPN) based on the frequency of the user's use of a single function, and determines the combined scenario modes with the highest usage frequency (TOPN) based on the frequency of the user's use of combined scenario modes. It should be noted that each combined scenario mode is associated with one or more vehicle control functions. Then, the frequently used single vehicle control functions are displayed on the quick control card of the vehicle control card in the super space module, and the frequently used combined scenario modes are displayed on the quick scenario card of the vehicle control card in the super space module. For example, as shown... Figure 4 The system includes quick control cards 43 and quick scene cards 44. This eliminates the need for users to navigate through all vehicle control functions in the super space module; the cards directly display their preferred single function or combination of scene modes, allowing for immediate operation with a single click. This reduces the time spent on multiple switching steps, making it particularly suitable for scenarios requiring quick vehicle control while driving. For multi-functional combination preferences, the control cards integrate them into a single trigger entry point. Users no longer need to adjust multiple independent functions individually; a single click activates the entire scene setting, resolving the pain point of cumbersome multi-step operations and improving the convenience of vehicle control.

[0146] Step 606: Based on the wallpaper, 3D car model, vehicle condition card, and vehicle control card, generate a hand-car interconnection page with super space that is adapted to the vehicle's infotainment display screen.

[0147] In this embodiment, based on environmental information, first vehicle data, and user preference functions, the wallpaper, 3D car model, vehicle status card, and vehicle control card are dynamically adjusted and rendered. These four elements are then integrated into a single page for vehicle-to-hand communication with a super-space interface. This page is then projected onto the vehicle's display screen in a way that adapts to the screen. Users do not need to switch between multiple independent modules on the vehicle's device (e.g., from the vehicle status page to the vehicle control page) to simultaneously obtain vehicle status and perform vehicle control operations on the same interface, significantly reducing the number of steps required while driving and improving interaction efficiency. Furthermore, the page elements are linked with the mobile super-space interface, allowing users to seamlessly switch between their phone and vehicle's settings without needing to readjust the interface logic and function locations. This enhances the integrated vehicle-to-hand communication experience and avoids the feeling of disconnect when using different devices.

[0148] It should be noted that, since the 3D car model rendering affects the terminal energy consumption, in some embodiments, different refresh frame rates can be set for the three-dimensional car model based on the state of the three-dimensional car model; wherein the state of the three-dimensional car model includes a static state and a dynamic state. When the three-dimensional car model is in the static state, the refresh frame rate of the three-dimensional car model can be reduced to 1 frame per second (FPS); when the three-dimensional car model is in the dynamic state, the system-level refresh frame rate can be called to set the refresh frame rate of the function interface to 30 FPS, and the refresh frame rate of the three-dimensional car model is also reduced to 30 FPS, so as to reduce the power consumption. Of course, in other embodiments, the three-dimensional car model can also be designed differently based on different three-dimensional scenes; wherein the three-dimensional scene includes a charging scene and a dynamic scene; when the three-dimensional scene is the charging scene, the rendering details of the three-dimensional car model can be weakened to reduce the power consumption.

[0149] The vehicle control method provided by the embodiments of the present application will be described below through a specific embodiment.

[0150] In order to solve the business pain points of controlling the car in the hand-car interconnection mode, and realize the deep system integration of the mobile phone terminal and the car terminal, the embodiments of the present application design a car control system and a car control method in a super space based on the hand-car interconnection mode.

[0151] Referring to Figure 4 , the embodiments of the present application provide a super space function based on the hand-car interconnection mode, and the main elements on the interface include: dynamic wallpaper, 3D car model, car condition card and car control card. Among them, the dynamic wallpaper will display day / night mode according to the current time, and also dynamically adjust the different forms of the wallpaper according to the weather data obtained by the background on the same day, including sunny, rainy, cloudy, snowy and cloudy, a total of five forms; when entering the super space, the current vehicle condition data will be actively obtained, including but not limited to seat ventilation / heating, air conditioning, four-door window, sunroof, sunshade, four-door lock, etc. A total of more than 50 vehicle condition data, and the above states are synchronized to the 3D car model; the cards in the interface include but are not limited to the car condition card and the car control card, and the car condition card includes but is not limited to displaying the remaining mileage of the vehicle, four-door tire pressure, etc. The car control card opens the control authority of the user to the car control function, including but not limited to the rear door, wireless charging, rearview mirror, combination scene mode, etc. At the same time, the user can also set the color of the car model according to his own preference.

[0152] To realize the functions in Figure 4 , refer to Figure 2As shown, the car control system in the super space based on the hand-car interconnection mode provided by the embodiments of the present application includes a mobile phone terminal (corresponding to the mobile terminal described above) and a car terminal, wherein the mobile phone terminal refers to a device connected to the car display screen, including but not limited to Huawei, OPPO, Honor, etc., supporting application programs, containing long-range and short-range communication modules, long-range and short-range fusion modules; the car terminal refers to a device supporting screen projection input, containing a display screen, a semantic analysis module, a short-range and long-range communication module; the cloud platform refers to a terminal that completes remote control logic, data transmission and storage; the body controller refers to a domain controller that completes the control logic of the actuator; the actuator (corresponding to the execution unit described above) refers to an air conditioner, a window, a door, etc. connected to the body controller.

[0153] In some embodiments, with reference to Figure 5 As shown, the embodiments of the present application provide a super space car control method based on an interconnection mode, applied to the car control system described above, which can complete intelligent car control while saving car machine computing power, giving users an immersive experience, including the following steps:

[0154] Step 51, in the case of starting the screen projection application of the car display on the car terminal, establish the screen projection connection between the mobile phone terminal and the car terminal.

[0155] Here, after the car terminal starts the screen projection application of the car display, the mobile phone terminal and the car terminal establish a connection through the SDK of Hicar, Carlink, HONOR CarConnect, through the connection mode of BLE and WIFI, after the hand-car interconnection channel (corresponding to the screen projection transmission channel described above) is established between the mobile phone and the car, the subsequent application of the mobile phone terminal is sent to the car display screen after proportionally adapting, realizing the ability to synchronize the content of the mobile phone to the car, facilitating display and user operation. For example Figure 1 As shown, the mobile phone terminal and the car terminal interconnection schematic diagram.

[0156] Step 52, the mobile phone terminal acquires IP and VIN code, and establishes an independent WIFI interconnection channel.

[0157] Here, the independent WIFI channel corresponds to the short-range data transmission channel described above.

[0158] Here, after the establishment of the hand-car interconnection channel, the mobile phone end sends an autonomous WIFI channel establishment request to the car machine end. The car machine end sends feedback information carrying the IP, VIN code and communication port of the car machine end to the mobile phone end based on the request. The mobile phone end verifies the identity of the car machine end user and the mobile phone end user based on the VIN. If it passes, it connects the WIFI channel autonomously based on the car machine end IP, communication port and pre-agreed MQTT protocol through the WiFi hotspot, for short-range transmission of vehicle control data and unified packaging of data as MQTT protocol, and autonomously designs a double-end encryption scheme. Thus, the vehicle control data is autonomously scalable, and the transmission safety of the data is ensured. As shown in Figure 1 The mobile phone end and the car machine end interconnection schematic diagram.

[0159] In one implementation, the establishment process of the manufacturer hand-car interconnection data channel (i.e. the above-mentioned hand-car interconnection channel) and the self-built WIFI channel is completely non-invasive, providing convenience and better experience for users. The specific process is as follows: after the user opens the Bluetooth and WIFI on the mobile phone end and the car machine end, the car machine end continuously sends Bluetooth broadcast; the mobile phone end scans the Bluetooth device, discovers the connectable device, generates a Bluetooth pairing code, and sends a Bluetooth pairing request carrying the Bluetooth pairing code to the car machine end; the car machine end is paired and connected with the mobile phone end based on the Bluetooth pairing code, and the WiFi hotspot of the car machine end is opened. The mobile phone end establishes a non-invasive connection with the car machine end through the car machine end WIFI hotspot. After the hand-car interconnection channel is established, the mobile phone end will automatically initiate an autonomous WIFI channel connection request. The car machine end feeds back the car machine IP, communication port and vehicle VIN code after receiving the request. The mobile phone end verifies the user information according to the vehicle VIN code, and completes the establishment of the autonomous WIFI channel according to the car machine IP, communication port and MQTT protocol after the verification is passed. In this way, the establishment process of the autonomous WIFI channel is efficient and non-invasive, and does not require manual operation by the user.

[0160] Step 53, the mobile phone end selects the autonomous WIFI channel and the remote channel to transmit data through the near-far range fusion module.

[0161] Here, the vehicle control data corresponds to the above-mentioned vehicle data. The remote channel corresponds to the above-mentioned remote data transmission channel.

[0162] Here, the car machine end and the mobile phone end prefer to transmit data through the autonomous WIFI channel. The mobile phone end acquires vehicle condition data and completes state display in the super space, and then listens to the click event of the car machine end. In this process, the mobile phone end arbitrates between the autonomous WIFI channel and the remote channel based on the 4G / 5G network state and the near-range WIFI connection state, selects a stable transmission channel, and completes data channel switching.

[0163] Step 54, the mobile phone end receives the click event from the car machine end, determines the control car instruction corresponding to the click event, and transmits it to the car machine end through a long-range channel or a short-range channel. The car machine end intelligently analyzes the control car instruction through a semantic analysis module and sends it to the vehicle body controller.

[0164] Here, the core control logic of the hyperspace is transferred from the car machine end to the mobile phone end. The mobile end receives the instruction request from the car machine display screen and implements the function through the core logic unit. Real-time vehicle condition data is obtained from the car end through the self-built WIFI communication channel / remote control channel, and the state is displayed on the 3D car model. At the same time, the mobile phone end listens to the touch control instruction (also known as click event or touch event) of the car machine screen. After the hot area click event of the control car interface on the car machine screen is recognized, it is reported to the mobile phone end. After the mobile phone end analyzes the touch control instruction, it determines the control car instruction. The mobile phone end sends the control car instruction to the car machine end to the car machine end. Specifically, there are two main ways to send the control car instruction to the car end through the communication module: (1) the mobile phone end sends the control car instruction to the car end through the 4G / 5G network module, customizes the MQTT protocol, and uses the private key to encrypt and send the control car instruction to the car end. The Tbox module of the car end receives the instruction, decrypts it through the public key, and sends the data to the semantic analysis module of the car machine end for intelligent analysis; (2) the mobile phone end sends the control car instruction to the car end through the self-built WiFi channel, customizes the MQTT protocol, and uses the private key to encrypt and send the control car instruction to the car end. The WiFi module of the car end receives the instruction, decrypts it through the public key, and sends the data to the semantic analysis module of the car machine end for intelligent analysis. The long-range and short-range channels share a set of MQTT protocol interfaces and encryption methods.

[0165] It should be noted that after the car machine end receives the data transmitted by the mobile phone end long / short range, the semantic analysis module is an artificial intelligence (AI) semantic analysis module, which has intelligent analysis capability for generalized semantics, so the transmission freedom of the instruction is high and the instruction understanding capability is intelligent and accurate.

[0166] Step 55, the vehicle body controller receives the control car instruction and transmits it to the actuator to complete the instruction execution.

[0167] Here, the semantic analysis module of the car machine end sends the analyzed control car instruction to the vehicle body controller and transmits it to the execution end for instruction execution.

[0168] As can be known from the above, in the embodiment of the application, the mobile phone as the control core main body, compared with the current control mode taking the car machine as the main body, can share the car machine end computing power to the mobile phone end, use Hicar, Carlink, HONORCarConnect and other channels to complete the video stream data synchronization of the mobile phone end and the car machine end, and display the super space on the car machine center screen. The user enjoys immersive experience on the projection interface, the 3D car model completes the car end state display, and triggers the car control instruction by clicking the car control card hot area. After the mobile phone listens, it intelligently judges the network situation, and then selects the remote or short-range channel to issue the instruction to the car machine end. After the car machine end local intelligent semantic analysis module analyzes, the controller end responds to the car control instruction. In this way, the development of the car machine software is reduced, the car machine computing power resources are saved, the resource sharing of the mobile phone and the car is realized, and the software of the function is quickly and efficiently iterated on the mobile phone end. At the same time, the expansibility, flexibility and understanding of the car control instruction are improved, and the fusion of long and short range ensures the stability of the car machine and mobile phone end car control instruction transmission.

[0169] In an implementable manner, taking the mobile phone as the mobile end, Hicar, Carlink, HONOR CarConnect and the like as the projection channel, WIFI / remote control channel as the data transmission channel, the working process of controlling the vehicle based on the super space function under the interconnection mode is as shown in Figure 6 The detailed working process is as follows:

[0170] The first step is to open the BLE broadcast of the car machine end and listen to whether there is a connection event access. If there is a connection event access, the car and the mobile phone are paired 1 through Bluetooth, the mobile phone end inputs the vehicle pairing code to establish connection and identify the user identity, and if the Bluetooth pairing is successful, the WiFi hot spot of the car machine end is opened.

[0171] The second step is the establishment process of the screen projection channel and the data channel. Here, the process of establishing the screen projection channel: if the Bluetooth pairing is successful, the mobile phone establishes a WiFi channel 2 for screen projection by connecting to the WiFi hotspot of the vehicle terminal. Here, the data transmission channel includes an autonomous WIFI channel (also known as a short-range channel) and a remote channel. The establishment process of the autonomous WIFI channel: after completing the vehicle connection, the mobile terminal automatically initiates a channel establishment request for the autonomous WIFI channel to the vehicle terminal. The vehicle terminal receives the channel establishment request and feeds back the vehicle IP, communication port, and vehicle VIN code to the mobile terminal. The mobile terminal verifies the user information based on the vehicle VIN code. After verification, the autonomous WIFI channel is established based on the vehicle IP, communication port, and MQTT protocol. In this way, the establishment process of the autonomous WIFI channel is efficient and non-intrusive, without the need for manual user operation. It should be noted that after completing the vehicle connection, the mobile terminal can use the vehicle connection screen projection method in related technologies to project the original vehicle connection interface to the vehicle terminal. The original vehicle connection interface displays a 3D vehicle control entry corresponding to the super space. At this time, the vehicle terminal display interface can display the original vehicle connection interface. The user clicks the 3D vehicle control entry. The SDK intent of Hicar, Carlink, and HONOR CarConnect in the vehicle terminal pulls up the super space module by package name and page name, and automatically feeds back the IP, communication port, and VIN code of the vehicle to the vehicle terminal. The mobile terminal verifies whether the mobile user and the vehicle owner are the same user based on the received VIN code. If yes, the SDK agrees to pull up the super space module; if no, the SDK does not agree to pull up the space module. After pulling up the super space module, the short-range data transmission channel between the super space application in the mobile terminal and the vehicle terminal is officially connected. Subsequently, the transmission of data and instructions can be reliably transmitted between the vehicle terminal and the mobile terminal through this special link.

[0172] The establishment process of the 4G / 5G remote channel: establish a 4G / 5G interconnection channel between the vehicle terminal and the cloud, and a 4G / 5G interconnection channel between the cloud and the mobile terminal. The remote channel is a 4G / 5G remote channel indirectly established between the vehicle terminal and the mobile terminal through the cloud. The mobile terminal obtains the event code reported by the vehicle terminal through the cloud and formulates a communication interface between the mobile APP and the cloud. By calling the communication interface, the vehicle condition information reported by the vehicle terminal is obtained for display of the vehicle state.

[0173] It should be noted that the data format and encryption algorithm of the autonomous WIFI channel are customized: (1) The data transmission encryption verification process of the mobile phone end and the vehicle machine end is as follows: the mobile phone and the vehicle machine end prestore KEY2 of HMAC in the Keystore for signing. The mobile phone end dynamically generates a pair of RSA public key pub1 and private key priv1 (generated through Android Keystore), and then transmits the public key pub1 in plaintext to the vehicle machine end. After receiving the public key pub1, the vehicle machine end dynamically generates an AES (AES128 and above) symmetric key KEY1, and then transmits the AES symmetric key KEY1 to the mobile phone end through RSA public key pub1 encryption, and the mobile phone end uses the private key priv1 to decrypt to obtain the symmetric key key1, so that the mobile phone end and the vehicle machine end complete key negotiation. The mobile phone end uses the AES symmetric key KEY1 to encrypt data and sends it to the vehicle machine end. It should be noted that the mobile phone end and the vehicle machine end use KEY2 of HMAC for signing and verification when transmitting data to ensure the integrity of data transmission. In this way, this encryption scheme considers saving the performance of both ends while meeting the information security requirements of short-range transmission. (2) MQTT protocol format definition: mqtt message topic cluster based on json serialization, as shown in Table 1. JSON message structure description: payload uses a json string, as described in Table 2. It should be noted that the long-range and short-range protocols remain the same and can be switched independently.

[0174] Of course, the sender adds a timestamp (TS) in the content, and the receiver verifies the timestamp to achieve the purpose of tamper-proofing; in addition, the private key priv1 needs to ensure confidentiality, the public key pub1 needs to prevent tampering, key1 is stored in a secure module or encrypted storage, and the encrypted key is stored in a secure module or encrypted storage to achieve secure storage.

[0175] Third step, after establishing the screen projection channel and the data transmission channel, the mobile phone end integrates an interconnection application program APP, and the application program APP data in the mobile phone end includes but is not limited to wallpaper, 3D car model, vehicle condition and car control card in the super space function. The mobile phone converts the screen projection data associated with the super space function in the APP data to form video data. The video data includes the data of the function interface of the super space function. Specifically, in this process, the application program can call the vehicle machine end interface to obtain the window size information of each vehicle machine display screen, and the interface display of the application program is adapted according to the window size, and the display of the application program interface and information is completed.

[0176] Fourth step, transmit the screen projection video data to the vehicle machine end 4 through WIFI screen projection channels such as Hicar, Carlink, HONOR CarConnect, etc.

[0177] The fifth step, after the WiFi module of the car machine end receives the screen projection video data, the screen projection video data is converted and adapted 5, and the original hand-car interconnection interface is displayed on the car machine display screen.

[0178] The sixth step, the mobile phone obtains the vehicle condition data 6 of the car machine end through the self-built WIFI channel or the 4G / 5G remote channel, and the hand-car interconnection page associated with the super space function is rendered and updated based on the vehicle condition data. The hand-car interconnection page associated with the super space function can be entered through the 3D car control entry in the original hand-car interconnection interface. The vehicle condition data can be the data in the business component or the core service module obtained by the car machine end through the interconnection APP, and the vehicle condition data can be actively captured by the mobile phone, or the data actively transmitted to the mobile phone by the car machine end after the user performs a touch operation.

[0179] It should be noted that the mobile phone end can arbitrate and feedback the long-range channel. After the self-built WIFI channel is pulled up, the long-range fusion module of the mobile phone end arbitrates the long-range channel selection. The short-range channel is preferentially selected. If the communication is disconnected during transmission and 3 times of reconnection fail, the long-range channel is switched. If an error code "the vehicle is in a network unstable state" is received during transmission in the long-range channel, the short-range transmission channel is switched.

[0180] The seventh step, the user generates a screen click event through the car machine display screen, and the interconnection APP listens to the car machine display screen click event 7 through the super space. When there is a click event, the click event is reported 8 to the WiFi module or the Tbox module. The WiFi module of the car machine end reports the click event 8 to the WiFi module of the mobile phone end, and reports the click event to the mobile phone. The WiFi module of the mobile phone notifies 9 the interconnection APP of the click event. The APP listens to the trigger of the folding button of the rearview mirror after being turned on, generates a car control instruction, and the car control instruction can be an MQTT car control instruction.

[0181] It should be noted that the car end control logic software is placed in the mobile phone end to obtain the vehicle condition instruction from the long-range channel, which is used for 3D car model state display, and then the folding of the rearview mirror, the trunk, the scene mode and other setting software are realized by the mobile phone, and the car control instruction is transmitted. At the same time, combined with the data storage of the mobile phone server for analysis, such as controlling the day / night mode of the wallpaper according to the current time, controlling the sunny, rainy, cloudy, snowy and cloudy modes of the wallpaper according to the weather data; the user can also set the color of the car model according to his own preference.

[0182] The eighth step, the application APP transmits the encrypted MQTT car control instruction 10 to the WiFi module or Tbox module of the car machine end through the self-built WIFI channel or 4G / 5G remote channel after the MQTT car control instruction is encrypted by a private key.

[0183] The ninth step is that the WIFI module or the Tbox module of the vehicle machine end transmits the encrypted vehicle control instruction to the Internet APP. The Internet APP decrypts the received encrypted vehicle control instruction through the public key exchanged in advance, and transmits the decrypted vehicle control instruction to the local semantic analysis module. The semantic analysis module intelligently analyzes the generalized semantics, and transmits the final vehicle control instruction to the vehicle body controller 11. The vehicle body controller transmits the received vehicle control instruction to the corresponding actuator to execute the vehicle control instruction 12. After the execution of the actuator is completed, the execution result is fed back to the mobile phone end 13 through the vehicle body controller, the Internet APP, and the WIFI module. The mobile phone end receives the feedback execution result through the WIFI module, and transmits the execution result to the Internet APP of the mobile phone end. The Internet APP displays the execution result after decryption, and displays the execution result to the vehicle machine display screen through the screen projection channel, which is synchronized with the current state of the vehicle model rearview mirror displayed on the vehicle machine display screen.

[0184] As can be seen from the above, the embodiment of the application decouples the vehicle control instruction transmission data channel from the mobile phone manufacturer by self-establishment, and has strong autonomy and expandability. The control link can be intelligently selected according to the WIFI and 4G / 5G network conditions, the long-range and short-range fusion is completed, and the long-range and short-range protocols both use the standardized MQTT interface, which can be reused. The vehicle machine end receives the control instruction sent by the mobile phone terminal, transmits the control instruction to the vehicle machine semantic AI module, can complete intelligent semantic analysis, and reuses the voice execution link to reduce the occupation of vehicle machine resources.

[0185] The embodiment of the application provides a vehicle control device, as shown in Figure 7 , and Figure 7 The embodiment of the application provides a vehicle control device, as shown in

[0186] The obtaining module 701 is configured to obtain first vehicle data transmitted by the vehicle machine end through the short-range data transmission channel if the mobile terminal and the vehicle machine end integrated with the Internet software establish a short-range data transmission channel and a screen projection transmission channel. The short-range data transmission channel and the screen projection transmission channel are two different transmission channels established by using the same communication module. The message transmission protocol of the short-range data transmission channel is a target message transmission protocol, and the target message transmission protocol is a message queue telemetry transmission protocol.

[0187] The processing module 702 is configured to generate a hand-vehicle interconnection page with a super space adapted to the vehicle machine display screen of the vehicle machine end in the super space module of the mobile terminal based on the first vehicle data, and project the hand-vehicle interconnection page to the vehicle machine display screen through the screen projection transmission channel. The super space module is a functional module capable of generating and displaying vehicle state information and vehicle functions, and controlling the vehicle functions.

[0188] The obtaining module 701 is further configured to obtain a touch event for the vehicle-machine interconnection page reported by the vehicle terminal through the short-range data transmission channel.

[0189] The determining module 703 is configured to determine a vehicle control instruction corresponding to the touch event.

[0190] The sending module 704 is configured to send the vehicle control instruction to the vehicle terminal through the short-range data transmission channel by using the target message transmission protocol, so that the vehicle terminal receives the vehicle control instruction and delivers the vehicle control instruction to an execution unit corresponding to the vehicle control instruction for instruction execution.

[0191] An embodiment of the present application provides a hardware entity schematic diagram of a computer device, as shown in the figure, the computer device can be a mobile terminal, and the hardware entity of the computer device 8 includes a processor 801, a memory 802 and a transceiver 803. Figure 8 The memory 802 stores a computer program executable on the processor 801, and the transceiver 803 is configured to perform the operation of receiving data or sending data under the control of the processor; and the processor 801 implements part or all of the steps of the vehicle control method in the above embodiment when executing the computer program.

[0192] The memory 802 stores a computer program executable on the processor 801, and the memory 802 is configured to store instructions and applications executable by the processor 801, and can also cache data (for example, image data, audio data, voice communication data and video communication data) to be processed by the processor 801 and each module in the computer device 8, which can be realized by FLASH or RAM.

[0193] The processor 801 executes the program to implement the steps of the vehicle control method of any one of the above embodiments. The processor 801 generally controls the overall operation of the computer device 8.

[0194] The transceiver 803 receives data or sends data under the control of the processor.

[0195] An embodiment of the present application provides a computer readable storage medium, which stores one or more computer programs, and the one or more computer programs can be executed by one or more processors to implement part or all of the steps of the above method. The storage medium can be transitory or non-transitory.

[0196] An embodiment of the present application provides a computer program, which includes computer readable code, and when the computer readable code runs in a computer device, a processor in the computer device executes part or all of the steps of the above method.

[0197] The embodiments of the present application provide a computer program product, which comprises a non-transitory computer readable storage medium storing a computer program. The computer program is read and executed by a computer to implement some or all of the steps in the above method. The computer program product can be implemented by hardware, software or a combination thereof. In some embodiments, the computer program product is embodied in a computer storage medium. In other embodiments, the computer program product is embodied in a software product, such as a software development kit (SDK) or the like.

[0198] It should be noted that the above description of the various embodiments tends to emphasize the differences between the various embodiments, and the same or similar parts can be referred to each other. The above description of the device, storage medium, computer program and computer program product embodiments is similar to the description of the method embodiments, and has similar advantages. For technical details not disclosed in the device, storage medium, computer program and computer program product embodiments of the present application, please refer to the description of the method embodiments.

[0199] The processor can be at least one of an application specific integrated circuit (ASIC), a digital signal processor (DSP), a digital signal processing device (DSPD), a programmable logic device (PLD), a field programmable gate array (FPGA), a central processing unit (CPU), a controller, a microcontroller, and a microprocessor. It can be understood that the electronic device for implementing the functions of the processor can also be other devices, and the embodiments of the present application are not limited.

[0200] The computer storage medium / memory can be a Read Only Memory (ROM), a Programmable Read-Only Memory (PROM), an Erasable Programmable Read-Only Memory (EPROM), an Electrically Erasable Programmable Read-Only Memory (EEPROM), a Ferromagnetic Random Access Memory (FRAM), a Flash Memory, a magnetic surface storage, an optical disc, a Compact Disc Read-Only Memory (CD-ROM), or the like memory; or can be various terminals including one or any combination of the above memories, such as a mobile phone, a computer, a tablet device, a personal digital assistant, and the like.

[0201] It should be understood that every monetary term referred to throughout the specification means that the specific feature, structure or characteristic related to the embodiment is included in at least one embodiment of the present application. Therefore, "in one embodiment" or "in an embodiment" appearing throughout the specification does not necessarily refer to the same embodiment. In addition, these specific features, structures or characteristics can be combined in any suitable manner in one or more embodiments. It should be understood that the size of the serial number of each step / process described above does not mean the order of execution, and the execution order of each step / process should be determined according to its function and inherent logic, and should not constitute any limitation on the implementation process of the embodiments of the present application. The serial number of the above embodiments of the present application is only for description, not representing the advantages and disadvantages of the embodiments.

[0202] It should be noted that in this document, the terms "comprise", "comprising", or any other variant thereof are intended to cover non-exclusive inclusions, so that processes, methods, articles or devices that include a series of elements not only include those elements, but also include other elements not explicitly listed, or include elements inherent to such processes, methods, articles or devices. Without more limitations, the element defined by the statement "comprising a" does not exclude the presence of other identical elements in the process, method, article or device including the element.

[0203] In several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other manners. The described device embodiments are merely illustrative, for example, the division of the units is only a logical function division, and there can be another division manner in actual implementation, for example, a plurality of units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the displayed or discussed coupling, or direct coupling or communication connection between the components can be indirect coupling or communication connection through some interfaces, devices or units, and can be electrical, mechanical or other forms.

[0204] The units described as separate components can or can not be physically separate, and the components shown as units can or can not be physical units; they can be located in one place, or distributed on multiple network units; and some or all of the units can be selected according to actual needs to achieve the purpose of the embodiment.

[0205] In addition, each functional unit in each embodiment of the present application can be integrated into one processing unit, or each unit can be a separate unit, or two or more units can be integrated into one unit; the integrated unit can be realized in the form of hardware or in the form of hardware plus software functional units.

[0206] Those of ordinary skill in the art can understand that all or part of the steps of the above-mentioned method embodiments can be completed by a program instructing related hardware, and the foregoing program can be stored in a computer readable storage medium, and the program executes the steps of the above-mentioned method embodiments when executed; and the foregoing storage medium includes mobile storage devices, read-only memories (ROM), magnetic discs or optical discs, and various storage media that can store program codes.

[0207] Alternatively, the integrated units of the present application, if implemented in the form of software functional modules and sold or used as independent products, can also be stored in a computer readable storage medium. Based on this understanding, the technical solutions of the present application can be embodied in the form of a software product, and the computer software product is stored in a storage medium, and includes a number of instructions for causing a vehicle terminal (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the methods described in the embodiments of the present application. The foregoing storage medium includes mobile storage devices, ROM, magnetic discs or optical discs, and various storage media that can store program codes.

[0208] The above merely provides the implementation of the present application, but the protection scope of the present application is not limited thereto, any person skilled in the art can easily think of the change or replacement within the technical range disclosed by the present application, which should be covered in the protection scope of the present application.

Claims

1. A vehicle control method, characterized in that, Applied to mobile devices, the method includes: If the mobile terminal establishes a short-range data transmission channel and a screen projection transmission channel with the vehicle terminal integrated with the interconnection software, the first vehicle data transmitted by the vehicle terminal through the short-range data transmission channel is obtained; wherein, the short-range data transmission channel and the screen projection transmission channel are two different transmission channels established using the same communication module, and the message transmission protocol of the short-range data transmission channel is a target message transmission protocol, and the target message transmission protocol is a message queue telemetry transmission protocol; Based on the first vehicle data, a hand-vehicle interconnection page with super space is generated in the super space module of the mobile terminal and adapted to the vehicle display screen of the vehicle terminal. The hand-vehicle interconnection page is then projected to the vehicle display screen through the projection transmission channel. The super space module is a functional module that can generate and control vehicle functions to display vehicle status information and vehicle functions. The system obtains touch events for the vehicle-to-vehicle interconnection page reported by the vehicle-mounted terminal through the short-range data transmission channel. The vehicle control command corresponding to the touch event is determined, and the vehicle control command is sent to the vehicle terminal through the short-range data transmission channel using the target message transmission protocol, so that after the vehicle terminal receives the vehicle control command, it passes it to the execution unit corresponding to the vehicle control command for command execution.

2. The method according to claim 1, characterized in that, The establishment process of the screen projection transmission channel and the short-range data transmission channel includes: In response to the mobile terminal scanning for a Bluetooth device in the vehicle's infotainment system, a Bluetooth pairing code is displayed; Send a Bluetooth pairing request carrying the Bluetooth pairing code to the vehicle's infotainment system so that the vehicle's infotainment system can automatically verify the pairing based on the Bluetooth pairing code and turn on the Wi-Fi hotspot. In response to the mobile terminal connecting to the vehicle's WiFi hotspot, it is determined that the mobile terminal establishes the screen projection transmission channel with the vehicle's WiFi hotspot; The system interface is invoked to send a channel establishment request for the short-range data transmission channel to the vehicle-mounted terminal; The system receives feedback information sent by the vehicle-mounted terminal through the system interface, and based on the feedback information, establishes a short-range data transmission channel between the mobile terminal and the vehicle-mounted terminal via a WiFi hotspot.

3. The method according to claim 1, characterized in that, The step of sending the vehicle control command to the vehicle terminal via the short-range data transmission channel using the target message transmission protocol includes: A first asymmetric key pair is pre-generated through a first secure storage module; wherein the first asymmetric key pair includes a first private key and a first public key; The vehicle control command is encrypted using the first private key to obtain the ciphertext vehicle control command; The encrypted vehicle control command is sent to the vehicle terminal via the short-range data transmission channel using the target message transmission protocol, so that the vehicle terminal can decrypt the encrypted vehicle control command using the pre-transmitted first public key to obtain the vehicle control command.

4. The method according to claim 1, characterized in that, The step of sending the vehicle control command to the vehicle terminal via the short-range data transmission channel using the target message transmission protocol includes: A second asymmetric key pair is pre-generated through the first secure storage module, wherein the second asymmetric key pair includes a second private key and a second public key; The vehicle-mounted terminal obtains the encrypted symmetric key and the first signature value sent through the short-range data transmission channel; wherein, the encrypted symmetric key is a symmetric key dynamically generated by the vehicle-mounted terminal through the second secure storage module, and is obtained by encrypting the symmetric key using the pre-transmitted second public key; the first signature value is obtained by the vehicle-mounted terminal digitally signing the encrypted symmetric key using the first key hash operation message verification code HMAC. The encrypted symmetric key is signed and verified using the second HMAC stored in the first secure storage module. If the verification is successful, the encrypted symmetric key is decrypted using the second private key to obtain the symmetric key; wherein, the first HMAC and the second HMAC are the same. The vehicle control command is encrypted using the symmetric key to obtain a ciphertext vehicle control command. The ciphertext vehicle control command is then digitally signed using the second HMAC to obtain a second signature value. Through the short-range data transmission channel, using the target message transmission protocol, the encrypted vehicle control command and the second signature value are sent to the vehicle terminal, so that the vehicle terminal can verify the second signature value through the first HMAC. If the verification is successful, the encrypted vehicle control command is decrypted using the symmetric key to obtain the vehicle control command.

5. The method according to any one of claims 1 to 4, characterized in that, The step of determining the vehicle control command corresponding to the touch event includes: The target application being touched is determined based on the touch event; Based on the touch event and the functions configured in the target application, determine the target function indicated by the touch event; According to the structured dataset corresponding to the target function in the preset corpus, the vehicle control command for the target function is generated; wherein, the preset corpus is jointly determined by the mobile terminal and the vehicle terminal, so that the vehicle control command generated based on the preset corpus can reuse the semantic parsing module of the vehicle terminal.

6. The method according to any one of claims 1 to 4, characterized in that, The super space module includes one or more of the following elements: wallpaper, 3D car model, vehicle condition card, and vehicle control card. Based on the first vehicle data, the super space module on the mobile terminal generates a vehicle-to-mobile interconnection page with super space adapted to the vehicle's infotainment display screen, including: The system can obtain environmental information and user preferences regarding the location of the vehicle-mounted device. Based on the environmental information, the form of the wallpaper in the super space module is dynamically adjusted; Based on the body component status data in the first vehicle data, the status of the three-dimensional vehicle model in the super space module is displayed synchronously; wherein, the body component status data is used to indicate the working status of each body component in the vehicle. Based on the vehicle detection data in the first vehicle data, the vehicle condition card in the super space module is displayed synchronously; wherein, the vehicle detection data is used to indicate the vehicle's basic operating capabilities, safety status, and key functional performance. Based on the user preference function, the vehicle control function displayed on the vehicle control card in the super space module is determined; wherein, the vehicle control function is used to instruct the user to perform a touch operation; Based on the wallpaper, the 3D car model, the vehicle condition card, and the vehicle control card, a user interface page with super space is generated that is adapted to the vehicle's infotainment display screen.

7. The method according to any one of claims 1 to 4, characterized in that, The method further includes: If the mobile terminal establishes a remote data transmission channel with the vehicle terminal, it obtains the first network connection information of the short-range data transmission channel and the second network connection information of the remote data transmission channel; wherein, the message transmission protocol of the short-range data transmission channel and the remote data transmission channel is the same; Based on the first network connection information and / or the second network connection information, the short-range data transmission channel and the long-range data transmission channel are selected to determine the target data transmission channel, wherein the target data transmission channel is used to transmit the first vehicle data and / or the vehicle control command.

8. The method according to any one of claims 1 to 4, characterized in that, The data transmission format corresponding to the target message transmission protocol is the data format agreed upon between the mobile terminal and the vehicle terminal.

9. A vehicle control device, characterized in that, The device includes: The module is configured to obtain first vehicle data transmitted by the vehicle terminal through the short-range data transmission channel if the mobile terminal establishes a short-range data transmission channel and a screen projection transmission channel with the vehicle terminal integrated with the interconnection software; wherein the short-range data transmission channel and the screen projection transmission channel are two different transmission channels established using the same communication module, and the message transmission protocol of the short-range data transmission channel is a target message transmission protocol, and the target message transmission protocol is a message queue telemetry transmission protocol; The processing module is used to generate a vehicle-to-hand communication page with super space adapted to the vehicle display screen of the vehicle terminal in the super space module of the mobile terminal based on the first vehicle data, and project the vehicle-to-hand communication page to the vehicle display screen through the projection transmission channel. The super space module is a functional module that can generate a display of vehicle status information and vehicle functions and control the vehicle functions. The obtaining module is also used to obtain touch events for the vehicle-to-vehicle interconnection page reported by the vehicle terminal through the short-range data transmission channel; The determination module is used to determine the vehicle control command corresponding to the touch event; The sending module is used to send the vehicle control command to the vehicle terminal via the short-range data transmission channel using the target message transmission protocol, so that after receiving the vehicle control command, the vehicle terminal can pass it to the execution unit corresponding to the vehicle control command for command execution.

10. A computer device, characterized in that, The computer device includes: a transceiver, a memory, and a processor. The memory stores computer programs that can run on a processor; The transceiver is used to perform the operation of receiving or sending data under the control of the processor; When the processor executes the computer program, it implements the vehicle control method as described in any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores one or more computer programs, which can be executed by one or more processors to implement the vehicle control method as described in any one of claims 1 to 8.

12. A computer program product, comprising a computer program or instructions, characterized in that, When the computer program or instructions are executed by the processor, they implement the vehicle control method according to any one of claims 1 to 8.

Images