Vehicle and intelligent terminal interaction method and device, equipment and medium

Abstract

The invention provides an interaction method and device of a vehicle and an intelligent terminal, electronic equipment and a storage medium, and the method comprises the steps: generating a terminal control instruction corresponding to a target intelligent terminal at a vehicle side under the condition that an interaction triggering condition is satisfied; searching a target application programming interface corresponding to the terminal control instruction of the target intelligent terminal from an application programming interface set, wherein application programming interfaces collected from at least one intelligent terminal are collected in the application programming interface set; and calling the target application programming interface to obtain an execution result corresponding to the terminal control instruction. According to the scheme, data, functions and the like of various intelligent terminals can be uniformly packaged and integrated to the vehicle-mounted control platform, and when certain triggering conditions are met, the vehicle automatically calls or controls the functions of the intelligent terminals, so that autonomous interaction between the vehicle and the various intelligent terminals is realized.

Description

Technical Field

[0001] The present application relates to the technical field of Internet of Things, and in particular to a method, device, electronic device and storage medium for interaction between a vehicle and a smart terminal. Background Art

[0002] With the development of automobile intelligence and information technology, the in-vehicle system has assumed more and more functions, and as one of the communication hubs of the Internet of Things, it communicates and interacts with a variety of external devices. Smart terminal devices of different types or produced by different manufacturers are often designed based on different operating systems and software architectures, resulting in inconsistent functional interfaces of various smart terminal devices, which cannot be called through a separate control platform; and different types of smart terminals are usually connected to the Internet of Things through different wireless communication methods. As a result, there is isolation between the functional design and communication methods of various smart terminals, making it difficult to control them uniformly. Summary of the invention

[0003] The embodiments of the present application provide a method, device, electronic device and storage medium for interaction between a vehicle and a smart terminal to solve one or more of the above-mentioned technical problems.

[0004] In a first aspect, an embodiment of the present application provides a method for interaction between a vehicle and a smart terminal, including:

[0005] When the interaction triggering condition is met, a terminal control instruction corresponding to the target intelligent terminal is generated on the vehicle side;

[0006] Searching for a target application programming interface corresponding to the terminal control instruction of the target intelligent terminal from an application programming interface set, wherein the application programming interface set includes application programming interfaces collected from at least one intelligent terminal;

[0007] The target application programming interface is called to obtain an execution result corresponding to the terminal control instruction.

[0008] In a second aspect, an embodiment of the present application provides an interactive device between a vehicle and a smart terminal, including:

[0009] A terminal control instruction generation module, used to generate a terminal control instruction corresponding to a target intelligent terminal on the vehicle side when an interaction triggering condition is met;

[0010] An application programming interface determination module, used to search for a target application programming interface corresponding to the terminal control instruction of the target intelligent terminal from an application programming interface set, wherein the application programming interface set includes application programming interfaces collected from at least one intelligent terminal;

[0011] The application programming interface calling module is used to call the target application programming interface to obtain the execution result corresponding to the terminal control instruction.

[0012] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the above methods when executing the computer program.

[0013] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the above is implemented.

[0014] Compared with the related art, this application has the following advantages:

[0015] According to the embodiment of the present application, when the interaction triggering conditions are met, a terminal control instruction corresponding to the target smart terminal is generated on the vehicle side; a target application programming interface corresponding to the terminal control instruction of the target smart terminal is searched from an application programming interface set, wherein the application programming interface set includes application programming interfaces collected from at least one smart terminal; and the target application programming interface is called to obtain an execution result corresponding to the terminal control instruction. Through the above scheme, the data and functions of various smart terminals produced by different manufacturers, based on different communication protocols, and of different types can be uniformly abstracted and encapsulated, and the encapsulated application programming interface can be integrated into the vehicle control platform. When certain triggering conditions are met, the vehicle automatically completes the call to the application programming interface to control the corresponding smart terminal to execute the corresponding instruction, thereby realizing autonomous and intelligent interaction between the vehicle and various smart terminals.

[0016] The above description is only an overview of the technical solution of the present application. In order to more clearly understand the technical means of the present application, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present application more obvious and easy to understand, the specific implementation methods of the present application are listed below. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In the accompanying drawings, unless otherwise specified, the same reference numerals throughout the multiple drawings represent the same or similar parts or elements. These drawings are not necessarily drawn to scale. It should be understood that these drawings only depict some embodiments according to the present application and should not be regarded as limiting the scope of the present application.

[0018] Figure 1 A schematic diagram showing a flow chart of a method for interaction between a vehicle and an intelligent terminal provided in an embodiment of the present application;

[0019] Figure 2 A schematic diagram of a flow chart of generating interactive control rules in a method for interacting with a vehicle and an intelligent terminal provided in an embodiment of the present application is shown;

[0020] Figure 3 A schematic diagram showing a communication link between a vehicle and an intelligent terminal provided in an embodiment of the present application is shown;

[0021] Figure 4 A structural block diagram of a vehicle-intelligent terminal interaction device provided in an embodiment of the present application is shown; and

[0022] Figure 5 A block diagram of an electronic device used to implement an embodiment of the present application is shown. DETAILED DESCRIPTION

[0023] In the following, only some exemplary embodiments are briefly described. As those skilled in the art will appreciate, the described embodiments may be modified in various ways without departing from the concept or scope of the present application. Therefore, the drawings and descriptions are considered to be exemplary in nature and not restrictive.

[0024] To facilitate understanding of the technical solutions of the embodiments of the present application, the following describes the related technologies of the embodiments of the present application. The following related technologies can be combined with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application.

[0025] The Internet of Things (IoT) is a system of interconnected computing devices, machines, and digital machines with a universal unique identifier (UID) and the ability to transmit data over the Internet without the need for human-to-human or human-to-device interaction. The application areas of the Internet of Things mainly include the following: transportation and logistics, industrial manufacturing, health care, smart environment (home, office, factory), personal and social fields, etc. The method of interaction between a vehicle and an intelligent terminal provided in an embodiment of the present application belongs to the application of Internet of Things technology in the field of smart cars. The vehicle system can receive operation requests for functional components in the vehicle or intelligent terminals outside the vehicle, and communicate with the corresponding components or terminals to enable them to provide services corresponding to the operation requests, so as to realize the interconnection and intercommunication between the vehicle and various intelligent terminals.

[0026] The realization of the Internet of Things requires that each object connected to the Internet of Things be assigned a unique identifier or address. The Internet of Things has a variety of networking technologies to choose from, which can be divided into short-range wireless, medium-range wireless, long-range wireless, and wired technologies according to the effective transmission distance. Among them, wireless access is the core technology of the Internet of Things device side. Wireless IoT devices are generally divided into three categories. One category is that they naturally support TCP / IP and can directly access the Internet of Things, such as WI-FI, GPRS / 3G / 4G / 5G and other devices; the other category is that they do not support IP protocols and require a gateway (protocol conversion) to access the Internet of Things, such as Zigbee, Bluetooth devices, or devices that communicate directly with smart devices through medium- and short-range wireless communications (such as microwave radio).

[0027] In order to control the devices connected to the Internet of Things, it is also necessary to encapsulate the functions or services of each device into a unified interface so that the application can call each interface to implement the corresponding function. At present, each smart device manufacturer often configures proprietary application software for its manufactured devices to manage them. The device management platforms of different manufacturers are not interoperable, and the data transmission specifications and communication protocols they follow are quite different. That is, the application equipped by manufacturer A for its smart device cannot be used to control the smart device produced by manufacturer B, which brings difficulties to the interaction of devices in the Internet of Things.

[0028] The vehicle-to-intelligent terminal interaction method provided in the embodiment of the present application integrates intelligent devices of different types and produced by different manufacturers into a unified platform. After receiving an operation instruction for the intelligent terminal, the platform can call the corresponding functional interface of the intelligent terminal through the communication link corresponding to the intelligent terminal to control it to provide corresponding services.

[0029] The execution subject of the embodiment of the present application may be an application, service, instance, software-based functional module, virtual machine (VM), container or cloud server, etc., or a hardware device with data processing function (such as a server or terminal device) or hardware chip (such as CPU, GPU, FPGA, NPU, AI acceleration card or DPU), etc. The device for realizing the interaction between the vehicle and the intelligent terminal can be deployed on the computing device of the application party providing the corresponding service or on the cloud computing platform providing computing power, storage and network resources.

[0030] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0031] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The several specific embodiments listed can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0032] Figure 1 FIG. 1 is a flow chart of a method 100 for interacting between a vehicle and a smart terminal provided in an embodiment of the present application. Figure 1 As shown, the method 100 may include the following steps S101-S103: S101, generating a terminal control instruction corresponding to the target smart terminal on the vehicle side when an interaction trigger condition is met; S102, searching for a target application programming interface corresponding to the terminal control instruction of the target smart terminal from an application programming interface set, wherein the application programming interface set includes application programming interfaces collected from at least one smart terminal; and S103, calling the target application programming interface to obtain an execution result corresponding to the terminal control instruction.

[0033] The method 100 is described in detail step by step below.

[0034] In step S101, when the interaction triggering condition is met, a terminal control instruction corresponding to the target smart terminal is generated on the vehicle side.

[0035] In one possible implementation, the smart terminal in the embodiment of the present application may include a vehicle's on-board equipment, smart home equipment, smart facilities or smart mobile devices; the smart terminal communicates with the vehicle through one or more communication methods including Ethernet, 4 / 5G network, narrowband Internet of Things, microwave radio, Bluetooth and WI-FI.

[0036] That is, the interactive method provided in the embodiments of the present application covers different types of smart terminals, including smart home terminals (such as smart TVs, smart lamps, smart air conditioners, etc.), smart public facilities (such as smart charging piles), smart mobile devices (such as drones), cars (specifically, their car systems), etc. In fact, the present application expands the scope of smart devices connected to the Internet of Things to general pan-intelligent terminals, that is, devices equipped with sensors, actuators and can communicate with other devices. It can be understood that the vehicle equipped with an application for managing and controlling the smart terminal, that is, the vehicle as the subject of the method implementation in the embodiments of the present application, its configured functional units or execution components also belong to the scope of the above-mentioned pan-intelligent terminals, and the vehicle system of the vehicle can communicate data with the components or functional units in the vehicle through communication methods such as Ethernet and CAN to achieve control operations on them.

[0037] In some embodiments, when the interaction trigger conditions are met, generating terminal control instructions corresponding to the target smart terminal on the vehicle side includes: obtaining a control operation submitted through a control entry corresponding to the target smart terminal in the vehicle-mounted interface, and generating a terminal control instruction corresponding to the target smart terminal on the vehicle side.

[0038] The vehicle-mounted interface may run an application for managing and controlling the smart terminal. The interaction trigger condition may be a control operation or control instruction input by the user through interaction with the user interface of the application. The system parses the received control instruction and obtains the terminal control instruction corresponding to the target smart terminal. The submitted control operation may include one or more control operations on the same or different devices, and the terminal control instruction corresponding to each of the above control operations is generated on the vehicle side.

[0039] In a possible embodiment, the system can receive control operations on the target smart terminal input by the user by touching or clicking a function button (such as "start drone") of the smart device on the user interface of the application in the vehicle touch screen through air gestures; similarly, the control operation can also be a control operation input by the user through the user interface on a smart device such as a mobile phone or tablet computer, and the control operation can be first uploaded to the cloud server and then forwarded by the cloud server to the vehicle system for processing.

[0040] In a possible embodiment, the control operation can also be input by the user into the system voice receiving module in the form of voice, and the control instruction for the smart terminal is obtained through voice recognition and semantic recognition conversion.

[0041] In some embodiments, when the interaction trigger condition is met, generating a terminal control instruction corresponding to the target smart terminal on the vehicle side also includes: when the vehicle sensor detects that the operating state of the vehicle meets the trigger condition, generating a terminal control instruction corresponding to the target smart terminal on the vehicle side.

[0042] In an embodiment of the present application, one or more control operations on the smart terminal and trigger conditions corresponding to the terminal control instructions can be pre-defined, that is, when the system detects that the trigger condition is met, a terminal control instruction corresponding to the target smart terminal is automatically generated.

[0043] In some embodiments, the interaction trigger condition is, for example, that a sensor deployed on the vehicle detects that a certain change has occurred in a parameter of the vehicle's operating state or that a certain value has been satisfied. For example, the interaction trigger condition is set to the vehicle being parked in a parking space (a distance sensor deployed on the vehicle detects that the distance to a certain fixed position in the parking space is less than a preset value), and the terminal control instruction corresponding to the interaction trigger condition may be closing the vehicle's windows, turning on the indoor living room lights, and turning on the TV.

[0044] The interactive triggering conditions in the embodiments of the present application may include changes in the state of the vehicle itself, changes in the internal and external environments of the vehicle detected by sensors, detectors and other equipment deployed on the vehicle, and signals received by the vehicle from smart terminals that communicate with the vehicle through a network (including wired and wireless methods). That is, the IoT smart terminal that communicates with the vehicle is both a trigger (triggering other smart terminals in the IoT to perform certain actions) and an actuator (itself receiving external signals to trigger it to perform certain actions). For example, in the above example of a vehicle parked in a parking space, its interactive triggering condition may be set to a distance sensor deployed by the vehicle detecting that the distance between the vehicle body and a fixed position in the parking space is less than a preset value, or may be set to a signal sent by a smart sensing facility set at the parking space received by the vehicle communication module, indicating that the distance between the vehicle body and the parking space has reached a preset value (i.e., the vehicle has parked in the parking space), and the signal triggers the vehicle to control the controlled components in the vehicle and / or notify the smart terminal communicating with the vehicle to perform corresponding actions, such as closing the windows, turning on the indoor living room lights, and turning on the TV.

[0045] In some embodiments, if the interaction trigger condition is consistent with the interaction trigger condition of the interaction control rule pre-set for the vehicle, generating the terminal control instruction corresponding to the target smart terminal on the vehicle side includes: calling the rule execution engine to determine the interaction control instruction corresponding to the interaction control rule as the terminal control instruction corresponding to the target smart terminal.

[0046] The interactive control rule includes one or more control operations of the intelligent terminal connected to the vehicle system (via wired or wireless mode) and the interactive triggering conditions for triggering the control operation, which are generated in advance by the user or the system. When the interactive triggering conditions are met, the rule execution engine will automatically retrieve and parse the interactive control rules corresponding to the interactive triggering conditions to obtain the corresponding control instructions for each intelligent terminal. The rule execution engine parses and arbitrates the interactive triggering rules, obtains the control instructions contained therein, and judges the legality of each control instruction according to the operating status of each functional module or component of the current vehicle. If the control instruction is legal, it generates the corresponding control instructions for each intelligent terminal. For example, when an interactive control rule designed by the user is: when the outside temperature is lower than 28 degrees Celsius, open the door and turn off the air conditioner, when the vehicle is driving at high speed, even if the vehicle's temperature sensor detects that the outside temperature meets the above conditions, the rule execution engine will determine the interactive control rule as illegal and refuse to execute its corresponding control instruction. In addition, the rule execution engine can also determine the priority of each interactive control rule, that is, when the interaction trigger conditions corresponding to multiple interactive control rules are met, determine whether to execute the control instructions corresponding to each interactive control rule and their execution order according to the priority of each interactive control rule.

[0047] The interactive control rules can be pre-designed by the vehicle-side application developer and stored in the vehicle-side / cloud, or they can be created by the user through the graphical operation interface of the vehicle-mounted application platform, through voice input, keyboard clicks, touch clicks, etc. Figure 2 The present invention provides a flow chart of generating interactive control rules based on user voice input in a method for interacting with a vehicle and a smart terminal in an embodiment of the present invention. Figure 2 As shown, the interaction control rules are generated through the following steps S201-S204:

[0048] In step S201, the user's voice data is acquired, and a semantic recognition result corresponding to the voice data is generated;

[0049] In step S202, determining that the semantic recognition result corresponds to an instruction to create a vehicle-based interactive control rule;

[0050] In step S203, an interaction control rule is generated according to the semantic recognition result, wherein the interaction control rule includes an interaction triggering condition and an interaction control instruction;

[0051] In step S204, the interaction control rule is provided to the rule execution engine of the vehicle, so that the rule execution engine notifies the vehicle or smart device to execute the corresponding control instruction sequence after detecting that the corresponding interaction trigger condition is met.

[0052] In step S201, the user's voice data is acquired, and a semantic recognition result corresponding to the voice data is generated.

[0053] The voice data is the voice control instructions input by the user into the user terminal (such as a smart phone, smart tablet, etc.) or the central control system of the vehicle through the voice interaction function. The user terminal or the vehicle central control system uploads the input voice control instructions to the cloud server, and the cloud server inputs the received voice data into the neural network model to obtain the semantic recognition result corresponding to the voice data. The semantic recognition result may include the domain and intent classification of the voice data.

[0054] In step S202, it is determined that the semantic recognition result corresponds to an instruction to create a vehicle-based interactive control rule.

[0055] When the output domain is "vehicle control" and the intention is "create interactive control rules", it is determined that the semantic recognition result corresponds to an instruction to create interactive control rules. For example, when the user issues a voice command that is only "please close the windows and play music", its semantic recognition result belongs to the vehicle control domain, but its intention is not to create interactive control rules, but to execute the given instructions; when the user's voice command contains a sentence like "please help me design a new scene card (that is, the functional name of the interactive control rules on the user side)", the corresponding domain is vehicle control, and the intention is to create interactive control rules. At this time, it can be determined that the semantic recognition result corresponds to an instruction to create interactive control rules for vehicles and smart devices, and the subsequent process of creating interactive control rules can continue.

[0056] In step S203, an interaction control rule is generated according to the semantic recognition result, and the interaction control rule includes an interaction triggering condition and an interaction control instruction.

[0057] When the user's purpose is to create a new interactive control rule, the voice data received by the system not only includes "create a new interactive control rule" or similar statements, but also includes the content of the interactive control rule designed by the specific user. In terms of the actual operation process, the specific voice data acquisition process can be that after the system determines that the semantic recognition result of the voice command issued by the user corresponds to the creation of the interactive control rule, it prompts the user to continue to input the content of the interactive control rule, and obtains the user's complete design of the interactive control rule through multiple rounds of dialogue; it can also obtain the user's complete voice command at one time. When the system determines that the semantic recognition result of the complete voice command includes the creation of the interactive control rule between the vehicle and the smart device, it further generates the interactive control rule based on the remaining part of the complete voice command, that is, the natural language description of the interactive control rule designed by the user.

[0058] In some embodiments, the cloud server generated by the above interactive control rules can send the generated interactive control rules to the vehicle-side scene engine for storage and execution of the vehicle control system. After the vehicle-mounted system detects that the interactive triggering condition is met, it will call the corresponding vehicle-side scene engine to send instructions to the vehicle controlled components or smart terminals described in the interactive control rules to enable them to complete the corresponding actions. The newly created interactive control rules can also be stored in the cloud so that the vehicle-side engine can call them when connected to the Internet.

[0059] In step S204, the interaction control rule is provided to the rule execution engine of the vehicle, so that the rule execution engine notifies the vehicle or smart device to execute the corresponding control instruction sequence after detecting that the corresponding interaction trigger condition is met.

[0060] In some embodiments, when the car detects that the interaction trigger condition is met, it can also arbitrate the interaction control rule to determine whether the current vehicle or smart device state can execute the instruction sequence in the interaction control rule. For example, the interaction control rule set by the user is to open the car's rear cover at 10 o'clock every night to charge the battery. However, at 10 o'clock in the evening, the car is in driving state. At this time, the vehicle-side scene engine determines through arbitration that the interaction control rule cannot be executed at present. If there is no conflict between the current vehicle or smart device state and the instructions in the interaction control rule, the vehicle-side scene engine will send a command signal to the corresponding vehicle controlled component or smart device according to the interaction control rule to notify the vehicle controlled component or smart device to execute the corresponding control instruction sequence.

[0061] The above is step S101, and then proceed to step S102.

[0062] In step S102, a target application programming interface corresponding to the terminal control instruction of the target intelligent terminal is searched from an application programming interface set, wherein the application programming interface set includes application programming interfaces collected from at least one intelligent terminal.

[0063] The control instruction sequence described in the interactive control rules in this application contains one or more control instructions, which are used to instruct or notify the controlled components of the vehicle or the intelligent terminal to complete corresponding actions, which correspond to the atomic functions of the terminal. The atomic function, or atomic capability, is the functional module of the smallest unit of the terminal device. For example, for the air conditioner, turning it on and off, adjusting the temperature, and adjusting the wind speed are all atomic functions. "Turn on the air conditioner to 24 degrees Celsius" contains at least two atomic functions: turning on the air conditioner and adjusting the temperature. In order to achieve refined control of terminals of different types and produced by different manufacturers, the atomic functions of the terminal can be encapsulated into a unified state and operation interface according to the relevant setting documents of different equipment manufacturers for system calls, thereby controlling the terminal to perform the required operations.

[0064] In some embodiments, the application programming interface set includes at least one first-class subset corresponding to different terminal attributes, and the first-class subset includes a first-class application programming interface for executing atomic functions of smart terminals; searching for a target application programming interface corresponding to the terminal control instructions of the target smart terminal from the application programming interface set includes: determining the first-class subset including the target application programming interface according to the terminal attributes of the target smart terminal; and searching for a target application programming interface corresponding to the terminal control instructions of the target smart terminal from the first-class subset.

[0065] In the embodiment of the present application, the application programming interface set includes APIs (Application Programming Interface) corresponding to the atomic functions encapsulated by each smart terminal that accesses the vehicle-mounted application platform through a wired or wireless network, and the atomic function APIs are classified into different sub-sets according to the type or attribute of the smart terminal. The vehicle-side scene engine calls the APIs of the atomic functions of the smart terminal involved in the terminal control instruction according to the generated terminal control instruction to request the corresponding smart terminals to provide corresponding services, and obtains the execution results returned by each smart terminal. The above terminal control instruction may include one or more instructions.

[0066] In a possible embodiment, the API set may include the following subsets: smart home API, drone API, car API and infrastructure API. The smart home API includes atomic functional interfaces of smart home terminals such as smart home appliances, smart lights, and smart door controls; the drone API includes atomic functional interfaces for drone-related operations; the car API includes control interfaces for functional service modules of vehicles other than the vehicle; and the infrastructure API includes functional interfaces for smart public facilities such as public charging piles and parking meters. It should be noted that the above subsets are only for illustration purposes. The actual API set may include APIs for more types of terminals, or may be classified into different API subsets according to different classification rules.

[0067] In some embodiments, the atomic function API in the API set can also be a merged atomic function API obtained by further integrating the initial atomic function set when the smart terminal is produced. For example, for a smart door and window system that includes multiple windows in different locations in the home, in addition to individually encapsulating the atomic function of each window to obtain the corresponding API (i.e., opening / closing a single window, etc.), multiple window functions can also be encapsulated as a whole (opening / closing all windows as a whole). Since users often want to close all windows in the home when they go out, in this scenario, it is more convenient to directly call the encapsulated closing all windows function API than to call the closing function API of each window in turn.

[0068] Each API in the API set involved in the embodiment of the present application can be a function or function library, software, executable file, etc. The present application does not limit its form. As long as it can realize the interaction and calling between different software, applications or functional modules, the above-mentioned functional modules can be software, hardware or a combination of software and hardware. The present application does not limit this.

[0069] In some embodiments, the application programming interface set also includes a second category subset, and the second category subset includes a second category application programming interface used to arrange the first category application programming interface; when there are multiple corresponding target application programming interfaces found from the first category subset, the search for the address information of the target application programming interface corresponding to the terminal control instruction of the target smart terminal from the application programming interface set also includes: determining other target application programming interfaces from the second category subset for arranging the target application programming interface found from the first category subset.

[0070] In an embodiment of the present application, the common atomic functions provided by different types of smart terminals are abstracted and encapsulated to obtain a public atomic function API, which is the second subset of the application programming interface set. The public atomic function API does not correspond to the specific functions of each smart terminal, but describes or specifies the execution logic of the atomic functions of the smart terminal corresponding to each instruction in the terminal control instruction. As an example, the public atomic function API may include a clock API, a serializer API, and a parallelizer API, which are shared by multiple smart terminals. In a possible embodiment, the pre-set interactive control rule is to automatically charge the drone at ten o'clock in the evening. When the vehicle-side engine executes the interactive control rule, it not only calls the API related to charging the drone, but also calls the clock API to obtain the current time. In another possible embodiment, the pre-set interactive control rules are that when someone sits in the co-pilot seat, the windows are automatically closed, the music is turned on, and the front cabin ambient light is turned on. At this time, while calling the corresponding API to control the corresponding functional module to execute instructions, the parallelizer API is also called to determine the execution order of each instruction to be executed simultaneously. Similarly, when the instruction execution process described by the interactive control rule is executed sequentially, the serializer API can be called to implement it.

[0071] The above is step S102, and then proceed to step S103.

[0072] In step S103, the target application programming interface is called to obtain an execution result corresponding to the terminal control instruction.

[0073] The vehicle-side scene engine calls one or more application programming interfaces determined in the application programming interface set according to the terminal control instruction to send a control request corresponding to the terminal control instruction to one or more smart terminals, and obtains the execution result returned by the smart terminal. In the above process, for the smart terminal that accesses the vehicle-mounted application wirelessly, it is necessary to perform data interaction through its corresponding communication link to realize the call of its functional interface.

[0074] Figure 3 The following is a schematic diagram of a communication link between a vehicle and a smart terminal provided in an embodiment of the present application. Figure 3As shown, in some embodiments, the terminal control instructions may include control instructions for smart home devices (IOT devices). Smart home devices may include rice cookers, cameras, televisions, living room lights, air conditioners, etc. The smart home terminal can first be connected to the home local area network through WIFI, Bluetooth (BT) or ZigBee (a wireless network protocol for low-speed and short-distance transmission), and the local area network communicates with the smart home cloud platform (public cloud platform) through narrowband Internet of Things (NB-IOT) / 3G / 4G / 5G / 6G networks, and then accesses the TSP (automotive teleservice provider) platform through a dedicated VPN (virtual private network). The TSP platform can communicate with smart cars, smart phones, tablet computers, smart public facilities and other devices. The intelligent terminal control application can be deployed on the vehicle side, and the TSP can communicate with the vehicle-side intelligent terminal control application of the smart car (this vehicle) through existing IoT communication protocols such as HTTP (Hypertext Transfer Protocol) / MQTT (Message Queue Telemetry Transport Protocol); or, the aforementioned intelligent terminal control application can be directly deployed in the TSP (i.e., cloud application, and the vehicle side is correspondingly configured with the operation interface of the application for users to remotely access the application on the vehicle side) to communicate with the smart home terminal; similarly, the intelligent infrastructure (intelligent public facilities) can also be connected to the corresponding cloud platform through V2I (Vehicle-to-infrastructure), and then communicate with the vehicle-side application platform through protocols such as HTTP / MQTT.

[0075] In some embodiments, the terminal control instructions may include control instructions for functional modules of other vehicles (smart cars). The vehicle (i.e., the vehicle that is the subject of the method execution in the embodiments of the present application) can communicate with other vehicles via Bluetooth (BT), WIFI (wireless network) or V2X (vehicle-to-vehicle communication, such as DSRC, dedicated short-range communication). This communication method is suitable for close distances and occasions with high communication real-time requirements, such as the perception and avoidance of other vehicles during driving; the vehicle can also access the cloud TSP through the on-board Ethernet communication module, and then communicate with other vehicles. This communication method is suitable for long distances and occasions with low communication real-time requirements.

[0076] In some embodiments, the terminal control instructions may also include control instructions for intelligent remote control terminals such as drones. Taking a drone as an example, it communicates with the vehicle-side communication module through a dedicated radio spectrum (e.g., 2.4GHz / 5.8GHz wireless spectrum / infinite radio microwave / 5G) to access the vehicle-side application platform.

[0077] The above-mentioned communication module used for data interaction with different smart terminals through different communication methods can be integrated in the same functional module in the form of software, hardware or a combination of software and hardware, for example, integrated in the vehicle-mounted T-BOX. T-BOX, the full name of Telematics-BOX (Internet of Vehicles control unit), refers to an embedded system installed on a car for controlling and tracking the car. It is mainly composed of a mobile communication unit (4G / 5G), a C-V2X (cellular vehicle-to-everything) communication unit, a GNSS (Global Navigation Satellite System) high-precision positioning module, a microprocessor, an in-vehicle bus controller, a memory and other parts. The communication module used for data interaction with different smart terminals through different communication methods can also be operated as an independent functional module, and this application does not impose any restrictions on this.

[0078] In one possible implementation, before searching for a target application programming interface corresponding to the terminal control instruction of the target smart terminal from an application programming interface set, the atomic function of the target smart terminal can be disassembled and the atomic function of the target smart terminal can be encapsulated into an application programming interface used to execute the atomic function.

[0079] That is, in an embodiment of the present application, the atomic force of the smart terminal can be decomposed in advance and turned into atomic force that can be edited and controlled by the scene engine editor. Atomic force is an atomic function or atomic capability, which is a functional module of the smallest unit of a terminal device. For example, opening, closing, temperature, 37 degrees Celsius, air conditioning, car doors, etc. are all atomic functions, and "turning on the air conditioner to 24 degrees Celsius" includes at least two atomic functions: turning on the air conditioner and adjusting the temperature. In order to achieve refined control of terminals of different types and produced by different manufacturers, the atomic functions of smart terminal devices can be disassembled in advance according to the relevant setting documents of different equipment manufacturers, and encapsulated into a unified state and operation interface for the control system on the vehicle side to call, thereby controlling the smart terminal to perform corresponding operations.

[0080] After the atomic functions of the disassembled target smart terminal are encapsulated into the application programming interface, the target application programming interface corresponding to the terminal control instruction of the target smart terminal can be searched from the application programming interface set, and then the target application programming interface is called to obtain the execution result corresponding to the terminal control instruction. Alternatively, after the atomic functions of the disassembled target smart terminal are encapsulated into the application programming interface, when the interaction triggering conditions are met, the terminal control instruction corresponding to the target smart terminal is generated on the vehicle side, and the target application programming interface corresponding to the terminal control instruction of the target smart terminal is searched from the application programming interface set, and then the target application programming interface is called to obtain the execution result corresponding to the terminal control instruction.

[0081] Furthermore, vehicle-side application developers, users of vehicle-mounted terminals, or vehicle-mounted terminals can independently design interactive control rules corresponding to the packaged application programming interface. When the control system of the vehicle-mounted terminal detects that the interactive trigger conditions are met, for example, the interactive trigger conditions are consistent with the interactive control conditions of the interactive control rules pre-set for the vehicle, the control system of the vehicle-mounted terminal can call the rule execution engine, determine the interactive control instructions corresponding to the interactive control rules as terminal control instructions corresponding to the target smart terminal, and then search for the corresponding application programming interface.

[0082] The interactive control rules here can be regarded as scene cards for the interaction between vehicles and smart terminals. Vehicle-side application developers, vehicle-mounted terminal users or vehicle-mounted terminals can manually or automatically edit through the vehicle-mounted terminal scene engine to form a scene sequence that gives users high perception. The "high perception" here can be a scene sequence generated according to the status of the vehicle and smart terminal and in line with user habits. For example, the interactive control rules can be pre-designed by vehicle-side application developers and stored in the vehicle / cloud, or created by vehicle-mounted terminal users through the graphical operation interface of the vehicle application platform through voice input, keyboard clicks, touch clicks, etc., or intelligently generated by the vehicle terminal based on the status of the vehicle, the status of the smart terminal, and the user's habits.

[0083] For example, the vehicle terminal can automatically call the vehicle-side scene engine to form a new scene card according to the state of the vehicle, the state of the smart terminal, the user's habits, etc. For example, during long-distance driving, the congestion ahead can be checked by a drone, or a video of the city of interest can be completed. At the same time, according to the congestion, road conditions, distance, average speed, time, etc. checked by the drone, the time when the user of the vehicle terminal arrives home can be intelligently predicted, and the home smart devices can be adjusted to a state that conforms to the user's habits before the predicted time. For example, if it is predicted that you can enter the house at 5 pm, then turn on the air conditioner and water heater at 4:45, and automatically adjust the air conditioner temperature (e.g., 26 degrees Celsius), water heater temperature (e.g., 38 degrees Celsius), TV programs or music of interest according to the time of arrival, turn on the air purifier to intelligently adjust the indoor air, etc. This method of automatically calling the scene card (i.e., interactive control rules) formed by the vehicle-side scene engine according to the state of the vehicle, the state of the smart terminal, the user's habits, etc. can be generated by a pre-trained neural network model or a large language model, etc. This application does not impose any restrictions on the model type of the neural network model or the large language model, etc., and the method of generating interactive control rules. For example, when training a large language model, a vehicle-mounted terminal can be created based on the user's voice instructions to automatically call the vehicle-side scene engine to form a new scene card based on the vehicle status, the status of the smart terminal, the user's habits, etc. The user's voice data can be obtained first (the voice data may include the scene described by the user, etc.), and a semantic recognition result corresponding to the voice data can be generated. The semantic recognition result is then determined to correspond to the instruction to create a vehicle-based interactive control rule, and the interactive control rule is generated based on the semantic recognition result. The interactive control rule may include interactive trigger conditions and interactive control instructions. Finally, the interactive control rule is provided to the vehicle's rule execution component so that the rule execution engine can notify the vehicle or smart terminal to execute the corresponding control instruction sequence after detecting that the corresponding interactive trigger condition is met.

[0084] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides an interactive device between a vehicle and a smart terminal. Figure 4 The figure shows a structural block diagram of a vehicle and smart terminal interaction device 400 according to an embodiment of the present application. The device 400 may include:

[0085] The terminal control instruction generating module 401 is used to generate a terminal control instruction corresponding to the target intelligent terminal on the vehicle side when the interaction triggering condition is met;

[0086] an application programming interface determination module 402, configured to search an application programming interface set for a target application programming interface corresponding to the terminal control instruction of the target intelligent terminal, wherein the application programming interface set includes application programming interfaces collected from at least one intelligent terminal; and

[0087] The application programming interface calling module 403 is used to call the target application programming interface to obtain the execution result corresponding to the terminal control instruction.

[0088] In one possible implementation, the terminal control instruction generation module 401 may include a user input submodule for acquiring control operations submitted through a control entry corresponding to the target smart terminal in the vehicle interface, and generating terminal control instructions corresponding to the target smart terminal on the vehicle side.

[0089] In a possible implementation, the terminal control instruction generation module 401 may include an interactive control submodule for generating a terminal control instruction corresponding to the target smart terminal on the vehicle side when the vehicle sensor detects that the operating state of the vehicle meets the trigger condition.

[0090] In some embodiments, the application programming interface set includes at least one first-class subset corresponding to different terminal attributes, and the first-class subset includes a first-class application programming interface for executing the atomic function of the smart terminal. The application programming interface determination module 402 may include: an interface set determination submodule, used to determine the first-class subset including the target application programming interface according to the terminal attribute of the target smart terminal; a function interface determination submodule, used to search for a target application programming interface corresponding to the terminal control instruction of the target smart terminal from the first-class subset.

[0091] In some embodiments, the application programming interface set further includes a second subset, and the second subset includes a second type of application programming interface for arranging the first type of application programming interface. When there are multiple corresponding target application programming interfaces found from the first type of subset, the application programming interface determination module 402 may also include: a rule interface determination submodule, which is used to determine other target application programming interfaces for arranging the target application programming interface found from the first type of subset from the second subset.

[0092] In one possible implementation, if the interaction trigger condition is consistent with the interaction trigger condition of the interaction control rule pre-set for the vehicle, the terminal control instruction generation module 401 may include: a rule execution module, used to call a rule execution engine to determine the interaction control instruction corresponding to the interaction control rule as the terminal control instruction corresponding to the target smart terminal.

[0093] In some embodiments, the smart terminal includes a vehicle's on-board equipment, smart home equipment, smart facilities or smart mobile devices; the smart terminal communicates with the vehicle through one or more communication methods including Ethernet, 4 / 5G network, narrowband Internet of Things, microwave radio, Bluetooth and WI-FI.

[0094] In a possible implementation, the rule execution module also includes a rule generation submodule, and the interactive control rules are generated by the rule generation submodule. The rule generation submodule may include: a voice data acquisition unit, used to acquire the user's voice data and generate a semantic recognition result corresponding to the voice data; a recognition result determination unit, used to determine that the semantic recognition result corresponds to an instruction to create a vehicle-based interactive control rule; a control rule generation unit, used to generate an interactive control rule according to the semantic recognition result, and the interactive control rule includes an interactive trigger condition and an interactive control instruction; a control instruction execution unit, used to provide the interactive control rule to the rule execution component of the vehicle, so that the rule execution engine can notify the vehicle or smart device to execute a corresponding control instruction sequence after detecting that the corresponding interactive trigger condition is met.

[0095] In a possible implementation, before searching for the target application programming interface corresponding to the terminal control instruction of the target smart terminal from the application programming interface set, the device may also include: an atomic function disassembly module for disassembling the atomic function of the target smart terminal; and an atomic function encapsulation module for encapsulating the disassembled atomic function of the target smart terminal into an application programming interface for executing the atomic function.

[0096] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0097] Figure 5 FIG. 1 is a block diagram of an electronic device used to implement an embodiment of the present application. Figure 5 As shown, the electronic device includes: a memory 501 and a processor 502. The memory 501 stores a computer program that can be run on the processor 502. When the processor 502 executes the computer program, the method in the above embodiment is implemented. The number of the memory 501 and the processor 502 can be one or more.

[0098] The electronic device also includes:

[0099] The communication interface 503 is used to communicate with external devices and perform data exchange transmission.

[0100] If the memory 501, the processor 502 and the communication interface 503 are implemented independently, the memory 501, the processor 502 and the communication interface 503 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 5 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0101] Optionally, in a specific implementation, if the memory 501, the processor 502 and the communication interface 503 are integrated on a chip, the memory 501, the processor 502 and the communication interface 503 can communicate with each other through an internal interface.

[0102] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by a processor.

[0103] An embodiment of the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method provided in any embodiment of the present application is implemented.

[0104] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0105] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the present application.

[0106] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0107] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting description, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).

[0108] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0109] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0110] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0111] Any process or method described in the flow chart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0112] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or used in combination with these instruction execution systems, devices or apparatuses.

[0113] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0114] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0115] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope recorded in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for interaction between a vehicle and an intelligent terminal, comprising: When the interaction triggering condition is met, a terminal control instruction corresponding to the target intelligent terminal is generated on the vehicle side; Searching for a target application programming interface corresponding to the terminal control instruction of the target intelligent terminal from an application programming interface set, wherein the application programming interface set includes application programming interfaces collected from at least one intelligent terminal; The target application programming interface is called to obtain an execution result corresponding to the terminal control instruction.

2. The method according to claim 1, wherein: When the interaction triggering condition is met, generating a terminal control instruction corresponding to the target intelligent terminal on the vehicle side includes: The control operation submitted through the control entry corresponding to the target intelligent terminal in the vehicle interface is obtained, and a terminal control instruction corresponding to the target intelligent terminal is generated on the vehicle side.

3. The method according to claim 1, wherein: When the interaction triggering condition is met, generating a terminal control instruction corresponding to the target intelligent terminal on the vehicle side includes: When the vehicle sensor detects that the running state of the vehicle meets the trigger condition, a terminal control instruction corresponding to the target intelligent terminal is generated on the vehicle side.

4. The method according to claim 1, wherein: The application programming interface set includes at least one first-category subset corresponding to different terminal attributes, and the first-category subset includes a first-category application programming interface for executing an atomic function of a smart terminal; The step of searching the target application programming interface corresponding to the terminal control instruction of the target intelligent terminal from the application programming interface set comprises: Determine a first subset including the target application programming interface according to the terminal attribute of the target intelligent terminal; A target application programming interface corresponding to the terminal control instruction of the target intelligent terminal is searched from the first subset.

5. The method according to claim 4, wherein: The application programming interface set further includes a second type of subset, wherein the second type of subset includes a second type of application programming interface for arranging the first type of application programming interface; When there are multiple corresponding target application programming interfaces found from the first subset, searching the target application programming interface corresponding to the terminal control instruction of the target intelligent terminal from the application programming interface set further includes: Other target application programming interfaces for arranging the target application programming interfaces found from the first subset are determined from the second subset.

6. The method according to claim 3, wherein: If the interaction triggering condition is consistent with the interaction triggering condition of the interaction control rule preset for the vehicle, the generating of the terminal control instruction corresponding to the target intelligent terminal on the vehicle side includes: A rule execution engine is called to determine the interactive control instruction corresponding to the interactive control rule as a terminal control instruction corresponding to the target intelligent terminal.

7. The method according to claim 1, wherein: The smart terminal includes the vehicle's on-board equipment, smart home equipment, smart facilities or smart mobile devices; the smart terminal communicates with the vehicle through one or more communication methods including Ethernet, 4 / 5G network, narrowband Internet of Things, microwave radio, Bluetooth and WI-FI.

8. The method according to claim 6, wherein: The interaction control rules are generated by the following steps: Acquire the user's voice data and generate a semantic recognition result corresponding to the voice data; Determining that the semantic recognition result corresponds to an instruction to create a vehicle-based interactive control rule; Generate an interaction control rule according to the semantic recognition result, wherein the interaction control rule includes an interaction trigger condition and an interaction control instruction; The interactive control rules are provided to the rule execution component of the vehicle so that the rule execution engine can notify the vehicle or smart terminal to execute the corresponding control instruction sequence after detecting that the corresponding interactive triggering condition is met.

9. The method according to claim 1, wherein: Before searching the target application programming interface corresponding to the terminal control instruction of the target intelligent terminal from the application programming interface set, the method further includes: Disassemble the atomic functions of the target intelligent terminal; The atomic functions of the disassembled target intelligent terminal are encapsulated into an application programming interface for executing the atomic functions.

10. An interactive device between a vehicle and an intelligent terminal, comprising: A terminal control instruction generation module, used to generate a terminal control instruction corresponding to a target intelligent terminal on the vehicle side when an interaction triggering condition is met; An application programming interface determination module, used to search for a target application programming interface corresponding to the terminal control instruction of the target intelligent terminal from an application programming interface set, wherein the application programming interface set includes application programming interfaces collected from at least one intelligent terminal; The application programming interface calling module is used to call the target application programming interface to obtain the execution result corresponding to the terminal control instruction.

11. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the processor implements the method according to any one of claims 1 to 9 when executing the computer program.

12. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1 to 9 is implemented.

Images