Vehicle-based linkage control deployment method and device, electronic equipment and storage medium

Abstract

The invention provides a linkage control deployment method and device for a vehicle, electronic equipment and a storage medium, and the method comprises the steps: obtaining voice data of a user, generating a semantic recognition result corresponding to the voice data, and after determining that the semantic recognition result correspondingly indicates the creation of a linkage control rule based on a vehicle state, carrying out the linkage control deployment of the vehicle. A linkage control rule is generated according to the semantic recognition result, the linkage control rule comprises a linkage triggering condition and a control instruction sequence, and then the linkage control rule is provided for a rule execution assembly of the vehicle. And the rule execution assembly notifies the vehicle or the intelligent equipment to execute the corresponding control instruction sequence after monitoring that the vehicle state meets the corresponding linkage triggering condition. According to the scheme, the information related to the operation functions of the vehicle and the intelligent equipment can be accurately extracted and the corresponding linkage control rule can be generated based on the voice instruction of the user, so that the intelligent and personalized control of the vehicle and the intelligent terminal can be realized.

Description

Technical Field

[0001] The present application relates to the field of automobile technology, and in particular to a method, device, electronic device and storage medium for deploying linkage control of a vehicle. Background Art

[0002] As the level of automobile electrification, networking and intelligence continues to improve, scene cards, which are control rules that include trigger conditions and execution actions, can be used to perform personalized control of vehicles and various smart terminals connected to the vehicle through the network. However, currently, creating vehicle control scene cards often requires manual arrangement of instructions, which is cumbersome and difficult to popularize. Summary of the invention

[0003] The embodiments of the present application provide a vehicle-based linkage control deployment method and device to solve the above-mentioned problems.

[0004] In a first aspect, the present application provides a vehicle-based linkage control deployment method, comprising:

[0005] Acquire the user's voice data and generate a semantic recognition result corresponding to the voice data;

[0006] Determining that the semantic recognition result corresponds to an instruction to create a linkage control rule based on the vehicle state;

[0007] Generate a linkage control rule according to the semantic recognition result, wherein the linkage control rule includes a linkage trigger condition and a control instruction sequence;

[0008] The linkage control rule is provided to the rule execution component of the vehicle, so that the rule execution component notifies the vehicle or smart device to execute the corresponding control instruction sequence after monitoring that the vehicle state meets the corresponding linkage trigger condition.

[0009] In a second aspect, an embodiment of the present application provides a vehicle-based linkage control deployment device, including:

[0010] A speech recognition module, used to obtain speech data of a user, generate a semantic recognition result corresponding to the speech data, and determine that the semantic recognition result corresponds to an instruction to create a linkage control rule based on a vehicle state;

[0011] A rule generation module, used to generate a linkage control rule according to the semantic recognition result, wherein the linkage control rule includes a linkage trigger condition and a control instruction sequence;

[0012] The rule execution module is used to notify the vehicle or smart device to execute the corresponding control instruction sequence after monitoring that the vehicle status meets the corresponding linkage trigger condition.

[0013] In a third aspect, an embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the above methods when executing the computer program.

[0014] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method described in any one of the above is implemented.

[0015] Compared with the related art, this application has the following advantages:

[0016] According to the embodiment of the present application, the voice command input by the user is first semantically recognized, so as to accurately extract the linkage trigger conditions and control instruction sequences corresponding to the atomic functions of the vehicle and the smart device, and the linkage trigger conditions and control instruction sequences are compiled using the large language model to generate corresponding linkage control rules. When the user's preset operating conditions are detected to be met, the vehicle rule execution component calls the linkage control rules corresponding to the operating conditions, and sends the control instruction sequence corresponding to the linkage control rules to the vehicle or smart device or its controlled components to notify them to perform corresponding actions. Through the above scheme, the functions or capabilities of the vehicle or smart device are decoupled to obtain multiple minimum action units, that is, atomic functions, and the semantic recognition results of the user's voice commands are matched with the atomic functions, which can accurately determine the linkage trigger conditions and control instruction sequences corresponding to the user's voice commands, and realize the free combination of the atomic functions of the vehicle and the smart terminal; at the same time, the linkage trigger conditions and control instruction sequences are compiled using the prominent text understanding and text generation capabilities of the large language model, which can accurately and efficiently generate scene cards corresponding to the user's voice commands and can be executed by the vehicle rule execution component, thereby realizing personalized and diversified control of vehicles and smart devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. These drawings only depict some implementation methods according to the present application and should not be regarded as limiting the scope of the present application.

[0018] Figure 1 A flow chart of a vehicle-based linkage control deployment method provided in an embodiment of the present application;

[0019] Figure 2 A flowchart of a method for generating linkage control rules provided in an embodiment of the present application;

[0020] Figure 3 A schematic diagram of a vehicle-based linkage control deployment method provided in an embodiment of the present application;

[0021] Figure 4 A schematic diagram of generating linkage control rules through a large language model provided in an embodiment of the present application;

[0022] Figure 5 A structural block diagram of a vehicle-based linkage control deployment device provided in an embodiment of the present application, and

[0023] Figure 6 A block diagram of an electronic device used to implement an embodiment of the present application is shown. DETAILED DESCRIPTION

[0024] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0025] Some embodiments of the present application are described in detail below in conjunction with the accompanying drawings. In the case where the embodiments do not conflict with each other, the following embodiments and the features in the embodiments can be combined with each other. The steps of the method described in the embodiment can be interchanged as long as they can comply with the vehicle linkage control deployment method implemented by the method provided in the embodiment of the present application.

[0026] The embodiment of the present application provides a vehicle-based linkage control deployment method for generating linkage control rules. A complete linkage control rule includes a linkage trigger condition and a control instruction sequence, and the control instruction sequence is used to indicate the actions to be performed by the vehicle controlled component or intelligent terminal and the order or process information of performing the actions. When a linkage trigger condition is met, the vehicle or intelligent terminal will perform a series of actions corresponding to the condition in a preset order or process according to the linkage control rule corresponding to the condition.

[0027] In an embodiment of the present application, a linkage trigger condition of a linkage control rule is a change in the operating state of the hardware or software or the environmental state monitored by the central control system of the vehicle. For example, the linkage trigger condition may be a sensor or other detection device configured on the vehicle side detecting a change in a state parameter or reaching a preset threshold, and the state parameter may be an operating state parameter of a component in each part of the vehicle (engine, chassis, body and electrical system), such as engine speed and battery power, or detected environmental parameters inside and outside the vehicle, such as temperature, distance to external objects, etc. For example, if the linkage trigger condition of linkage control rule A is set to a battery power as low as 20%, then when the battery management system detects that the battery power is lower than 20%, it will trigger the vehicle and other related intelligent terminals to execute a series of actions corresponding to the control instruction sequence described in the linkage control rule.

[0028] Specifically, in some possible embodiments, the linkage trigger condition may also be a timing condition. For example, linkage control rule B is set to execute actions related to charging the battery car when the system time is 10 pm, then the system time reaching 10 pm is a linkage trigger condition.

[0029] In some possible embodiments, the linkage trigger condition may also be a data signal sent by other intelligent terminals received by the central control system of the vehicle. The intelligent terminal in this application is a general terminal with certain computing and communication capabilities, such as smart home devices (such as smart TVs, air conditioners, access control systems and other smart home appliances or smart home device systems), road infrastructure (such as parking meters and charging piles), unmanned remote control terminals (such as drones), etc. Various intelligent terminals can send data to the cloud server through corresponding communication protocols and communication methods, and the cloud server will send the data to the vehicle side. The vehicle side determines that the condition is triggered based on the received data, and notifies the corresponding controlled components and related intelligent terminals in the vehicle to start executing the actions corresponding to the control instruction sequence described in the linkage control rules.

[0030] The control instruction sequence described in the linkage control rules in this application includes one or more control instructions, which are used to instruct or notify the controlled components of the vehicle or the intelligent terminal to complete corresponding actions. These actions are the atomic functions or atomic capabilities of the terminal. The atomic function is the functional module of the smallest unit of the terminal device. For example, for the air conditioner, turning it on and off, adjusting the temperature, and adjusting the wind speed are all atomic functions. "Turn on the air conditioner to 24 degrees Celsius" includes at least two atomic functions: turning on the air conditioner and adjusting the temperature. In order to achieve refined control of terminals of different types and produced by different manufacturers, the atomic functions of the terminal can be encapsulated into a unified state and operation interface according to the relevant setting documents of different equipment manufacturers for system calls, thereby controlling the terminal to perform the required operations.

[0031] The execution subject of the embodiment of the present application may be an application, service, instance, software-based functional module, virtual machine (VM), container or cloud server, etc., or a hardware device with data processing function (such as a server or terminal device) or hardware chip (such as CPU, GPU, FPGA, NPU, AI acceleration card or DPU), etc. The device for realizing vehicle information collection can be deployed on the computing device of the application party providing the corresponding service or on the cloud computing platform providing computing power, storage and network resources.

[0032] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0033] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The several specific embodiments listed can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0034] The present application embodiment provides a vehicle-based linkage control deployment method, such as Figure 1 The flowchart shown is a vehicle-based linkage control deployment method 100 according to an embodiment of the present application. The method may include: step S101, obtaining user voice data and generating a semantic recognition result corresponding to the voice data; step S102, determining that the semantic recognition result corresponds to an instruction to create a linkage control rule based on the vehicle status; step S103, generating a linkage control rule according to the semantic recognition result; and step S104, providing the linkage control rule to the rule execution component of the vehicle, so that the rule execution component notifies the vehicle or smart device to execute the corresponding control instruction sequence after monitoring that the vehicle status meets the corresponding linkage trigger condition.

[0035] The following is a detailed description of each step.

[0036] In step S101, voice data of a user is acquired, and a semantic recognition result corresponding to the voice data is generated.

[0037] The voice data is the voice control instructions input by the user into the user terminal (such as a smart phone, smart tablet, etc.) or the central controller of the vehicle through the voice interaction function. The user terminal or the vehicle central control system uploads the input voice control instructions to the cloud server, and the cloud server inputs the received voice data into the neural network model to obtain the semantic recognition results corresponding to the voice data.

[0038] In some embodiments, before obtaining the user voice data, step S101 may also include a wake-up process for the voice interaction function. The wake-up process may be a sound wake-up, such as the user speaking a preset wake-up keyword to wake up the device function; or a traditional key or button wake-up. After the device responds to the wake-up instruction and enters a state ready to receive a voice control instruction, the user issues a voice control instruction to start creating a linkage control rule.

[0039] In some embodiments, the semantic recognition result includes the domain and intent of the speech data. The neural network model is pre-trained to process NLU (Natural Language Understanding) tasks. NLU is a general term for all methods, models or tasks that support machine understanding of text content. NLU plays a very important role in text information processing systems, and has important applications in automatic reasoning, machine translation, question and answer, news collection, text classification, voice activation, archiving and large-scale content analysis. Intent recognition is an important task branch in the field of NLU. Most applications related to intelligent conversations based on semantic understanding use this technology to classify the intent expressed by a sentence. Among them, a commonly used classification model is to pre-divide multiple "domains" and "intents", the domains and intents are orthogonal to each other, and the text is classified according to the two dimensions of domain and intent. For example, the instruction “Please set an alarm for 8 a.m. tomorrow” can be classified into the “Reminder” domain and further classified into the intent category of “Create Alarm”; while “Cancel all alarms tomorrow” still belongs to the “Reminder” domain, but the intent is completely opposite, which is “Cancel Alarm”.

[0040] Before inputting the voice data into the neural network model, step S101 also includes: converting the voice data into corresponding text data.

[0041] In this application, the conversion of speech data to text data can be achieved through automatic speech recognition technology. The goal of automatic speech recognition (ASR) is to convert the vocabulary content in human speech into computer-readable input, such as keystrokes, binary codes or character sequences. It should be noted that automatic speech recognition only converts speech data into corresponding text and does not involve changes in meaning.

[0042] The above is step S101, and then proceed to step S102.

[0043] In step S102, it is determined that the semantic recognition result corresponds to an instruction to create a linkage control rule based on the vehicle state.

[0044] In an embodiment of the present application, the neural network model outputs corresponding semantic recognition results for the user's voice data, and the semantic recognition results include the domain and intent classification of the voice data.

[0045] In a possible embodiment, when the output domain is "vehicle control" and the intention is "create linkage control rules", it is determined that the semantic recognition result corresponds to an instruction to create linkage control rules based on the vehicle status. For example, when the user issues a voice command that is only "please close the windows and play music", its semantic recognition result belongs to the vehicle control domain, but its intention is not to create linkage control rules, but only to execute the given instructions; when the user's voice command contains a sentence like "please help me design a new scene card (that is, the functional name of the linkage control rule on the user side)", the corresponding domain is vehicle control, and the intention is to create linkage control rules. At this time, it is determined that the semantic recognition result corresponds to an instruction to create linkage control rules between vehicles and smart devices, and the subsequent process of creating linkage control rules can continue.

[0046] The above is step S102, and then proceed to step S103.

[0047] In step S103, a linkage control rule is generated according to the semantic recognition result.

[0048] When the user's purpose is to create a new linkage control rule, the voice data received by the system not only includes "create a new linkage control rule" or similar sentences, but also includes the content of the linkage control rule designed by the specific user. In terms of the actual operation process, the specific voice data acquisition process can be that after the system determines that the semantic recognition result of the voice command issued by the user corresponds to the creation of a linkage control rule, it prompts the user to continue to input the content of the linkage control rule, and obtains the user's complete design of the linkage control rule through multiple rounds of dialogue; it can also obtain the user's complete voice command at one time. When the system determines that the semantic recognition result of the complete voice command includes the creation of a linkage control rule between the vehicle and the smart device, it further generates the linkage control rule based on the remaining part of the complete voice command, that is, the natural language description of the linkage control rule designed by the user.

[0049] Next, combine Figure 2 Step S103 will be described in detail. Figure 2 2 is a flow chart of a method 200 for generating linkage control rules provided in an embodiment of the present application. Figure 2 As shown, step S103 may specifically include the following steps: step S301, determining the atomic rule information matching the semantic recognition result from the rule database, wherein the rule database includes the atomic linkage trigger conditions and control instructions split from the standard linkage control rules, and step S302, creating new linkage control rules according to the atomic linkage trigger conditions and control instructions matching the semantic recognition result.

[0050] In step S301, atomized rule information matching the semantic recognition result is determined from a rule database, wherein the rule database includes atomized linkage trigger conditions and control instructions separated from standard linkage control rules.

[0051] As described above, the semantic recognition result in the embodiment of the present application includes a recognition result of the user's intention to create linkage control rules, and further includes text data corresponding to the linkage control rules described by the user in natural language. Hereinafter, unless there is any ambiguity, "semantic recognition result" will be used to refer to the text data corresponding to the linkage control rules described by the user in natural language.

[0052] In actual usage scenarios, since the language descriptions used by users are usually colloquial, and users often lack professional knowledge of vehicle controlled components or smart terminal functional modules, it is necessary to first compare and match the linkage control rule text data described by the user in natural language with the atomic rule information in the pre-established rule database, wherein the rule database includes atomic linkage trigger conditions and control instructions separated from standard linkage control rules. The atomic control instructions are control instructions that correspond to the encapsulated atomic capabilities of the vehicle (or its controlled components) and the smart terminal described above and cannot be further split or simplified, and the atomic linkage trigger conditions are also single trigger conditions that cannot be further split. In other words, the rule database can be understood as a corpus of atomic functional information of vehicles and smart terminals.

[0053] In some embodiments, before determining the atomic rule information that matches the semantic recognition result from the rule database, the method 100 also includes: obtaining multiple standard linkage control rules, splitting the atomic linkage trigger conditions and control instructions from the standard linkage control rules, and constructing the rule database based on the split atomic linkage trigger conditions and control instructions.

[0054] The standard linkage control rules in the embodiments of the present application are standardized descriptions of the functions that are designed in advance by professionals and determined based on the official configuration files or user manuals of the vehicles or smart terminals provided by the manufacturers. The terms are relatively standardized and unified, and can encompass all atomic functions supported by the product. Therefore, they can be used as a sample source for the standard rule library to match the description language of the linkage control rules provided by the user.

[0055] The rule database does not store atomic linkage trigger conditions and control instructions in natural language, but their corresponding vector representations, i.e., corresponding text vectors. Therefore, it is necessary to use a neural network model to extract and segment the standard linkage control rules in natural language through vector features and encode them into vectors, which are then stored in the rule database.

[0056] In some embodiments, determining the atomic rule information that matches the semantic recognition result from the rule database also includes: obtaining a text vector corresponding to the semantic recognition result, and determining the atomic rule information that matches the semantic recognition result based on the vector similarity corresponding to the text vector corresponding to the semantic recognition result and the text vectors of the atomized linkage trigger conditions and control instructions in the rule database.

[0057] Similar to the data in the rule sample library, the semantic recognition result also needs to be processed by the neural network model and converted into a vector form. It should be noted that the neural network model used to process the semantic recognition result in the embodiment of the present application is exactly the same as the neural network model used to process the standard linkage control rule. Calculate the vector similarity corresponding to the text vector corresponding to the semantic recognition result and the text vectors of the atomized linkage trigger conditions and control instructions in the rule database, and determine the atomized rule information matching the semantic recognition result according to the preset judgment criteria. For example, the cosine similarity between the text vector corresponding to the semantic recognition result and the text vector of the atomized linkage trigger conditions and control instructions in the rule database can be calculated respectively, and the atomized linkage trigger conditions and control instructions corresponding to the text vector in the standard rule library with the highest cosine similarity with the text vector corresponding to the semantic recognition result are determined as the atomized rule information matching the semantic recognition result. The atomized rule information is essentially consistent with the linkage control rule content designed by the user, and includes multiple vectors, each of which represents the atomized linkage trigger conditions or control instructions described above.

[0058] The above is step S301, and then proceed to step S302.

[0059] In step S302, a new linkage control rule is created according to the atomized rule information matching the semantic recognition result.

[0060] Although the above-mentioned atomized rule information matched with the semantic recognition result includes linkage trigger conditions and atomized control instructions, it is only an unordered collection of this information and needs to undergo specific transformation and processing to create new linkage control rules. In an embodiment of the present application, by inputting the atomized rule information into a large language model, the powerful analysis and generation performance of the large language model is utilized to calculate and output a new linkage control rule.

[0061] In an embodiment of the present application, creating a new linkage control rule based on the atomic rule information matching the semantic recognition result includes: inputting the atomic rule information into a large language model, the large language model calling an atomic instruction database to generate the linkage control rule, and the atomic instruction database collects atomic-level control instructions executable by vehicles and various terminal devices.

[0062] Large language models (LLM) usually refer to language models that are trained on large-scale text corpora and contain tens of billions of parameters (or more), such as GPT-3, PaLM, LLaMA, etc. The current large language models use Transformer architectures and pre-training objectives (such as Language Modeling) similar to small models. The main difference from small models is the increase in model size, training data, and computing resources. The performance of large language models often follows the law of expansion, but for some capabilities, they will only appear when the scale of the language model reaches a certain level. These capabilities are called "emergent capabilities". Representative emergent capabilities include contextual learning, instruction following, and step-by-step reasoning. Although large language models have no essential difference from traditional small models in terms of model architecture and training methods, they have achieved breakthrough improvements in natural language processing tasks (including natural language understanding and natural language generation) based on far more training data and parameter numbers than traditional models.

[0063] The embodiment of the present application utilizes the ability of the large prediction model to understand and generate text to process the atomized rule information and obtain new linkage control rules. Compared with the traditional deep learning model, the large language model can receive the semantic recognition results (including the user's intention information to create linkage control rules, the linkage trigger conditions of the linkage control rules, and control instructions) at one time and analyze them as a whole to achieve end-to-end calculation, without the need to receive user instructions step by step like the traditional model, and perform the analysis or calculation of the next step based on the results output by each step.

[0064] It should be noted that the large language model can also be used to perform the above-mentioned text processing of the neural network model of the semantic recognition results and the standard linkage control rules, that is, the process of encoding natural language into vectors for the two. Using the large language model to vector model the standard linkage control rules and establish a database can enable the large language model to "learn" the knowledge in this field. The process of the large language model receiving the semantic recognition results and searching and matching similar results in the standard linkage rule database is the process of the large language model using the learned knowledge to deal with new problems.

[0065] Figure 3 A schematic diagram of a vehicle-based linkage control deployment method provided in an embodiment of the present application. Figure 3As shown, the voice data input by the user through the mobile phone or the vehicle terminal is first converted into corresponding text data through voice recognition processing, and then processed through semantic recognition to obtain a semantic recognition result, which contains the instruction issued by the user to create a linkage control rule. Next, the large language model receives the semantic recognition result for processing and outputs the linkage control rule, which is sent to the rule execution component. The rule execution component sends a control instruction to the smart terminal or vehicle according to the control instruction sequence in the linkage control rule to notify it to perform the corresponding action.

[0066] Although the performance of the large language model is good, the quality or accuracy of its output results still depends on the accuracy and clarity of the commands input by the user. In order to interact with the model efficiently, a prompt can be entered in the model. A prompt is a text fragment used to guide a large language model to generate natural language text, which is used to guide the model to produce text results that meet our expectations. Usually, high-quality prompts are clear and specific, and provide sufficient context or background information. For example, "How to get more people to my birthday party" and "I'm preparing for a birthday party and need to invite about 20 people to participate. What suggestions do you have to help me increase the attendance rate?" Compared with the latter, it is a higher quality prompt, and the output results obtained are usually more accurate and detailed. Therefore, in an embodiment of the present application, the atomized rule information can be filled into a pre-set rule template, that is, a prompt template, so as to obtain atomized rule information suitable for the input specification of the large language model, and the atomized rule information is input into the large language model to obtain a new linkage control rule.

[0067] In some embodiments, the linkage control rules output by the above-mentioned large language model are script files in JSON format, and creating new linkage control rules based on the atomic rule information matching the semantic recognition results also includes: inputting the linkage control rules in JSON format into a script parser for script parsing to obtain the parsed linkage control rules.

[0068] JSON file is a file of simple data structure and object stored in JavaScript Object Notation (JSON) format, which is a standard data exchange format. It is mainly used to transfer data between Web applications and servers. JSON file is a lightweight, text-based, human-readable file that can be edited using a text editor. In the embodiment of the present application, the script file in JSON format output by the large language model based on the input atomized rule information records the triggering conditions, start nodes, end nodes, and control instruction sequences (including the contents of each instruction and the sequential relationship between them) of the linkage control rules. By pre-defining the output format of the large language model, it outputs a JSON format script file, which is easy for developers to understand and process, and is easy to convert into a file format that matches the data protocol of the rule execution component responsible for executing the linkage control rules.

[0069] Figure 4 A schematic diagram of generating linkage control rules through a large language model provided in an embodiment of the present application. Figure 4 As shown, the large language model receives atomic rule information and outputs a JSON format script file, and then uses a script parser to convert the JSON format script file to obtain the parsed linkage control rules and send them to the rule execution component.

[0070] In some embodiments, the above-mentioned JSON script file also needs to be input into the script parser, and the parsed linkage control rules are output. JSON format script files are suitable for data transmission between servers, systems and devices, but they are not executable files themselves. In a possible embodiment, the parsed linkage control rules can be, for example, XML format files, which match the data protocol of the rule execution component of the vehicle, so that the rule execution component can notify the vehicle or smart device to execute the corresponding control instruction sequence after monitoring that the vehicle state meets the corresponding linkage trigger condition.

[0071] The above is step S103, and then proceed to step S104.

[0072] In step S104, the linkage control rule is provided to the rule execution component of the vehicle, so that the rule execution component notifies the vehicle or smart device to execute the corresponding control instruction sequence after monitoring that the vehicle state meets the corresponding linkage trigger condition.

[0073] The cloud server can send the generated linkage control rules to the vehicle-side scene engine for storage in the vehicle control system and calling of the rule execution component. After detecting that the linkage trigger condition is met, the rule execution component will send instructions to the vehicle controlled components or smart terminals described in the linkage control rules to enable them to complete the corresponding actions.

[0074] In some embodiments, when the car detects that the linkage trigger condition is met, it can also arbitrate the linkage control rule to determine whether the current vehicle or smart device state can execute the instruction sequence in the linkage control rule. For example, the linkage control rule set by the user is to open the car's rear cover at 10 o'clock every night to charge the battery. However, at 10 o'clock in the evening, the car is in driving state. At this time, the vehicle-side scene engine determines through arbitration that the linkage control rule cannot be executed at present. If there is no conflict between the current vehicle or smart device state and the instructions in the linkage control rule, the vehicle-side scene engine will send a command signal to the corresponding vehicle controlled component or smart device according to the linkage control rule to notify the vehicle controlled component or smart device to execute the corresponding control instruction sequence.

[0075] In some embodiments, the newly created linkage control rules can also be stored in the cloud so that the vehicle-side engine can call them when connected to the Internet. In addition, the newly created linkage control rules can be further processed and stored in the aforementioned standard rule database to enrich the corpus data that can be used by the large language model.

[0076] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides a vehicle-based linkage control deployment device. Figure 5 The figure shows a structural block diagram of a vehicle-based linkage control deployment device 500 according to an embodiment of the present application. The device 500 may include:

[0077] The speech recognition module 501 is configured to obtain speech data of the user, generate a semantic recognition result corresponding to the speech data, and determine that the semantic recognition result corresponds to an instruction to create a linkage control rule based on the vehicle state;

[0078] A rule generation module 502 is configured to generate a linkage control rule according to the semantic recognition result, wherein the linkage control rule includes a linkage trigger condition and a control instruction sequence;

[0079] The rule execution module 503 is configured to notify the vehicle or smart device to execute a corresponding control instruction sequence after monitoring that the vehicle state meets the corresponding linkage trigger condition.

[0080] In some embodiments, the speech recognition module 501 may include a speech acquisition submodule for acquiring speech data uploaded by a user terminal or the vehicle.

[0081] In some embodiments, the speech recognition module 501 may further include:

[0082] A text recognition submodule, used to recognize text content corresponding to the voice data;

[0083] The intention recognition submodule is used to input the recognized text content into the neural network model, and obtain the domain classification result and the intention classification result output by the neural network model as the semantic recognition result corresponding to the speech data.

[0084] In some embodiments, the speech recognition module 501 may further include a semantic determination submodule for determining that the domain classification result and the intention classification result output by the neural network model correspond to instructions for creating linkage control rules between the vehicle and the smart device.

[0085] In some embodiments, the rule generation module 502 may include:

[0086] A rule information determination submodule, used to determine the atomic rule information matching the semantic recognition result from a rule database, wherein the rule database includes atomic linkage trigger conditions and control instructions separated from standard linkage control rules;

[0087] The linkage control rule creation submodule is used to create a new linkage control rule according to the atomized rule information matching the semantic recognition result.

[0088] In some embodiments, the rule generation module 502 may further include:

[0089] A rule input submodule, used to input the atomic rule information into a large language model, the large language model calls an atomic instruction database to generate the linkage control rule, the atomic instruction database collects atomic-level control instructions executable by vehicles and various terminal devices;

[0090] The output acquisition module is used to obtain the linkage control rules output by the large language model.

[0091] In some embodiments, the rule generation module 502 may further include a rule arrangement submodule, which is used to arrange the atomized rule information according to a rule template to generate atomized rule information suitable for a large language model input specification.

[0092] In some embodiments, the rule generation module 502 may further include:

[0093] A standard rule splitting submodule is used to obtain a plurality of standard linkage control rules and split the standard linkage control rules into atomic linkage trigger conditions and control instructions;

[0094] The database creation submodule is used to construct the rule database according to the split atomic linkage trigger conditions and control instructions.

[0095] In some embodiments, the rule generation module 502 may also include a rule matching submodule for obtaining a text vector corresponding to a semantic recognition result; and determining the atomic rule information that matches the semantic recognition result based on the vector similarity between the text vector corresponding to the semantic recognition result and the text vectors of the atomized linkage trigger conditions and control instructions in the rule database.

[0096] In some embodiments, the rule generation module 502 may further include a rule parsing submodule, which is used to input the linkage control rule in JSON format into a script parser for script parsing to obtain the parsed linkage control rule.

[0097] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0098] Figure 6 FIG. 1 is a block diagram of an electronic device used to implement an embodiment of the present application. Figure 6 As shown, the electronic device includes: a memory 601 and a processor 602. The memory 601 stores a computer program that can be run on the processor 602. When the processor 602 executes the computer program, the method in the above embodiment is implemented. The number of the memory 601 and the processor 602 can be one or more.

[0099] The electronic device also includes:

[0100] The communication interface 603 is used to communicate with external devices and perform data exchange transmission.

[0101] If the memory 601, the processor 602 and the communication interface 603 are implemented independently, the memory 601, the processor 602 and the communication interface 603 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0102] Optionally, in a specific implementation, if the memory 601, the processor 602 and the communication interface 603 are integrated on a chip, the memory 601, the processor 602 and the communication interface 603 can communicate with each other through an internal interface.

[0103] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by a processor.

[0104] An embodiment of the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method provided in any embodiment of the present application is implemented.

[0105] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0106] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the present application.

[0107] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0108] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting description, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DRRAM).

[0109] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0110] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0111] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0112] Any process or method described in the flow chart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0113] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or used in combination with these instruction execution systems, devices or apparatuses.

[0114] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0115] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0116] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope recorded in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A vehicle-based linkage control deployment method, comprising: Acquire the user's voice data and generate a semantic recognition result corresponding to the voice data; Determining that the semantic recognition result corresponds to an instruction to create a linkage control rule based on the vehicle state; Generate a linkage control rule according to the semantic recognition result, wherein the linkage control rule includes a linkage trigger condition and a control instruction sequence; The linkage control rule is provided to the rule execution component of the vehicle, so that the rule execution component notifies the vehicle or smart device to execute the corresponding control instruction sequence after monitoring that the vehicle state meets the corresponding linkage trigger condition.

2. The method according to claim 1, wherein: The obtaining of the user's voice data comprises: Acquire voice data uploaded by a user terminal or the vehicle.

3. The method according to claim 2, wherein: The generating of the semantic recognition result corresponding to the speech data comprises: Identifying text content corresponding to the voice data; The recognized text content is input into the neural network model, and the domain classification result and the intention classification result output by the neural network model are obtained as the semantic recognition result corresponding to the speech data.

4. The method according to claim 3, wherein: Determining that the semantic recognition result corresponds to an instruction to create a linkage control rule based on the vehicle state includes: Determine that the domain classification result and the intention classification result output by the neural network model correspond to instructions for creating linkage control rules between the vehicle and the smart device.

5. The method according to claim 4, wherein: Generating linkage control rules according to the semantic recognition results includes: Determining atomic rule information matching the semantic recognition result from a rule database, wherein the rule database includes atomic linkage trigger conditions and control instructions separated from standard linkage control rules; A new linkage control rule is created according to the atomized rule information matching the semantic recognition result.

6. The method according to claim 5, wherein: The creating a new linkage control rule according to the atomized rule information matching the semantic recognition result includes: The atomic rule information is input into a large language model, and the large language model calls an atomic instruction database to generate the linkage control rule, wherein the atomic instruction database collects atomic-level control instructions executable by vehicles and various terminal devices; Get the linkage control rules output by the large language model.

7. The method according to claim 6, wherein: Before inputting the atomized rule information into the large language model, creating a new linkage control rule according to the atomized rule information matching the semantic recognition result further includes: The atomized rule information is arranged and processed according to a rule template to generate atomized rule information suitable for a large language model input specification.

8. The method according to claim 5, wherein: The method further comprises: Acquire multiple standard linkage control rules, and separate atomic linkage trigger conditions and control instructions from the standard linkage control rules; The rule database is constructed based on the split atomic linkage trigger conditions and control instructions.

9. The method according to claim 5, wherein: The atomic linkage trigger conditions and control instructions in the rule database are represented as corresponding text vectors; The determining of the atomized rule information matching the semantic recognition result from the rule database includes: Get the text vector corresponding to the semantic recognition result; Atomized rule information matching the semantic recognition result is determined based on the vector similarities corresponding to the text vector corresponding to the semantic recognition result and the text vectors of the atomized linkage trigger conditions and control instructions in the rule database.

10. The method according to claim 6, wherein: The linkage control rule output by the large language model is a script file in JSON format, and the creation of a new linkage control rule according to the atomized rule information matching the semantic recognition result further includes: The linkage control rules in JSON format are input into the script parser for script parsing to obtain the parsed linkage control rules.

11. A vehicle-based linkage control deployment device, the device comprising: A speech recognition module, used to obtain the user's speech data and generate a semantic recognition result corresponding to the speech data, and Determining that the semantic recognition result corresponds to an instruction to create a linkage control rule based on the vehicle state; A rule generation module, used to generate a linkage control rule according to the semantic recognition result, wherein the linkage control rule includes a linkage trigger condition and a control instruction sequence; The rule execution module is used to notify the vehicle or smart device to execute the corresponding control instruction sequence after monitoring that the vehicle status meets the corresponding linkage trigger condition.

12. A computer-readable storage medium, wherein: The computer-readable storage medium includes a stored program, and when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the linkage control deployment method according to any one of claims 1 to 10.

13. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the computer program implements the method according to any one of claims 1 to 10 when executed by the processor.

Images