Vehicle-based linkage control rule creation method

Abstract

The invention provides a linkage control rule creation method and device based on a vehicle, electronic equipment and a storage medium, and the method comprises the steps: analyzing a standard linkage control rule in advance, so as to determine an operation condition and an operation action corresponding to a single operation from the standard linkage control rule; under the condition that the user voice instruction is monitored, performing semantic recognition on the user voice instruction; searching and recognizing the analysis results of the plurality of standard linkage control rules to obtain operation conditions and operation actions which have matching relations with the semantic recognition results; and combining the searched operation conditions and operation actions into a new linkage control rule. According to the scheme, the information related to the operation functions of the vehicle and the intelligent equipment can be accurately extracted based on the natural language description of the user on the linkage control rule, and the corresponding linkage control rule is generated, so that the automatic control of the vehicle and the intelligent terminal is realized.

Description

Technical Field

[0001] The present application relates to the field of automobile technology, and in particular to a method, device, electronic device and storage medium for creating vehicle-based linkage control rules. Background Art

[0002] As the level of automobile electrification, networking and intelligence continues to improve, scenario cards, which are automated control rules that contain trigger conditions and execution actions, can be used to perform personalized control of vehicles and various smart terminals connected to the vehicle through the network. One way to create a scenario card is to input the user's voice command into a large language model to generate a new scenario card. However, since the user's voice is more colloquial and non-professional, and the general large language model lacks professional knowledge in the field of vehicle control, the generated scenario cards are often not accurate enough. Summary of the invention

[0003] The embodiment of the present application provides a vehicle-based linkage control rule creation method to solve the above-mentioned problem.

[0004] In the first aspect, the present application provides a method for creating a vehicle-based linkage control rule, including: pre-parsing the standard linkage control rules to determine the operating conditions and operating actions corresponding to a single operation from the standard linkage control rules; when a user voice command is detected, semantically recognizing the user voice command; from the parsed results of multiple standard linkage control rules, searching and identifying the operating conditions and operating actions that have a matching relationship with the semantic recognition results; and combining the found operating conditions and operating actions into a new linkage control rule.

[0005] In the second aspect, an embodiment of the present application provides a vehicle-based linkage control rule creation device, including: a rule parsing module, which is used to pre-parse standard linkage control rules to determine the operating conditions and operating actions corresponding to a single operation from the standard linkage control rules; a voice recognition module, which is used to perform semantic recognition on user voice commands when user voice commands are detected; a rule matching module, which is used to search and identify the parsed results of multiple standard linkage control rules to obtain operating conditions and operating actions that have a matching relationship with the semantic recognition results; a rule creation module, which combines the found operating conditions and operating actions into a new linkage control rule.

[0006] In a third aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, any of the methods described above is implemented.

[0007] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory, wherein the processor implements any of the above methods when executing the computer program.

[0008] The present application has the following advantages: according to the embodiment of the present application, the standard linkage control rules for controlling vehicles or smart devices are first parsed to determine the operating conditions and operating actions corresponding to the atomic functions of the vehicles or smart devices from the standard linkage control rules; when the user voice command is detected, the user voice command is semantically recognized; from the parsed results of multiple standard linkage control rules, the operating conditions and operating actions that have a matching relationship with the semantic recognition results are obtained after searching and identifying; the found operating conditions and operating actions are combined into new linkage control rules. Through the above scheme, the corpus corresponding to the atomic functions of vehicles and smart devices can be split and vectorized, and stored in the rule database for retrieval and matching by the large language model, and a prompt template is further introduced to convert the operating actions and operating conditions obtained by the large language model for matching the user voice command input into a format that meets the input specification of the large language model and then input into the large language model, so as to more efficiently and accurately use the large language model to generate new linkage control rules to achieve personalized control of vehicles and smart devices. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. These drawings only depict some implementation methods according to the present application and should not be regarded as limiting the scope of the present application.

[0010] Figure 1 A flowchart of a method for creating a vehicle-based linkage control rule according to an embodiment of the present application;

[0011] Figure 2 A schematic diagram of a method for generating operating conditions and operating actions applicable to a large language input specification provided by an embodiment of the present application;

[0012] Figure 3 A structural block diagram of a vehicle-based linkage control rule creation device provided in an embodiment of the present application; and

[0013] Figure 4 A block diagram of an electronic device created based on vehicle linkage control rules provided in an embodiment of the present application. DETAILED DESCRIPTION

[0014] In order to make the purpose, technical solution and advantages of the embodiments of the present application clearer, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0015] Some embodiments of the present application are described in detail below in conjunction with the accompanying drawings. In the case where the embodiments do not conflict with each other, the following embodiments and the features in the embodiments can be combined with each other. The steps of the method described in the embodiment can be interchanged as long as they can meet the linkage control rule creation implemented by the method provided in the embodiment of the present application.

[0016] In the field of intelligent vehicle control, the concept of "scenario" is sometimes used to describe the rules for automated control of vehicles and intelligent terminals. A scenario contains its trigger conditions and a series of predefined actions performed by the vehicle or intelligent terminal in the scenario. These trigger conditions can be the state of the vehicle, changes in the environment, received signals or user input, etc., and the actions can include control or communication of the vehicle and other intelligent terminals.

[0017] In some embodiments, the triggering condition of the scene is the change in the operating status of the hardware, software or environmental status monitored by the central control system of the vehicle. For example, the triggering condition may be that a sensor or other detection device configured on the vehicle side detects a change in a certain state parameter and reaches a preset threshold. The state parameter may be an operating state parameter of a component in each part of the vehicle (engine, chassis, body and electrical system), such as engine speed and battery power, or detected environmental parameters inside and outside the vehicle, such as temperature, distance to external objects, etc. As an example, a scene can be set, and the rule is that when it rains, actions such as closing the windows, turning on the wipers, and turning off the media playback are automatically executed to improve driving safety and comfort.

[0018] In some possible embodiments, the triggering condition of the scene can also be a data signal sent by other smart terminals received by the central control system of the vehicle. The smart terminal in this application is a general terminal with certain computing and communication capabilities, such as smart home devices (such as smart TVs, air conditioners, access control systems and other smart home appliances or smart home device systems), road infrastructure (such as parking meters and charging piles), unmanned remote control terminals (such as drones), etc. The remote smart terminal can send data to the cloud server through the corresponding communication protocol and communication method, and the cloud server will send the data to the vehicle side. The vehicle side determines that the condition is triggered according to the received data signal, and notifies the corresponding controlled components and related smart terminals in the vehicle to start executing a series of actions described in the corresponding scene.

[0019] In some possible embodiments, the scene trigger condition can also be a timing condition. For example, a scene can be designed with a rule that when the system time is 10 pm, actions such as opening the car trunk and charging the battery are performed. Then, when the system time reaches 10 pm, it is a trigger condition.

[0020] In this application, the subject that performs actions in the scene is not limited to the vehicle (or the controlled components of the vehicle), but also includes the various remote intelligent terminals mentioned above. As mentioned above, the vehicle and the remote intelligent terminal can communicate data with the cloud server as an intermediary. Based on this, the vehicle can send control instructions to each intelligent terminal through the corresponding communication channel according to the communication protocol supported by each intelligent terminal to control it to complete the actions described in the scene.

[0021] As an example, the garage door control device at the user's residence is equipped with a signal receiving module that can receive signals sent by the vehicle and forwarded via a cloud server. The following scenario can be designed: when the vehicle drives to a distance of about 50 meters from the garage of the residence, the vehicle receives the location signal of the satellite positioning system, which triggers the vehicle to perform actions such as slowing down, closing windows, and turning off media playback. At the same time, the vehicle sends a command signal to the signal receiving component of the garage control device to enable the garage control device to control the opening of the garage door. At the same time, the vehicle also sends a command signal to the signal receiving component of the air conditioning and lighting system in the residence to start operating.

[0022] From the user's perspective, a scenario can be designed by simply determining the trigger conditions and a series of execution actions. However, the above scenario is only a descriptive definition of the automated control process and cannot be directly parsed and executed by the system. It is necessary to parse the rules described in the scenario with a specific strategy to obtain the linkage control rules and translate the linkage control rules into control signals that can be recognized by vehicles and smart terminals.

[0023] The embodiment of the present application provides a method for creating a linkage control rule based on a vehicle. The linkage control rule can be understood as a standardized abstraction of the above-mentioned scenario. The linkage control rule includes operating conditions and operating actions. There can be multiple operating actions. The control instruction corresponding to the operating action is used to instruct or notify the controlled component or intelligent terminal of the vehicle to complete the corresponding action. Each operating action corresponds to an atomic function of the controlled component or intelligent terminal of the vehicle. The atomic function of the terminal or device, or the atomic capability is the functional module of the smallest unit of the terminal device. For example, for air conditioning, its turning on and off, adjusting the temperature, and adjusting the wind speed are all atomic functions. "Turn on the air conditioner to 24 degrees Celsius" includes at least two atomic functions: turning on the air conditioner and adjusting the temperature. In order to achieve refined control of terminals of different types and produced by different manufacturers, the atomic functions of the terminal can be encapsulated into a unified state and operation interface for system call according to the relevant setting documents of different equipment manufacturers, so as to control the terminal to perform the required operations.

[0024] The execution subject of the embodiment of the present application may be an application, service, instance, software-based functional module, virtual machine (VM), container or cloud server, etc., or a hardware device with data processing function (such as a server or terminal device) or hardware chip (such as CPU, GPU, FPGA, NPU, AI acceleration card or DPU), etc. The device for realizing vehicle information collection can be deployed on the computing device of the application party providing the corresponding service or on the cloud computing platform providing computing power, storage and network resources.

[0025] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.

[0026] The technical solution of the present application and how the technical solution of the present application solves the above-mentioned technical problems are described in detail below with specific embodiments. The several specific embodiments listed can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. The embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0027] The present application embodiment provides a method for creating a linkage control rule based on a vehicle, such as Figure 1The flowchart of the linkage control rule creation method 100 of an embodiment of the present application is shown, and the method may include: step S101, pre-parsing the standard linkage control rules to determine the operating conditions and operating actions corresponding to a single operation from the standard linkage control rules; step S102, when the user voice command is detected, semantic recognition of the user voice command; step S103, from the parsing results of multiple standard linkage control rules, searching and identifying to obtain the operating conditions and operating actions that have a matching relationship with the semantic recognition results; and step S104, combining the found operating conditions and operating actions into a new linkage control rule.

[0028] The following is a detailed description of each step.

[0029] In step S101, the standard linkage control rules are parsed in advance to determine the operation conditions and operation actions corresponding to a single operation from the standard linkage control rules.

[0030] In an embodiment of the present application, the vehicle can obtain the user's voice input through a voice recognition system, and extract the instructions for creating a scene card in the voice input to create a corresponding scene card. However, in actual use scenarios, since the language description used by the user is usually colloquial, and the user often lacks professional knowledge of the vehicle's controlled components or intelligent terminal functional modules, it is necessary to first compare and match the semantic recognition results, that is, the scene described by the user in natural language, with the data in the pre-established rule database, and the rule database includes atomic operation actions and operation conditions separated from the standard linkage control rules. The atomic operation action is an operation or corresponding control instruction that cannot be further separated or simplified, corresponding to the encapsulated atomic capabilities of the vehicle (or its controlled components) and the intelligent terminal described above, and the atomic operation condition is also a single trigger condition that cannot be further separated. That is to say, the rule database can be understood as a corpus of atomic functional information of vehicles and intelligent terminals.

[0031] In some embodiments, the method 100 further includes: separating the operating conditions and operating actions corresponding to a single operation from the standard linkage control rules, and saving them in a corresponding rule database.

[0032] The standard linkage control rules are pre-designed by professionals and are standardized descriptions of the functions determined based on the official configuration files or user manuals of the vehicles or smart terminals provided by the manufacturers. The terms used are relatively standardized and unified, and can cover all atomic functions supported by the product. Therefore, they can be used as a sample source for the standard rule library for matching with the semantic recognition results.

[0033] The above-mentioned standard linkage control rules need to be extracted through vector features, that is, they are segmented and encoded by the neural network model and then stored in the rule database. Segmentation is to segment the sentences in the standard linkage control rules into multiple sub-texts, and encoding is to encode each sub-text in the multiple sub-texts and obtain the corresponding text vector. Word segmentation and encoding are determined according to the structure and parameters of the training task and model. Therefore, the text encoding network can be set according to different needs, and the network can be trained through specific tasks to obtain a pre-trained network, so as to obtain the vector representation of the input text based on the expected split granularity for the input text. "Granularity" describes the smallest unit that constitutes the text sequence, that is, the smallest unit that the text sequence can be segmented, such as character granularity, word granularity, etc. By segmenting and encoding the text, natural language can be converted into vector data that can be calculated and analyzed by computers.

[0034] In one possible implementation, the standard linkage control rules can be split with atomic functions as the text splitting granularity. For the purpose of this application, using atomic functions as the smallest unit of text splitting can ensure that the model accurately matches user input with samples in the standard rule base. For example, the standard linkage control rules contain control instructions for the atomic operation action of "turn on the car air conditioner" (i.e., instructions corresponding to an atomic function), which will be encoded as an atomic operation action as a whole and stored in the rule database, and will not be split into three independent entities of "turn on", "car", and "air conditioning".

[0035] The above is step S101, and then proceed to step S102.

[0036] In step S102, when a user voice command is detected, semantic recognition is performed on the user voice command.

[0037] In the linkage control rule creation method provided in the embodiment of the present application, the voice data is the voice control instructions input by the user into the user terminal (such as a smart phone, smart tablet, etc.) or the vehicle-side central control system through the voice interaction function. The voice data is processed to obtain the corresponding semantic recognition result to determine whether the user's current demand is to create a new linkage control rule.

[0038] In some embodiments, the voice control instructions input by the user terminal or the vehicle central control system can be uploaded to the cloud server, and the cloud server inputs the received voice data into the neural network model to obtain the semantic recognition results corresponding to the voice data.

[0039] In some embodiments, before searching for operating conditions and operating actions that have a matching relationship with the semantic recognition results obtained after identification from the parsed results of multiple standard linkage control rules, the method 100 also includes: determining that the semantic recognition results correspond to instructions for creating linkage control rules based on the vehicle status.

[0040] In the embodiment of the present application, the voice data needs to be converted into corresponding text data through speech recognition technology such as Automatic Speech Recognition (ASR), and then input into the neural network model for semantic recognition. Automatic speech recognition technology can convert the vocabulary content in human speech into computer-readable input, such as keystrokes, binary codes or character sequences. It should be noted that automatic speech recognition only converts voice data into corresponding text, and does not involve changes in meaning. Then, the system can determine the corresponding semantic recognition result based on the text data corresponding to the voice data.

[0041] In some embodiments, determining that the semantic recognition result corresponds to an instruction to create a linkage control rule between the vehicle and the terminal device includes: inputting the semantic recognition result into a neural network model, and determining that the semantic recognition result corresponds to an instruction to create a linkage control rule between the vehicle and the smart device based on the domain classification result and the intent classification result of the neural network model.

[0042] For example, the NLU (Natural Language Understanding) neural network model can be used to handle the above semantic recognition tasks. NLU is a general term for all methods, models or tasks that support machine understanding of text content. Its purpose is to enable the system to read the information we input, so that it can understand the text, language and extract information to help perform downstream tasks such as text classification, grammatical analysis and information search. In the field of NLU in artificial intelligence, a commonly used semantic recognition classification method is based on domain-intent classification, that is, multiple domains and intent categories are pre-divided, and the domains and different intents are orthogonal to each other. For example, the instruction "Please help me turn on the air conditioner in the car" can be divided into the "air conditioning" domain and further classified into the intent category of "turn on the car air conditioning"; while "Please help me turn on the air conditioner at home in advance" still belongs to the "air conditioning" domain, but the intent is completely different, involving a one-time control of the air conditioner at home.

[0043] The above is step S102, and then proceed to step S103.

[0044] In step S103, the operation conditions and operation actions that have a matching relationship with the semantic recognition results are searched and identified from the analysis results of the plurality of standard linkage control rules.

[0045] In an embodiment of the present application, for voice data recognized as instructing the creation of linkage control rules between vehicles and smart devices, the semantic recognition result further includes text data corresponding to the scene information described by the user in natural language in the voice data. That is, in actual usage scenarios, the system receives voice information containing instructions to create a new scene and the subsequent specific design of the scene, and uploads the complete voice data to the cloud server for subsequent parsing and processing. In the following, where there is no ambiguity, "semantic recognition result" will be used to refer to the scene information described by the user in natural language.

[0046] Similar to the aforementioned standard linkage control rules, the semantic recognition results are also segmented and encoded by a neural network model. It should be noted that the neural network model used for text segmentation and encoding should be consistent with the neural network model used to segment and encode the standard linkage control rules, thereby ensuring that the same (similar) texts are encoded as the same (similar) text vectors.

[0047] In some embodiments, searching for operating conditions and operating actions that have a matching relationship with a semantic recognition result obtained after identification from the parsed results of multiple standard linkage control rules includes: generating a first vector corresponding to the semantic recognition result; calculating similarity relationship data between second vectors corresponding to the operating conditions and operating actions split out of the standard linkage control rules and the first vector, and screening out operating conditions and operating actions that have a matching relationship with the semantic recognition result based on the similarity relationship data.

[0048] By calculating the vector similarity corresponding to the second vector corresponding to the semantic recognition result and the first vector corresponding to the atomized operation action and operation condition in the rule database, and according to the preset judgment criteria, the operation condition and operation action matching the semantic recognition result are determined. For example, the cosine similarity between the second vector corresponding to the semantic recognition result and the first vector corresponding to the atomized operation condition and operation action in the rule database can be calculated respectively, and the atomized operation condition and operation action corresponding to the text vector in the standard rule library with the highest cosine similarity to the second vector corresponding to the semantic recognition result is determined as the operation condition and operation action matching the semantic recognition result. The combination of the operation condition and the operation action is essentially consistent with the scene content designed by the user, and includes multiple vectors, each of which represents the atomized operation condition and operation action described above. The linkage control rule creation method 100 provided in the embodiment of the present application establishes a rule database based on the atomized function information of the vehicle controlled components and the intelligent terminal in advance, and then matches the user's semantic recognition results with the rule data, thereby realizing the mapping of user input to the atomic function of the device.

[0049] The above is step S03, and then proceed to step S104.

[0050] In step S104, the found operating conditions and operating actions are combined into a new linkage control rule.

[0051] The above operating conditions and operating actions matching the semantic recognition results are only an unordered collection of such information, and need to be transformed and processed specifically to create new linkage control rules. The method 100 provided in the embodiment of the present application uses a large language model to process the above atomic operating conditions and operating actions.

[0052] In an embodiment of the present application, combining the found operating conditions and operating actions into new linkage control rules includes: calling an instruction database to generate new linkage control rules according to the found operating conditions and operating actions, wherein the instruction database stores control instructions corresponding to single operations, and the control instructions are collected from vehicles and various terminal devices, and the process of generating new linkage rules is achieved by calling a large language model.

[0053] The instruction database contains API interfaces of atomic functions or services of vehicle controlled components or smart devices, and these API interfaces of atomic functions or services have a corresponding relationship with the text vectors of operating conditions or operating actions in the rule database.

[0054] Large language models (LLM) usually refer to language models that are trained on large-scale text corpora and contain tens of billions of parameters (or more), such as GPT-3, PaLM, LLaMA, etc. The current large language models use Transformer architectures and pre-training objectives (such as Language Modeling) similar to small models. The main difference from small models is the increase in model size, training data, and computing resources. The performance of large language models often follows the law of expansion, but for some capabilities, they will only appear when the scale of the language model reaches a certain level. These capabilities are called "emergent capabilities". Representative emergent capabilities include contextual learning, instruction following, and step-by-step reasoning. Although large language models have no essential difference from traditional small models in terms of model architecture and training methods, they have achieved breakthrough improvements in natural language processing tasks (including natural language understanding and natural language generation) based on far more training data and parameter numbers than traditional models.

[0055] However, large models can answer more universal questions, but they usually lack knowledge and information unique to a professional field. When directly applied to a professional field, they lack knowledge depth and timeliness. There are currently two mainstream solutions. The first is to fine-tune the large language model based on the knowledge in the field as training samples. However, this method has a high overall investment cost and a low update frequency, so it is not suitable for all developers.

[0056] The embodiment of the present application adopts another scheme, namely the aforementioned construction of a rule database. In some embodiments, the aforementioned text processing of the neural network model of the semantic recognition result and the standard linkage control rule can also be performed by a large language model, that is, the same large language model is used to encode the two into vectors. Using a large language model to vector model the standard linkage control rule and establish a database, the large language model can "learn" the knowledge in this field. The process of the large language model receiving the semantic recognition result and searching for similar results in the standard linkage rule database is the process of the large language model using the learned knowledge to deal with new problems. By extracting the relevant knowledge in the field of vehicle control designed by this application through vector features to obtain atomic operation actions and operation conditions and storing them in the rule database, the user's input (i.e., the aforementioned semantic recognition result) is matched with the data in the rule database, i.e., the unique knowledge in this field stored in the large language model, so as to accurately generate scene cards, i.e., linkage control rules, for the user according to the user's requirements.

[0057] In some embodiments, before the operation conditions and operation action information are input into the large language model, the combination of the searched operation conditions and operation actions into a new linkage control rule further includes: arranging the operation conditions and operation actions into a format suitable for the input specification of the large language model. In the embodiment of the present application, the operation actions and operation conditions can be added to a pre-set rule template, i.e., a prompt template, so as to obtain operation actions and operation conditions suitable for the input specification of the large language model, and the operation actions and operation conditions are input into the large language model to obtain a new linkage control rule.

[0058] Although the performance of large language models is good, the quality or accuracy of their output results still depends on the accuracy and clarity of the commands entered by the user. In order to have an efficient conversation interaction with the model, you can enter prompts in the model. Prompts are text fragments used to guide large language models to generate natural language texts. They are used to guide the model to produce text results that meet our expectations. Generally, high-quality prompts are clear and specific, and provide sufficient context or background information.

[0059] As an example, the voice command issued by the user is "I want the air purifier and the living room lights to be turned on automatically every time I get home, and then if the outdoor temperature exceeds 30 degrees, turn the air conditioner to 26 degrees." The pre-set prompt template is "Please help me design a scene card. When the trigger condition is met, execute the instruction corresponding to the trigger condition: Trigger condition 1:..., execute instruction 1..., execute instruction 2...; Trigger condition 2, execute instruction 1, execute condition 2..." The above user voice instructions are semantically recognized, the operation actions and operation conditions are matched, and filled into the prompt template to obtain: "Please help me design a scenario card. When the trigger condition is met, the instruction corresponding to the trigger condition is executed: The vehicle positioning position is within 100 meters from the residence: start the air purifier; turn on the living room headlights. The vehicle positioning position is within 100 meters from the residence and the outdoor temperature is greater than 30 degrees: start the home air conditioner; set the home air conditioner cooling temperature to 26 degrees. "The above user instructions, prompt templates and arranged linkage control rules are only used as a figurative example, and are not intended to limit their content or form. It should be noted that the prompt templates and the arranged linkage control rules are not limited to natural language forms, they can also be corresponding text vectors. Both forms can be processed by the large language model. In summary, comparing and matching the user's semantic recognition results with the data in the rule database to obtain the corresponding operation actions and operation conditions is equivalent to standardizing the corpus, that is, splitting the user's more colloquial expressions and converting them into professional expressions in the professional field, and then using the prompt template to arrange and reorganize the operation actions and operation conditions, and further standardize them from the perspective of logical order, so as to obtain arranged operation actions and operation conditions that are more in line with the input specifications of the large language model, which is conducive to the large language model to output new linkage control rules more accurately and quickly.

[0060] Figure 2 Schematic diagram of a method for generating operation conditions and operation actions applicable to a large language input specification provided in an embodiment of the present application. Figure 2 As shown, the semantic recognition result, that is, the text data corresponding to the user input voice data, is converted into a corresponding first vector after vector feature extraction operation, that is, the text is split and encoded. The first vector is searched and matched with the second vector corresponding to the standard rule information stored in the rule database (including the operation conditions and operation actions split from the standard linkage control rules), and the operation action and operation condition corresponding to the semantic recognition result are determined. Then, the operation action and operation condition are input into the prompt template to obtain the arranged operation action and operation condition and input them into the large language model to generate the linkage control rules. The arranged operation action and operation condition are also the operation action and operation condition applicable to the input specification of the large language model.

[0061] In some embodiments, the method 100 also includes: sending the linkage control rule to a rule execution component of the vehicle, the rule execution component is used to monitor the vehicle status and send a notification to execute the operation action to the vehicle or smart device when the vehicle status meets the operation condition.

[0062] In some embodiments, the linkage control rules output by the above-mentioned large language model are script files in JSON format, and creating new linkage control rules based on operation actions and operation conditions that match the semantic recognition results also includes: inputting the linkage control rules in JSON format into a script parser for script parsing to obtain the parsed linkage control rules; and providing the parsed linkage control rules to the rule execution component of the vehicle.

[0063] JSON file is a file of simple data structure and object stored in JavaScript Object Notation (JSON) format, which is a standard data exchange format. It is mainly used to transfer data between Web applications and servers. JSON file is a lightweight, text-based, human-readable file that can be edited using a text editor. In the embodiment of the present application, the script file in JSON format output by the large language model based on the input operation action and operation condition describes the triggering conditions, start node, end node, and control instruction sequence (including the content of each instruction and the sequential relationship between them) of the linkage control rule. By pre-defining the output format of the large language model, it outputs a JSON format script file, which is easy for developers to understand and process, and is easy to convert into a file format that matches the data protocol of the rule execution component responsible for executing the linkage control rule.

[0064] After parsing, the linkage control rules in the JSON format are converted into codes that can be parsed and executed by the vehicle-side rule execution component, which can be a vehicle-side rule engine configured in the vehicle control domain for arbitrating and executing the code. After monitoring that the vehicle state meets the corresponding operation conditions, the vehicle or smart device is notified to execute the control instructions corresponding to the corresponding operation actions in the linkage control rules.

[0065] In some embodiments, the newly created linkage control rules can also be stored in the cloud server so that the vehicle-side engine can call them when connected to the Internet. In addition, the newly created linkage control rules can be further processed and stored in the aforementioned standard rule database to enrich the corpus data that can be used by the large language model.

[0066] Corresponding to the application scenario and method of the method provided in the embodiment of the present application, the embodiment of the present application also provides a vehicle-based linkage control deployment device. Figure 3The structure block diagram of a vehicle-based linkage control deployment device 300 according to an embodiment of the present application is shown. The device 300 may include:

[0067] A rule parsing module 301 is used to parse the standard linkage control rules in advance to determine the operation conditions and operation actions corresponding to a single operation from the standard linkage control rules;

[0068] The speech recognition module 302 is used to perform semantic recognition on the user's speech command when a user's speech command is detected;

[0069] A rule matching module 303 is used to search and identify the operation conditions and operation actions that have a matching relationship with the semantic recognition results from the parsing results of multiple standard linkage control rules;

[0070] The rule creation module 304 combines the found operation conditions and operation actions into a new linkage control rule.

[0071] In some embodiments, the rule creation module 304 is also specifically used to: call the instruction database to generate new linkage control rules according to the found operating conditions and operating actions, the instruction database stores control instructions corresponding to single operations, and the control instructions are collected from vehicles and various terminal devices. The process of generating new linkage rules is achieved by calling a large language model.

[0072] In some embodiments, the rule creation module 304 may further include a rule arrangement submodule for arranging the operation conditions and operation actions into a format suitable for the large language model input specification.

[0073] In some embodiments, the device 300 may further include a rule database creation module, which is used to separate the operation conditions and operation actions corresponding to a single operation from the standard linkage control rules and save them in the corresponding rule database.

[0074] In some embodiments, the rule matching module 303 may include:

[0075] A vector generation submodule, used to generate a first vector corresponding to the semantic recognition result;

[0076] The vector similarity calculation submodule is used to calculate the similarity relationship data between the second vector corresponding to the operating conditions and operating actions separated from the standard linkage control rules and the first vector, and to screen the operating conditions and operating actions that have a matching relationship based on the semantic recognition results according to the similarity relationship data.

[0077] In some embodiments, the device 300 further includes a linkage control rule creation determination module, which is used to determine whether the semantic recognition result corresponds to an instruction to create a linkage control rule based on the vehicle state.

[0078] In some embodiments, the linkage control rule creation and determination module is specifically used to input the semantic recognition result into a neural network model, and determine the semantic recognition result corresponding to the instruction to create a linkage control rule between the vehicle and the smart device based on the domain classification result and the intention classification result of the neural network model.

[0079] In a possible implementation, the linkage control rule is a script file in JSON format.

[0080] In some embodiments, the device 300 also includes a rule sending module for sending the linkage control rule to the rule execution component of the vehicle, and the rule execution component is used to monitor the vehicle status and send a notification to the vehicle or smart device to perform the operation action when the vehicle status meets the operation conditions.

[0081] In some embodiments, the rule sending module may further include:

[0082] The script parsing submodule is used to input the linkage control rules in JSON format into the script parser for script parsing to obtain the parsed linkage control rules;

[0083] The parsing rule sending submodule is used to provide the parsed linkage control rule to the rule execution component of the vehicle.

[0084] In a possible implementation manner, the user's voice data is uploaded based on a user terminal or the vehicle.

[0085] The functions of each module in each device in the embodiments of the present application can be found in the corresponding description in the above method, and have corresponding beneficial effects, which will not be repeated here.

[0086] Figure 4 is a block diagram of an electronic device used to implement an embodiment of the present application. Figure 4 As shown, the electronic device includes: a memory 401 and a processor 402. The memory 401 stores a computer program that can be run on the processor 402. When the processor 402 executes the computer program, the method in the above embodiment is implemented. The number of the memory 401 and the processor 402 can be one or more.

[0087] The electronic device also includes:

[0088] The communication interface 403 is used to communicate with external devices and perform data exchange transmission.

[0089] If the memory 401, the processor 402 and the communication interface 403 are implemented independently, the memory 401, the processor 402 and the communication interface 403 can be connected to each other through a bus and communicate with each other. The bus can be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 4 Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0090] Optionally, in a specific implementation, if the memory 401, the processor 402 and the communication interface 403 are integrated on a chip, the memory 401, the processor 402 and the communication interface 403 can communicate with each other through an internal interface.

[0091] An embodiment of the present application provides a computer-readable storage medium storing a computer program, which implements the method provided in the embodiment of the present application when the program is executed by a processor.

[0092] An embodiment of the present application also provides a computer program product, which includes a computer program, and when the computer program is executed by a processor, the method provided in any embodiment of the present application is implemented.

[0093] An embodiment of the present application also provides a chip, which includes a processor for calling and executing instructions stored in the memory from the memory, so that a communication device equipped with the chip executes the method provided by the embodiment of the present application.

[0094] An embodiment of the present application also provides a chip, including: an input interface, an output interface, a processor and a memory, wherein the input interface, the output interface, the processor and the memory are connected via an internal connection path, and the processor is used to execute the code in the memory. When the code is executed, the processor is used to execute the method provided in the embodiment of the present application.

[0095] It should be understood that the processor may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. It is worth noting that the processor may be a processor supporting the Advanced RISC Machines (ARM) architecture.

[0096] Further, optionally, the above-mentioned memory may include a read-only memory and a random access memory. The memory may be a volatile memory or a non-volatile memory, or may include both volatile and non-volatile memory. Among them, the non-volatile memory may include a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory may include a random access memory (RAM), which is used as an external cache. By way of exemplary but not limiting description, many forms of RAM are available. For example, static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM) and direct memory bus random access memory (DR RAM).

[0097] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function according to the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium.

[0098] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples, unless they are contradictory.

[0099] In addition, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, a feature defined as "first" or "second" may explicitly or implicitly include at least one of the features. In the description of this application, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0100] Any process or method described in the flow chart or otherwise described herein can be understood as a module, fragment or portion of a code representing one or more executable instructions for implementing the steps of a specific logical function or process. And the scope of the preferred embodiment of the present application includes other implementations, in which the functions may not be performed in the order shown or discussed, including in a substantially simultaneous manner or in a reverse order according to the functions involved.

[0101] The logic and / or steps described in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, which can be specifically implemented in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or used in combination with these instruction execution systems, devices or apparatuses.

[0102] It should be understood that the various parts of the present application can be implemented with hardware, software, firmware or a combination thereof. In the above embodiments, multiple steps or methods can be implemented with software or firmware stored in a memory and executed by a suitable instruction execution system. All or part of the steps of the above embodiment method can be completed by instructing the relevant hardware through a program, which can be stored in a computer-readable storage medium, and when the program is executed, it includes one of the steps of the method embodiment or a combination thereof.

[0103] In addition, each functional unit in each embodiment of the present application can be integrated into a processing module, or each unit can exist physically separately, or two or more units can be integrated into one module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of a software functional module. If the above-mentioned integrated module is implemented in the form of a software functional module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. The storage medium can be a read-only memory, a disk or an optical disk, etc.

[0104] The above is only an exemplary embodiment of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various changes or substitutions within the technical scope recorded in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for creating a vehicle-based linkage control rule, comprising: Preparing the standard linkage control rules to determine the operation conditions and operation actions corresponding to the single operation from the standard linkage control rules; When a user's voice command is detected, semantic recognition of the user's voice command is performed; From the parsing results of multiple standard linkage control rules, the operating conditions and operating actions that have a matching relationship with the semantic recognition results are obtained after searching and identifying; The found operating conditions and operating actions are combined into new linkage control rules.

2. The method according to claim 1, wherein: The step of combining the found operating conditions and operating actions into a new linkage control rule includes: The instruction database is called to generate new linkage control rules according to the found operating conditions and operating actions. The instruction database stores control instructions corresponding to single operations, and the control instructions are collected from vehicles and various terminal devices. The process of generating new linkage rules is achieved by calling a large language model.

3. The method according to claim 2, wherein: The combining of the found operating conditions and operating actions into a new linkage control rule also includes: The operation conditions and operation actions are organized into a format suitable for the large language model input specification.

4. The method according to claim 1, wherein: The method further comprises: The operating conditions and operating actions corresponding to a single operation are separated from the standard linkage control rules and saved in the corresponding rule database.

5. The method according to claim 1, wherein: The step of searching, from the parsing results of the plurality of standard linkage control rules, the operation conditions and operation actions having a matching relationship with the semantic recognition results obtained after recognition comprises: Generate a first vector corresponding to the semantic recognition result; The similarity relationship data between the second vectors corresponding to the operation conditions and operation actions separated from the standard linkage control rule and the first vector are calculated, and the operation conditions and operation actions having a matching relationship based on the semantic recognition result are screened according to the similarity relationship data.

6. The method according to claim 1, wherein: Before searching, from the parsing results of the plurality of standard linkage control rules, for operating conditions and operating actions having a matching relationship with the semantic recognition results obtained after recognition, the method further comprises: Determining that the semantic recognition result corresponds to an instruction to create a linkage control rule based on the vehicle state.

7. The method according to claim 6, wherein: Determining that the semantic recognition result corresponds to an instruction to create a linkage control rule between the vehicle and the terminal device includes: The semantic recognition result is input into a neural network model, and the semantic recognition result corresponds to an instruction to create a linkage control rule between the vehicle and the smart device according to the domain classification result and the intention classification result of the neural network model.

8. The method according to claim 1, wherein: The method further comprises: The linkage control rule is sent to a rule execution component of the vehicle, and the rule execution component is used to monitor the vehicle state and send a notification of executing an operation action to the vehicle or smart device when the vehicle state meets the operation condition.

9. The method according to claim 8, wherein: The linkage control rule is a script file in JSON format, and the rule execution component that provides the linkage control rule to the vehicle includes: Input the linkage control rules in JSON format into the script parser for script parsing to obtain the parsed linkage control rules; The parsed linkage control rule is provided to a rule execution component of the vehicle.

10. The method according to claim 1, wherein: The user's voice data is uploaded based on the user terminal or the vehicle.

11. A vehicle-based linkage control rule creation device, comprising: A rule parsing module, used to parse the standard linkage control rules in advance, so as to determine the operation conditions and operation actions corresponding to a single operation from the standard linkage control rules; A speech recognition module is used to perform semantic recognition on user speech commands when a user speech command is detected; A rule matching module is used to search and identify the operation conditions and operation actions that match the semantic recognition results from the parsing results of multiple standard linkage control rules; The rule creation module combines the found operating conditions and operating actions into new linkage control rules.

12. A computer-readable storage medium, wherein: The computer-readable storage medium includes a stored program, and when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the linkage control rule creation method according to any one of claims 1 to 10.

13. An electronic device comprising a memory, a processor and a computer program stored in the memory, wherein the computer program implements the method according to any one of claims 1 to 10 when executed by the processor.

Images