Method, apparatus, electronic device and readable storage medium for in-vehicle semantic understanding

Abstract

The present application relates to the technical field of intelligent cockpits, and provides a method, device, electronic device and readable storage medium for in-vehicle semantic understanding. The method includes: receiving an input text; matching a corresponding target in-vehicle entity for the input text according to the static rules and entity matching method corresponding to the input text in a grammar parser, where the static rules include semantic rules between specified texts and tags of in-vehicle entities, and the grammar parser includes static rules, entity matching methods and multi-mode matching methods; matching corresponding target control information for the input text according to the multi-mode matching method in the grammar parser; using the target in-vehicle entity and the target control information as the semantic understanding result, and controlling the target in-vehicle entity to perform corresponding operations according to the target control information. The method for in-vehicle semantic understanding provided by the present application can provide smooth semantic understanding when the vehicle-side network condition is poor or there is no network connection to obtain the semantic understanding result, thereby improving the user experience.

Description

Technical Field

[0001] This application relates to the technical field of intelligent cockpits, and particularly to a method, device, electronic device and readable storage medium for in-vehicle semantic understanding. Background Art

[0002] In the prior art, a deep neural network model in the field of semantic parsing can avoid the problem that a stable network connection cannot be established between the vehicle terminal and the cloud due to large network latency or no network connection during vehicle driving. However, the deep neural network model has high requirements for training data, long development time, and high computing cost. Using a grammar parser can achieve reliable data results, low latency, ensure the timeliness of sending instructions, and quickly solve faults and exceptions. However, the computing power of the vehicle terminal is limited. When the number of grammar rules included in the grammar parser reaches a certain amount, the matching time of the input text and the grammar rules is slow, and there are many in-vehicle entities, which easily leads to entity conflicts and incorrect matching of grammar rules. Summary of the Invention

[0003] In view of this, the embodiments of this application provide a method, device, electronic device and readable storage medium for in-vehicle semantic understanding to solve the problem of incorrect matching of grammar rules caused by entity conflicts when there are many in-vehicle entities in the prior art.

[0004] In the first aspect of the embodiments of this application, a method for in-vehicle semantic understanding is provided, including:

[0005] Receiving an input text;

[0006] Matching a corresponding target in-vehicle entity for the input text according to the static rule and entity matching method corresponding to the input text in the grammar parser. The grammar parser includes static rules, entity matching methods and multi-mode matching methods, and the static rules include semantic rules between specified texts and labels of in-vehicle entities;

[0007] Matching a corresponding target control information for the input text according to the multi-mode matching method in the grammar parser;

[0008] Taking the target in-vehicle entity and the target control information as semantic understanding results, and controlling the target in-vehicle entity to perform corresponding operations according to the target control information.

[0009] In the second aspect of the embodiments of this application, a device for in-vehicle semantic understanding is provided, including:

[0010] A receiving module, configured to receive an input text;

[0011] A matching module, configured to match a corresponding target vehicle-mounted entity for the input text according to the static rule and entity matching method corresponding to the input text in the grammar parser. The grammar parser includes a static rule, an entity matching method, and a multi-mode matching method. The static rule includes a semantic rule between a specified text and a label of a vehicle-mounted entity;

[0012] The matching module is further configured to match corresponding target control information for the input text according to the multi-mode matching method in the grammar parser;

[0013] A control module, configured to use the target vehicle-mounted entity and the target control information as a semantic understanding result, and control the target vehicle-mounted entity to perform corresponding operations according to the target control information.

[0014] In a third aspect of the embodiments of the present application, an electronic device is provided, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the above method are implemented.

[0015] In a fourth aspect of the embodiments of the present application, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are implemented.

[0016] The beneficial effects of the embodiments of the present application compared with the prior art are as follows: The input text is input into the grammar parser, and the grammar parser matches the corresponding target static rule, entity matching, and multi-mode matching method for the input text to match the corresponding target vehicle-mounted entity and target control information. Among them, the static rule can reduce the situation of vehicle-mounted entity conflicts caused by too many vehicle-mounted entities or entity matching rules during the entity matching process. The multi-mode matching can improve the matching speed, accuracy, and flexibility of the target control information. Thus, the grammar parser performs semantic understanding on the input text, improving the user's usage experience and the speed of processing the input text. At the same time, the grammar parser is set on the vehicle side, which can provide smooth semantic understanding when the vehicle-side network condition is poor or there is no network connection, and control the vehicle-mounted entity to perform corresponding operations according to the semantic understanding result, thereby improving the user's usage experience. Description of the Drawings

[0017] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following will briefly introduce the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings in the following description are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0018] Figure 1 It is a schematic diagram of an application scenario of an embodiment of the present application;

[0019] Figure 2 It is a schematic flowchart of a method for in-vehicle semantic understanding provided by an embodiment of the present application;

[0020] Figure 3 It is a schematic flowchart of a multi-mode matching method provided by an embodiment of the present application;

[0021] Figure 4 It is a schematic flowchart of an application method for in-vehicle semantic understanding provided by an embodiment of the present application;

[0022] Figure 5 It is a schematic diagram of a device for in-vehicle semantic understanding provided by an embodiment of the present application;

[0023] Figure 6 It is a schematic diagram of an electronic device provided by an embodiment of the present application. Detailed implementation manners

[0024] In the following description, specific details such as specific system architectures and technologies are presented for the purpose of illustration rather than limitation, so as to thoroughly understand the embodiments of the present application. However, those skilled in the art should clearly understand that the present application can also be implemented in other embodiments without these specific details. In other cases, detailed descriptions of well-known systems, devices, circuits, and methods are omitted to avoid unnecessary details from interfering with the description of the present application.

[0025] A method and a device for in-vehicle semantic understanding according to an embodiment of the present application will be described in detail below with reference to the accompanying drawings.

[0026] Figure 1 It is a schematic diagram of an application scenario of an embodiment of the present application. The application scenario may include a first terminal device 101, a second terminal device 102, a server 103, and a network 104.

[0027] The first terminal device 101 may be hardware or software. When the first terminal device 101 is hardware, it may be various electronic devices having a display screen and supporting communication with the server 103, including but not limited to smart phones, tablet computers, laptop portable computers, and desktop computers, etc.; when the first terminal device 101 is software, it may be installed in the above-mentioned electronic devices. The first terminal device 101 may be implemented as multiple software or software modules, or may be implemented as a single software or software module, and the embodiments of the present application do not limit this. Further, various applications may be installed on the first terminal device 101, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, etc.

[0028] The second terminal device 102 can be hardware or software. When the second terminal device 102 is hardware, it can be various electronic devices with a display screen and supporting communication with the server 103, including but not limited to in-vehicle computers and vehicle controllers, etc.; when the second terminal device 102 is software, it can be installed in the above-mentioned electronic devices. The second terminal device 102 can be implemented as multiple software or software modules, or can also be implemented as a single software or software module, and the embodiments of the present application do not limit this. Further, various applications can be installed on the second terminal device 102, such as data processing applications, instant messaging tools, social platform software, search applications, shopping applications, vehicle control applications, etc.

[0029] The server 103 can be a server providing various services. For example, it can be a background server that receives requests sent by terminal devices establishing communication connections with it. This background server can receive and analyze requests sent by terminal devices and generate processing results. The server 103 can be a single server, or can also be a server cluster composed of several servers, or can also be a cloud computing service center, and the embodiments of the present application do not limit this.

[0030] It should be noted that the server 103 can be hardware or software. When the server 103 is hardware, it can be various electronic devices providing various services for the first terminal device 101 and the second terminal device 102. When the server 103 is software, it can be multiple software or software modules providing various services for the first terminal device 101 and the second terminal device 102, or can also be a single software or software module providing various services for the first terminal device 101 and the second terminal device 102, and the embodiments of the present application do not limit this.

[0031] The network 104 can be a wired network connected by coaxial cables, twisted pairs, and optical fibers, or can also be a wireless network that can interconnect various communication devices without wiring, such as Bluetooth, Near Field Communication (NFC), Infrared, etc., and the embodiments of the present application do not limit this.

[0032] It should be noted that the specific types, quantities, and combinations of the first terminal device 101, the second terminal device 102, the server 103, and the network 104 can be adjusted according to the actual needs of the application scenario, and the embodiments of the present application do not limit this.

[0033] In the prior art, speech recognition technology relies on specific equipment, while the semantic understanding of text is basically based on the powerful computing power of cloud services. Therefore, when the vehicle needs to understand the input text semantically, it needs to understand it through cloud services. Therefore, smooth semantic understanding can only be performed when the network connection between the vehicle and the cloud is stable. However, during the driving process, it is inevitable to encounter large network delays or even no network connection, such as when the vehicle is driving in a tunnel or remote mountainous area. At this time, the vehicle cannot establish a network connection with the cloud service or the connection is unstable. If it relies on the cloud for semantic understanding, the in-vehicle voice dialogue function will not be available.

[0034] To avoid this situation, semantic understanding needs to be performed on the vehicle side, but the existing hardware cannot transplant the entire cloud service to the vehicle side. There are two most important problems. One is that the computing power on the vehicle side is insufficient, and the other is that the content of the entire semantic understanding is relatively large, which will almost occupy all the space of the vehicle side central control. Therefore, the above solution does not conform to the actual logic. To address the above problems, a semantic understanding solution that occupies less resources, consumes less computing power, and does not rely on the network is needed.

[0035] With the development of deep neural network models, they have also been applied more deeply in the field of semantic understanding. However, deep neural network models also have their own shortcomings that cannot be ignored, such as the existence of "black box" nature, large demand for training data for training models, long development time, and high computational cost. During vehicle driving, the driving section and road conditions are constantly changing, so the mobile signal fluctuates greatly and the stability of network connection cannot be guaranteed. The voice input and semantic understanding of vehicle-mounted equipment are extremely dependent on the network, and there are high requirements for the reliability of semantic understanding results, instruction timeliness, and troubleshooting timeliness. In this case, it is necessary to achieve a reliable method for semantic understanding, low latency, immediate instruction delivery, and rapid resolution of fault anomalies. The shortcomings of the deep neural network model do not meet the above requirements. The model based on grammar rules can meet the above requirements. Therefore, this application selects a model based on grammar rules (hereinafter referred to as a grammar parser) for semantic understanding.

[0036] At the same time, due to the limited computing power of the vehicle, when the number of rules in the grammar parser reaches a certain level, the process of matching with the input text takes a long time, and there are many on-board entities. In this case, it is easy to cause conflicts between multiple on-board entities and thus rule matching errors.

[0037] To address the above issues, this application writes static rules into the grammar parser, and together with entity matching, matches the target vehicle-mounted entities for the input text, and adopts a multi-mode matching method to match the target operation information. At the same time, combined with the unstable connection between the vehicle and the cloud network during driving and the hardware environment of the vehicle, a data update strategy with periodic incremental updates is introduced.

[0038] The process of obtaining a semantic understanding result through the grammar parser of the present application and controlling the target vehicle entity to perform corresponding operations according to the target control information has multiple advantages such as controllable expected results, fast development speed, easy maintenance and upgrade, low computational cost, and strong timeliness, and can solve the problems faced by semantic understanding in the vehicle field.

[0039] Figure 2 It is a schematic flowchart of a method for vehicle-mounted semantic understanding provided by an embodiment of the present application. As Figure 2 shown, the method for vehicle-mounted semantic understanding includes the following steps:

[0040] S201, receiving the input text;

[0041] S202, matching the corresponding target vehicle entity for the input text according to the static rules and entity matching method corresponding to the input text in the grammar parser;

[0042] S203, matching the corresponding target control information for the input text according to the multi-mode matching method in the grammar parser;

[0043] S204, taking the target vehicle entity and the target control information as the semantic understanding result, and controlling the target vehicle entity to perform corresponding operations according to the target control information.

[0044] Figure 2 The method for vehicle-mounted semantic understanding of Figure 1 can be executed by the second terminal device 102 and the server of

[0045] Among them, the grammar parser includes static rules, entity matching methods, and multi-mode matching methods. The static rules include semantic rules between specified texts and tags of vehicle entities.

[0046] In some embodiments, the input text of the user is received, and the input text is matched through the grammar parser. Among them, the corresponding target vehicle entity is matched for the input text according to the static rules and entity matching method in the grammar parser corresponding to the input text, and the corresponding target control information is matched for the input text according to the multi-mode matching method in the grammar parser corresponding to the input text.

[0047] Among them, the multi-mode matching method may be an AC (Aho-Corasick automaton) automaton matching method. The AC automaton is used to match the target control information that matches the input text, and can solve the problem of slow matching process speed of the grammar parser and the input text for matching the target control information caused by insufficient vehicle-side computing power. When it is necessary to match the pattern strings of multiple actions in the string corresponding to an input text, that is, the semantic understanding of actions other than vehicle entities in the input text can be implemented by using the AC automaton.

[0048] Among them, the input text received from the user can be received by a device related to the Automatic Speech Recognition (ASR) technology to recognize the voice message of the user and obtain the input text.

[0049] Use the target vehicle entity and target control information obtained by matching the input text according to the grammar parser as the semantic understanding result, and control the target vehicle entity to perform corresponding operations according to the target control information.

[0050] Before receiving the input text, abstract the vehicle entity object and establish the data information of the vehicle entity. The data information includes the labels and basic information of each vehicle entity. Upload the data information of all vehicle entities to the cloud. When the vehicle is connected to the network, the full-text search engine Lucene pulls the data information of the vehicle entities stored in the cloud through a scheduled task.

[0051] Among them, Lucene is a lightweight full-text search engine based on inverted index, which has the advantages of not relying on the network, small volume and fast search speed. Lucene can save the data information of vehicle entities on the local device, so there is no need to rely on network connection. The advantage of this local storage is that even when there is no network connection or the network connection is unstable, the in-vehicle voice dialogue system can still access and query the data information of vehicle entities. At the same time, because Lucene adopts lightweight data structures and indexing technologies, it occupies less resources and is suitable for use in environments with limited resources, that is, it is very suitable for the in-vehicle environment.

[0052] Continuing with the above example, after pulling the data information of the vehicle entities stored in the cloud, establish an inverted index through Lucene so that the corresponding vehicle entity can be determined by the label of the vehicle entity when performing semantic understanding on the input text.

[0053] According to the technical solution provided by the embodiment of the present application, the static rules in the grammar parser can reduce the situation of vehicle entity conflicts caused by too many vehicle entities or too many entity matching rules during the entity matching process. Multi-mode matching can improve the matching speed, accuracy and flexibility of the target control information, so as to perform semantic understanding on the input text through the grammar parser, improving the user experience and the speed of processing the input text. At the same time, because the grammar parser is set on the vehicle side, it can provide smooth semantic understanding when the vehicle-side network condition is poor or there is no network connection, and control the corresponding vehicle entity to perform corresponding operations according to the semantic understanding result, thereby improving the user experience.

[0054] In the aspect of matching vehicle-mounted entities in the embodiments of the present application, by utilizing the powerful search function of Lucene in combination with static rules, it is possible to quickly and accurately match the target vehicle-mounted entity corresponding to the user's input information. Lucene provides efficient indexing and search technologies, which can determine the target vehicle-mounted entity according to the keyword and entity matching method. Moreover, Lucene can also search for specific entities including names, locations, dates, etc. Updating the rules in Lucene based on the embodiments of the present application can achieve more types of matching, thereby accurately identifying the entity information in the user's input text.

[0055] In the aspect of multi-mode matching in the embodiments of the present application, the matching algorithm of the AC automaton is used to interpret and respond to the user's intention. The AC automaton has the following remarkable advantages:

[0056] High efficiency: In the process of searching and matching, compared with the method of matching one by one, the AC automaton can match multiple patterns simultaneously, thus greatly improving the matching speed between the input text and the target control information. When understanding the semantics of the input text including a large amount of control information and complex intentions, it can improve the speed and accuracy of semantic understanding;

[0057] Precision: By constructing a finite state machine, the AC automaton can match the input text with the control information in the grammar parser. This kind of matching is based on the theory of deterministic finite automata, so it can ensure the accuracy of the target control information matched according to the input text, avoiding misunderstandings and incorrect matches, enabling the method provided by the present application to better understand the user's needs and provide accurate answers or execute corresponding actions;

[0058] Flexibility: The AC automaton has high flexibility and can easily cope with various changes and expansions of control information. For example, by adding or deleting corresponding patterns in the AC automaton, the AC automaton can quickly adapt to the changes in the matching rules of new control information or business rules, so that the vehicle-mounted semantic understanding method provided by the present application has strong adaptability and scalability and can cope with the continuously evolving semantic understanding tasks.

[0059] In some embodiments, matching the corresponding target vehicle-mounted entity for the input text according to the static rules and entity matching method corresponding to the input text in the grammar parser includes:

[0060] Matching the corresponding vehicle-mounted entity for the input text based on the entity matching method;

[0061] When there are multiple vehicle-mounted entities, matching the corresponding target vehicle-mounted entity for the input text among multiple vehicle-mounted entities based on the static rules corresponding to the input text.

[0062] The grammar parser matches the input text with each node of the tree network of the grammar parser in a recursive manner. If the match is successful, it means that the semantics of the input text is the semantics represented by this node.

[0063] Upon receiving the input text, the corresponding vehicle-mounted entity is matched for the input text according to the entity matching method. The matching result may be one vehicle-mounted entity or multiple vehicle-mounted entities. When there is one vehicle-mounted entity, it is determined that this vehicle-mounted entity is the target vehicle-mounted entity. When there are multiple vehicle-mounted entities, the corresponding target vehicle-mounted entity is matched for the input text among the multiple vehicle-mounted entities according to the static rules corresponding to the input text.

[0064] Among them, the target vehicle-mounted entity may be one or multiple.

[0065] In an exemplary embodiment of the present application, taking the input text "Set the driving mode to sports" as an example, the keyword "sports" in the input text may represent multiple vehicle-mounted entities, such as multiple vehicle-mounted entities including the accelerator, brake, lights, seat angle, fan, etc. Among them, the lights are not the vehicle-mounted entities corresponding to the sports driving mode. The static rules include the semantic rules between the "sports mode" and the labels of the vehicle-mounted entities. Through the semantic rules between the "sports mode" and the labels of the vehicle-mounted entities in the static rules, the corresponding target vehicle-mounted entity is matched for the input text among the above multiple vehicle-mounted entities.

[0066] According to the technical solution provided by the embodiment of the present application, the entity matching of the input text can be performed through the grammar parser, and when there are multiple matched vehicle-mounted entities, the matching is performed again according to the static rules, so as to ensure that the grammar parser can accurately understand the semantics of the input text.

[0067] In some embodiments, when there are multiple vehicle-mounted entities, based on the static rules, matching the corresponding target vehicle-mounted entity for the input text among the multiple vehicle-mounted entities includes:

[0068] Obtain the data information of multiple vehicle-mounted entities, and the data information includes the labels and basic information of the vehicle-mounted entities;

[0069] Determine the static rule corresponding to the input text among all the static rules as the target static rule;

[0070] Based on the target static rule, screen the data information that conforms to the target static rule among the multiple data information, and use the vehicle-mounted entity corresponding to the data information of the static rule as the target vehicle-mounted entity.

[0071] Obtain the data information of multiple vehicle-mounted entities in the above embodiments, and determine the static rule corresponding to the input text among all the static rules in the grammar parser as the target static rule. According to the target static rule, screen the data information that conforms to the target static rule among the multiple data information, and use the vehicle-mounted entity corresponding to the data information screened by the static rule as the target vehicle-mounted entity.

[0072] Among them, the target vehicle-mounted entity can be one or multiple.

[0073] In an exemplary embodiment of the present application, taking the input text "Change the ambient light to sports" as an example for exemplary illustration. The keyword "sports" included in the input text may represent multiple vehicle-mounted entities. However, not all vehicle-mounted entities with the "sports" label meet the conditions of the input text. When receiving this input text, perform a text search through Lucene to obtain multiple vehicle-mounted entities that match "sports". Further, among the multiple vehicle-mounted entities, match according to the labels of the vehicle-mounted entities and the static rules, and select the vehicle-mounted entity that conforms to the input text "Change the ambient light to sports" as the target vehicle-mounted entity.

[0074] Furthermore, the static rules and the data information of the vehicle-mounted entities can be loaded into AbnfFuzzer when the program starts. AbnfFuzzer is a tool for multi-pattern matching and can efficiently match multiple static rules.

[0075] According to the technical solution provided by the embodiments of the present application, it is possible to more accurately match vehicle-mounted entities through static rules and labels to determine the target vehicle-mounted entity. At the same time, it is possible to avoid conflicts between multiple vehicle-mounted entities. Loading the static rules and labels into AbnfFuzzer can improve the matching efficiency and accuracy during the matching with the input text.

[0076] In some embodiments, matching the corresponding target control information for the input text according to the multi-pattern matching method in the grammar parser includes:

[0077] Repeat the following steps based on the multi-pattern matching method in the order of the strings of the input text:

[0078] If there is no matching result for the current string in the grammar rule, use the next string as the current string;

[0079] If there is a matching result for the current string in the grammar rule, use the matching result as the target control information, and then use the next string as the current string.

[0080] Among them, the grammar parser further includes grammar rules, and the grammar rules include syntax rules and semantic rules.

[0081] In an exemplary embodiment of the present application, taking the matching of corresponding target control information for the input text through the Aho-Corasick automaton (AC automaton) as an example, an exemplary illustration is given. The AC automaton uses the control information in the grammar parser as the pattern string and matches it with the input text. The specific method is as follows:

[0082] Use the grammar rule where the entry corresponding to the control information is located as the pattern string, and insert the pattern string into the AC automaton so that each label and control information corresponding to the grammar rule will be used as an independent pattern string to perform an exact match with the string through the AC automaton.

[0083] Continuing with the above example, during the process of matching for the input text, the AC automaton starts from the first string of the string corresponding to the input text and performs a state transition based on the current string, that is, it performs a match according to the grammar rule in the order of the string corresponding to the input text. When a pattern string is matched for the current string, it means that a target control information has been matched for this string, and the target control information is used as the matching result. If there is no matching result for the current string in the pattern string, the next string is used as the current string.

[0084] Figure 3 It is a schematic flowchart of a multi-pattern matching method provided by an embodiment of the present application. As Figure 3 shown, when the string corresponding to the input text is matched through the multi-pattern matching method, the Chinese character string of the input text is matched by the AC automaton. If the AC automaton successfully matches the Chinese character string of the input text, the target control information is determined. If the AC automaton fails to match the Chinese character string of the input text, the pinyin string of the input text is matched. If the match fails, the target control information match fails, indicating that there is no target control information for the current string. If the match is successful, the target control information for the current string is determined. Repeat the above steps until all strings corresponding to the input text are matched.

[0085] According to the technical solution provided by the embodiment of the present application, the AC automaton is used to match the target control information for the string of the input text. The AC automaton can automatically handle state transition and backtracking, so as to efficiently match multiple control information in the string of the input text, thereby obtaining the target control information.

[0086] In this application, an AC automaton is introduced to match target control information. The time complexity of the AC automaton has a linear relationship with the length of the input text and the total length of the pattern strings. The input text in vehicles is often short text, thus reducing the time overhead for matching target control information. Therefore, it can effectively solve the problem of slow matching speed of target control information caused by insufficient vehicle-side computing power, improve the matching efficiency, and provide better performance and user experience for related tasks (such as command matching and action recognition in vehicle-mounted systems) in practical applications.

[0087] In some embodiments, controlling a target vehicle-mounted entity to perform a corresponding operation according to target control information includes:

[0088] If the input text has completed the matching of all target vehicle-mounted entities, controlling all target vehicle-mounted entities to perform corresponding operations according to the target control information;

[0089] If the input text has not completed the matching of all target vehicle-mounted entities, record the target control information.

[0090] When the input text has completed the matching of all target vehicle-mounted entities, that is, all target vehicle-mounted entities corresponding to the input text have been determined, the target control information obtained by the AC automaton matching controls all target vehicle-mounted entities to perform corresponding operations. If the input text has not completed all target vehicle-mounted entities, that is, all target vehicle-mounted entities corresponding to the input text have not been determined, record the target control information obtained by the AC automaton matching, so as to control all corresponding target vehicle-mounted entities to perform corresponding operations according to the target control information when the matching of the target vehicle-mounted entities corresponding to the input text is completed.

[0091] According to the technical solution provided by the embodiments of this application, it is possible to process or record the target control information obtained by the AC automaton matching, so as to efficiently perform semantic understanding of the input text and improve the user's usage experience.

[0092] In some embodiments, it further includes:

[0093] Sending an update request to the cloud;

[0094] Receiving an update instruction that is fed back by the cloud based on the update request and has been compressed, so as to update the update content corresponding to the update instruction based on the current grammar parser and the data information of the vehicle-mounted entity. The update content is the update content of the latest grammar parser and the latest data information in the cloud that is different from the current grammar parser and the current data information.

[0095] A data packaging and loading method provided by an embodiment of this application is as follows: It is loaded on the vehicle side in the form of a data compression package and a Software Development Kit (SDK) to be called by the application programs on the vehicle side. Among them, the data compression package is decompressed to obtain a data packet. The data compression package mainly includes the relevant data of Lucene and the data stored by Lucene. The software development kit is simply called the SDK package.

[0096] Continuing with the above example, when initializing the in-vehicle computer, a part of the basic data can be pre-installed. If the user has a high requirement for the semantic understanding of offline voice during use, a prompt message can be set on the display screen of the vehicle side to prompt the user whether to download the offline voice package. If the user selects yes, the local data packet and SDK package are updated through a combination of compressed pull and incremental pull strategies.

[0097] Among them, the SDK package can also be provided to the in-vehicle terminal by compressing the entire set of application programs into a referenceable SDK package through a development tool.

[0098] Since the data information stored by Lucene and data packets such as grammar parsers need to be stored in the central control space of the vehicle, rather than performing semantic understanding through the cloud, factors such as data transmission traffic and space occupancy need to be considered during local semantic understanding. In the embodiment of this application, a combination of compressed pull and incremental pull strategies is adopted when updating the data packet and SDK package. Specifically:

[0099] Send an update request to the cloud and receive the update instruction that is fed back by the cloud according to the update request and has been compressed, so as to perform an update based on the update content corresponding to the update instruction on the basis of the current grammar parser and the data information of the current vehicle entity.

[0100] Among them, the update content is the update content in which the latest grammar parser and data information in the cloud are different from the current grammar parser and current data information, that is, the data corresponding to the latest or changed data information and grammar parser is updated.

[0101] Among them, the cloud synchronizes the data recorded in the vehicle-side Lucene. When the cloud feeds back an update instruction, the update content is determined by comparing all the data recorded in the cloud currently with the data included in the recorded vehicle-side Lucene.

[0102] According to the technical solution provided by the embodiments of the present application, the data corresponding to the update content is compressed and transmitted in the cloud, which can reduce the transmission time and the occupancy of network bandwidth. And when receiving an update instruction, it can record information such as the update time or version number, and at the same time perform incremental pull data transmission, which can reduce the data transmission volume and processing time to reduce the overhead of data transmission. It can also efficiently update and maintain data information under limited in-vehicle unit resources to ensure that the in-vehicle voice dialogue system always has the latest semantic understanding ability. At the same time, it can ensure that the data information of the in-vehicle entity is the latest version. The technical solution provided by the embodiments of the present application will regularly send update requests to the cloud to check the update status of the data information of the in-vehicle entity.

[0103] In some embodiments, the construction process of the grammar parser includes:

[0104] Establish a grammar parser based on the extended Backus-Naur form;

[0105] Write static rules into the grammar parser, and add the tags corresponding to the in-vehicle entities as rules to the grammar parser.

[0106] When constructing the grammar parser, a tree-like network is established for all rules (including static rules, grammar rules, entity matching, and multi-mode matching) in the grammar parser through a parent-child relationship. This tree-like network is similar to a binary tree, where each node is a semantic rule.

[0107] In an exemplary embodiment of the present application, an abnfFuzzer grammar tree-like structure parser is constructed by using antlr to generate a lexical analyzer, a syntax analyzer, and a tree analyzer, etc. When there is no rule data, the grammar parser disassembles the input text word by word, searches for in-vehicle entities through Lucene, uses static rules to avoid the problem of multiple in-vehicle entities, and uses an Aho-Corasick automaton to match the entries other than the in-vehicle entities in the input text. The specific rules for establishing a grammar parser based on the extended Backus-Naur form are as follows:

[0108] Start with "rule_[domain]_[intent]_[instruction]" and write the grammar rules for each domain. When writing weak rules, add the "_weak" identifier in the rule record.

[0109] An exemplary general rule is written as: rule_DOMAIN_INTENT_set_any = [anyword] / mid_source_set. This rule indicates a certain intent in a certain domain, DOMAIN is the corresponding domain, INTENT is the corresponding intent, set is the corresponding control information, any is the in-vehicle entity, and " / " represents two representations of the same semantics.

[0110] Among them, taking the rules in the DEVICE domain, a semantic rule representing opening a vehicle-mounted entity at a certain position as an example, it can be expressed as follows:

[0111] rule_DEVICE_OPEN=mid_DEVICE_OPEN_AIRCONDITIONER / mid_DEVICE_OPEN_WINDOW, indicating opening the air conditioner or the window;

[0112] rule_DEVICE_CLOSE=mid_DEVICE_CLOSE_AIRCONDITIONER / mid_DEVICE_CLOSE_WINDOW, which means opening the air conditioner or the car window;

[0113] rule_DEVICE_OPEN_weak=mid_DEVICE_OPEN_weak_1 / mid_DEVICE_OPEN_weak_2, indicating two weak rules related to opening. For example, words such as "开开" and "开开" can be identified as weak rules related to "open" to improve the versatility of the grammar parser.

[0114] Among them, the useless temporary variable of mid_ is in the form of: mid_DOMAIN_INTENT_set=anyword. The useless temporary variable can be written as follows:

[0115] open = "open" / "start" / "open", indicating the control information corresponding to the string;

[0116] seat=@co-driver seat? |(front|rear)row(left|right)(side|side)@, indicating the specific position of the vehicle-borne entity, including static rules;

[0117] mid_DEVICE_OPEN_AIRCONDITIONER=open@{{devicename}}@, indicating the correspondence between control information and vehicle-mounted entities;

[0118] mid_DEVICE_OPEN_WINDOW = open@{{seat}}@@{{devicename}}@, indicating the correspondence between the control information and the located vehicle-mounted entity, which is a refinement of the previous rule;

[0119] mid_DEVICE_OPEN_weak_1 = @^.{0,3} wants to @open@{{devicename}}@, which is the first weak rule included in the rule;

[0120] mid_DEVICE_OPEN_weak_2 = @^.{0,3}@open@{{devicename}}@@OK? @, which is the second weak rule included in the rule.

[0121] Among them, devicename comes from the data information of the vehicle entity stored in Lucene.

[0122] Figure 4 1 is a flow chart of an application method for vehicle-mounted semantic understanding provided by an embodiment of the present application. Figure 4 As shown, taking the input text "open the passenger window" and the corresponding semantic understanding result output by the grammar parser as an example, an exemplary explanation is given.

[0123] After receiving the input text "open the passenger window", it is input into the grammar parser. The grammar parser uses Lucene to perform a full-text search and finds that the corresponding target vehicle-mounted entity is "window". The static rule detects that the corresponding position of the vehicle-mounted entity is "passenger seat". The multi-mode matching detects that the target control information is "open", and the semantic understanding result is "open passenger seat window". The passenger seat window is controlled to open according to the semantic understanding result.

[0124] According to the technical solution provided in the embodiment of the present application, a grammar parser can be constructed through the extended Backus-Naur form, and static rules can be added to the grammar parser to avoid conflicts between the rules in the grammar parser and multiple on-board entities that appear after the number of on-board entities reaches a certain level, and AbnfFuzzer can be loaded into the program during program initialization to improve the accuracy of semantic understanding.

[0125] It should be understood that the size of the serial numbers of the steps in the above embodiments does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0126] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0127] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0128] Figure 5 Schematic diagram of a vehicle-mounted semantic understanding device provided in an embodiment of the present application. Figure 5 As shown, the vehicle-mounted semantic understanding device includes: a receiving module 501, a matching module 502 and a control module 503, wherein:

[0129] A receiving module 501, configured to receive input text;

[0130] A matching module 502, configured to match a corresponding target vehicle-mounted entity for the input text according to the static rule and entity matching method corresponding to the input text in a grammar parser, where the grammar parser includes static rules, entity matching methods, and multi-mode matching methods, and the static rules include semantic rules between specified text and tags of vehicle-mounted entities;

[0131] The matching module 502 is further configured to match a corresponding target control information for the input text according to the multi-mode matching method in the grammar parser;

[0132] A control module 503, configured to use the target vehicle-mounted entity and the target control information as a semantic understanding result, and control the target vehicle-mounted entity to perform corresponding operations according to the target control information.

[0133] In some embodiments, the matching module 502 is configured to match a corresponding target vehicle-mounted entity for the input text according to the static rule and entity matching method corresponding to the input text in the grammar parser, for:

[0134] Match a corresponding vehicle-mounted entity for the input text based on the entity matching method;

[0135] When there are multiple vehicle-mounted entities, match a corresponding target vehicle-mounted entity for the input text among the multiple vehicle-mounted entities based on the static rule corresponding to the input text.

[0136] In some embodiments, the matching module 502 is configured to, when there are multiple vehicle-mounted entities, match a corresponding target vehicle-mounted entity for the input text among the multiple vehicle-mounted entities based on the static rule, for:

[0137] Obtain data information of multiple vehicle-mounted entities, where the data information includes tags and basic information of the vehicle-mounted entities;

[0138] Determine the static rule corresponding to the input text among all static rules as the target static rule;

[0139] Based on the target static rule, screen the data information that conforms to the target static rule among the multiple data information, and use the vehicle-mounted entity corresponding to the data information of the static rule as the target vehicle-mounted entity.

[0140] In some embodiments, the grammar parser further includes grammar rules, where the grammar rules include syntax rules and semantic rules, and the matching module 502 is further configured to match a corresponding target control information for the input text according to the multi-mode matching method in the grammar parser, for:

[0141] Repeat the following steps based on the multi-mode matching method in the order of the input text string:

[0142] If the current string has no matching result in the grammar rule, the next string is used as the current string;

[0143] If the current string has a matching result in the grammar rule, the matching result is used as the target control information, and then the next string is used as the current string.

[0144] In some embodiments, the matching module 502 is configured to control the target vehicle-mounted entity to perform corresponding operations according to the target control information, including:

[0145] If the input text has completed the matching of all target vehicle-mounted entities, control all target vehicle-mounted entities to perform corresponding operations according to the target control information;

[0146] If the input text has not completed the matching of all target vehicle-mounted entities, record the target control information.

[0147] In some embodiments, the vehicle-mounted semantic understanding device is further configured to:

[0148] Send an update request to the cloud;

[0149] Receive the update instruction feedback by the cloud based on the update request and processed by compression, so as to update the update content corresponding to the update instruction based on the current grammar parser and the data information of the current vehicle-mounted entity. The update content is the update content of the latest grammar parser and the latest data information of the cloud that is different from the current grammar parser and the current data information.

[0150] In some embodiments, the matching module 502 is further configured to construct a grammar parser:

[0151] Build a grammar parser based on the extended Backus-Naur form;

[0152] Write the static rules into the grammar parser, and add the labels corresponding to the vehicle-mounted entities as rules to the grammar parser.

[0153] Figure 6 It is a schematic diagram of the electronic device 6 provided by the embodiment of the present application. As Figure 6 shown, the electronic device 6 of this embodiment includes: a processor 601, a memory 602, and a computer program 603 stored in the memory 602 and operable on the processor 601. When the processor 601 executes the computer program 603, the steps in the above-mentioned method embodiments are implemented. Alternatively, when the processor 601 executes the computer program 603, the functions of each module / unit in the above-mentioned device embodiments are implemented.

[0154] The electronic device 6 may be an electronic device such as a desktop computer, a notebook, a handheld computer, and a cloud server. The electronic device 6 may include, but is not limited to, a processor 601 and a memory 602. Those skilled in the art can understand that Figure 6 These are merely examples of the electronic device 6 and do not constitute a limitation on the electronic device 6. It may include more or fewer components than those shown in the figure, or different components.

[0155] The processor 601 may be a central processing unit (CPU), or may be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc.

[0156] The memory 602 may be an internal storage unit of the electronic device 6. For example, the hard disk or memory of the electronic device 6. The memory 602 may also be an external storage device of the electronic device 6. For example, a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a flash card, etc. equipped on the electronic device 6. The memory 602 may also include both an internal storage unit and an external storage device of the electronic device 6. The memory 602 is used to store computer programs and other programs and data required by the electronic device.

[0157] Those skilled in the art can clearly understand that, for the convenience and simplicity of description, only the above division of each functional unit and module is used as an example. In actual applications, the above functions may be allocated to different functional units and modules according to needs, that is, the internal structure of the device is divided into different functional units or modules to complete all or part of the functions described above. Each functional unit and module in the embodiment may be integrated into one processing unit, or each unit may exist physically alone, or two or more units may be integrated into one unit. The above integrated unit may be implemented in the form of hardware or in the form of a software functional unit.

[0158] When the integrated module / unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, to implement all or part of the processes in the above-described embodiment methods of this application, it can also be completed by a computer program instructing relevant hardware. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by a processor, the steps of the above-described various method embodiments can be implemented. The computer program can include computer program code, and the computer program code can be in the form of source code, object code, executable file, or some intermediate form, etc. The computer-readable medium can include: any entity or device capable of carrying computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disc, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal, and software distribution medium, etc. It should be noted that the content included in the computer-readable medium can be appropriately increased or decreased according to the requirements of legislation and patent practice within the jurisdiction. For example, in some jurisdictions, according to legislation and patent practice, the computer-readable medium does not include electrical carrier signals and telecommunication signals.

[0159] The above embodiments are only used to illustrate the technical solutions of this application, rather than to limit them; although this application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the various embodiments of this application, and should all be included in the protection scope of this application.

Claims

1. A method for in-vehicle semantic understanding, characterized in that, it includes: Receiving the input text; According to the static rules and entity matching methods corresponding to the input text in the grammar parser, matching the corresponding target in-vehicle entity for the input text. The grammar parser includes the static rules, the entity matching methods, and multi-mode matching methods. The static rules include semantic rules between specified texts and tags of in-vehicle entities; According to the multi-mode matching method in the grammar parser, matching the corresponding target control information for the input text; Taking the target in-vehicle entity and the target control information as the semantic understanding result, and controlling the target in-vehicle entity to perform corresponding operations according to the target control information.

2. The method according to claim 1, characterized in that, matching the corresponding target in-vehicle entity for the input text according to the static rules and entity matching methods corresponding to the input text in the grammar parser, including: Based on the entity matching method, matching the corresponding in-vehicle entity for the input text; When there are multiple in-vehicle entities, based on the static rules corresponding to the input text, matching the corresponding target in-vehicle entity for the input text among the multiple in-vehicle entities.

3. The method according to claim 2, characterized in that, When there are multiple in-vehicle entities, matching the corresponding target in-vehicle entity for the input text among the multiple in-vehicle entities based on the static rules, including: Obtaining the data information of the multiple in-vehicle entities, where the data information includes the tags and basic information of the in-vehicle entities; Determining the static rule corresponding to the input text among all the static rules as the target static rule; Based on the target static rule, screening the data information that conforms to the target static rule among the multiple data information, and taking the in-vehicle entity corresponding to the data information of the static rule as the target in-vehicle entity.

4. The method according to claim 1, the grammar parser further includes grammar rules, and the grammar rules include syntax rules and semantic rules. According to the grammar parser, matching the corresponding target control information for the input text based on the multi-mode matching method, including: In the order of the strings of the input text, repeating the following steps based on the multi-mode matching method: If the current string has no matching result in the grammar rules, taking the next string as the current string; If the current string has a matching result in the grammar rules, taking the matching result as the target control information, and then taking the next string as the current string.

5. The method according to claim 4, characterized in that, controlling the target in-vehicle entity to perform corresponding operations according to the target control information, including: If all the target in-vehicle entities of the input text have been matched, controlling all the target in-vehicle entities to perform corresponding operations according to the target control information; If all the target in-vehicle entities of the input text have not been matched, recording the target control information.

6. The method according to claim 1, It is characterized in that further comprising: sending an update request to the cloud; receiving an update instruction that is fed back by the cloud based on the update request and has been compressed, so as to update the update content corresponding to the update instruction based on the current grammar parser and the data information of the current vehicle-mounted entity, where the update content is the update content of the latest grammar parser and the latest data information of the cloud that are different from the current grammar parser and the current data information.

7. The method according to any one of claims 1 to 6, it is characterized in that the construction process of the grammar parser includes: establishing the grammar parser based on the extended Backus-Naur form; writing the static rules into the grammar parser, and adding the tags corresponding to the vehicle-mounted entities as rules to the grammar parser.

8. An in-vehicle semantic understanding device, it is characterized in that comprising: a receiving module configured to receive an input text; a matching module configured to match a corresponding target vehicle-mounted entity for the input text according to the static rule and entity matching method corresponding to the input text in the grammar parser, where the grammar parser includes the static rule, the entity matching method and a multi-mode matching method, and the static rule includes the semantic rule between the specified text and the tag of the vehicle-mounted entity; the matching module is further configured to match a corresponding target control information for the input text according to the multi-mode matching method in the grammar parser; a control module configured to use the target vehicle-mounted entity and the target control information as the semantic understanding result, and control the target vehicle-mounted entity to perform corresponding operations according to the target control information.

9. An electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, it is characterized in that when the processor executes the computer program, the steps of the method according to any one of claims 1 to 7 are implemented.

10. A computer-readable storage medium storing a computer program, it is characterized in that when the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.

Images