User operation instruction identification method, equipment control method and vehicle

Abstract

The invention discloses a user operation instruction recognition method and device and a vehicle. The user operation instruction recognition method comprises the steps that a user instruction text is acquired; performing text recognition on the user instruction text to obtain a text recognition result; and determining at least one user operation instruction and execution time period information of the at least one user operation instruction based on the text recognition result. The problem that an existing voice recognition scheme can only recognize the operation intention of the user and cannot meet the personalized instruction requirement of the user can be solved.

Description

Technical Field

[0001] The present application belongs to the field of intelligent control technology, and specifically relates to a user operation instruction recognition method, a device control method and a vehicle, as well as a user operation instruction recognition device, a device control device, a computer-readable storage medium and a computer program product. Background Art

[0002] In the field of intelligent control, the application of intelligent voice control systems has become more and more widespread. For example, with the development of automobile intelligence, in-vehicle intelligent voice systems have become an indispensable part of modern vehicles. The emergence of this system aims to improve the convenience, safety and comfort of driving. In the fast-paced modern life, users often need to perform multiple operations while driving, such as adjusting the temperature in the car, playing music, etc., but the existing voice recognition solutions can only recognize the user's operating intentions, such as turning off or on a certain function, and cannot meet the user's personalized command needs. Summary of the invention

[0003] The present application aims to solve at least one of the technical problems existing in the prior art. To this end, the present application proposes a user operation instruction recognition method, a device control method and a vehicle, as well as a user operation instruction recognition method device, a device control device, a computer-readable storage medium and a computer program product, which can solve the problem that the existing voice recognition scheme can only recognize the user's operation intention, such as turning off or turning on a certain function, and cannot meet the user's personalized instruction needs.

[0004] In a first aspect, an embodiment of the present application provides a method for identifying a user operation instruction, comprising:

[0005] Get the user command text;

[0006] Performing text recognition on the user instruction text to obtain a text recognition result, wherein the text recognition result includes at least one time operation text, at least one operation intention text, and an association relationship between the operation intention text and the time operation text;

[0007] Based on the text recognition result, at least one user operation instruction and execution time period information of the at least one user operation instruction are determined.

[0008] In some embodiments, it also includes:

[0009] A user operation instruction recognition result is generated based on at least one user operation instruction and execution time period information of at least one user operation instruction.

[0010] In some embodiments, the above-mentioned performing text recognition on the user instruction text to obtain the text recognition result includes:

[0011] Segmenting the user instruction text to obtain at least two sub-instruction texts included in the user instruction text, and positions of the sub-instruction texts in the user instruction text;

[0012] Acquire at least one time operation text and at least one operation intention text based on at least two sub-instruction texts;

[0013] Based on the position of the sub-instruction text in the user instruction text, the association relationship between the operation intention text and the time operation text is determined.

[0014] In some embodiments, the acquiring of at least one time operation text and at least one operation intention text based on at least two sub-instruction texts includes:

[0015] Text type recognition is performed on at least two sub-instruction texts to obtain at least one time operation text and at least one operation intention text.

[0016] In some embodiments, the above-mentioned determining the association relationship between the operation intention text and the time operation text based on the position of the sub-instruction text in the user text includes:

[0017] Based on the position of the sub-command text in the user command text, obtaining the position of the operation intention text in the user command text;

[0018] Based on the position of the operation intention text in the user instruction text, the association relationship between the operation intention text and the time operation text is determined.

[0019] In some embodiments, the above-mentioned sentence segmentation processing of the user instruction text includes:

[0020] The user instruction text is segmented through the deep learning segmentation model.

[0021] In some embodiments, the deep learning segmentation model includes at least two of a bidirectional encoder representation method, a convolutional neural network, and a conditional random field algorithm.

[0022] In some embodiments, the text type identification of at least two sub-instruction files includes:

[0023] Perform text type identification on at least two sub-instruction files based on the rule engine template.

[0024] In some embodiments, the determining of at least one user operation instruction and execution time period information of at least one user operation instruction based on the text recognition result includes:

[0025] Determine at least one user operation instruction based on at least one operation intention text;

[0026] Determine at least one operation time period on the time axis and the type of at least one operation time period based on at least one time operation text;

[0027] Based on the association relationship between the operation intention text and the time operation text, determine the index position of the user operation instruction in at least one operation time period;

[0028] Based on the index position of the user operation instruction in at least one operation time period and the type of the operation time period, the execution time period information of each user operation instruction is determined.

[0029] In some embodiments, the type of the above-mentioned operation time period includes a delay type or a continuous type.

[0030] In some embodiments, the above-mentioned determining the execution time period information of each user operation instruction based on the index position of the user operation instruction in at least one operation time period and the type of the operation time period includes:

[0031] If the user operation instruction is at the end position of the delay type operation time period, the execution time period of the user operation instruction is at least one operation time period after the end position; or,

[0032] If the user operation instruction is at the end position of the continuous type operation time period, the execution time period of the user operation instruction is the operation time period before the end position.

[0033] In a second aspect, an embodiment of the present application provides a device control method, including:

[0034] Obtaining a user operation instruction recognition result by the method of the first aspect;

[0035] Device control is performed based on the user operation instruction recognition results.

[0036] In some embodiments, the device is a vehicle.

[0037] In a third aspect, an embodiment of the present application provides a user operation instruction recognition device, comprising:

[0038] A first data acquisition module, used to acquire user instruction text;

[0039] A data recognition module, used to recognize the user instruction text to obtain a text recognition result, wherein the recognition result includes at least one time operation text, at least one operation intention text, and an association relationship between the operation intention text and the time operation text;

[0040] The information determination module is used to determine the execution time period information based on the text recognition result, at least one user operation instruction, and at least one user operation instruction.

[0041] In a fourth aspect, an embodiment of the present application provides a device control apparatus, including:

[0042] A second data acquisition module, used to obtain a user operation instruction recognition result through a user operation instruction recognition method;

[0043] The device control module is used to control the device based on the recognition result of the user operation instruction.

[0044] In a fifth aspect, an embodiment of the present application provides an electronic device, comprising: a processor and a memory, wherein the memory stores programs or instructions that can be run on the processor, and when the program or instructions are executed by the processor, the steps of the user operation instruction identification method described in the first aspect are implemented; and / or, when the program or instructions are executed by the processor, the steps of the device control method described in the second aspect are implemented.

[0045] In a sixth aspect, an embodiment of the present application provides a vehicle, comprising the electronic device described in the fifth aspect, wherein a memory of the electronic device stores a program or instruction that can be run on a processor, and when the program or instruction is executed by the processor, the steps of the user operation instruction identification method described in the first aspect are implemented; and / or, when the program or instruction is executed by the processor, the steps of the device control method described in the second aspect are implemented.

[0046] In a sixth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored, and when the program or instruction is executed by a processor, the steps of the user operation instruction identification method as described in the first aspect are implemented; and / or, when the program or instruction is executed by a processor, the steps of the device control method as described in the second aspect are implemented.

[0047] In the seventh aspect, an embodiment of the present application provides a computer program product, which, when executed by a vehicle processor, implements the steps of the user operation instruction identification method as described in the first aspect; and / or, when the program or instructions are executed by the processor, implements the steps of the device control method as described in the second aspect.

[0048] The technical solution provided by the present application requires first obtaining the user instruction text, then recognizing the user instruction text to obtain the text recognition result, and then determining at least one user operation instruction and the execution time period information of at least one user operation instruction based on the text recognition result. The above text recognition result includes at least one time operation text, at least one operation intention text, and the association between the operation intention text and the time operation text. Since the above technical solution uses a method combining deep learning and rule engine templates to recognize user operation instructions, it can better recognize user operation instructions containing delayed or continuous operations, so that users can preset operation content according to personal preferences and needs without distracting their current attention, and enable the device to automatically execute future operations, thereby not only ensuring the user's driving safety, but also providing users with a more personalized and intelligent driving experience.

[0049] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through the practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0050] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0051] Figure 1 A flowchart of a method for identifying a user operation instruction in an embodiment of the present application;

[0052] Figure 2 for Figure 1 A schematic diagram of a specific execution flow of step 102 of the illustrated embodiment;

[0053] Figure 3 for Figure 2 A schematic diagram of a specific execution flow of step 203 of the illustrated embodiment;

[0054] Figure 4 for Figure 1 A schematic diagram of a specific execution flow of step 103 of the illustrated embodiment;

[0055] Figure 5 This is a flow chart of a device control method in an embodiment of the present application;

[0056] Figure 6 This is a schematic diagram of the structure of a user operation instruction recognition device in an embodiment of the present application;

[0057] Figure 7 This is a schematic diagram of the structure of a device control device in an embodiment of the present application;

[0058] Figure 8This is a schematic diagram of the structure of an electronic device in an embodiment of the present application;

[0059] Fig. 9 It is a schematic diagram of the structure of a vehicle in an embodiment of the present application. DETAILED DESCRIPTION

[0060] The embodiments of the present application will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present application are shown in the accompanying drawings, it should be understood that the present application can be implemented in various forms and should not be construed as being limited to the embodiments described herein. Instead, these embodiments are provided to provide a more thorough and complete understanding of the present application. It should be understood that the drawings and embodiments of the present application are only for exemplary purposes and are not intended to limit the scope of protection of the present application.

[0061] It should be understood that the various steps described in the method implementation of the present application can be performed in different orders and / or performed in parallel. In addition, the method implementation may include additional steps and / or omit the steps shown. The scope of the present application is not limited in this respect.

[0062] As described in the background technology section, existing voice recognition solutions can only recognize the operation intentions contained in the voice command sentences when recognizing the user's voice operation instructions, and their response speed may not be fast enough, causing the driver to wait for a long time before seeing the corresponding operation results. When faced with some complex operation instructions or specific types of voice instructions, for example, turning on the air conditioner after 10 minutes, existing voice recognition solutions often cannot accurately recognize the operation requirements, which not only greatly affects the driver's user experience and reduces the smoothness and efficiency of the operation, but also cannot meet the user's demand for more personalized and refined command processing.

[0063] In addition, in the process of using the above-mentioned voice recognition scheme to identify and execute the user's operation instructions, there are also a series of practical problems. For example, since the driver will adjust the temperature in the car, play music and other operations while driving the vehicle, if the existing voice recognition scheme is used to complete the above-mentioned operations, it will distract the driver's attention and make it difficult for him to focus on the road conditions and driving itself, thereby increasing driving risks; moreover, for electric vehicles or hybrid vehicles to achieve the above-mentioned operations, they need to maintain an idle waiting state, which will cause unnecessary energy waste. This energy consumption will not only reduce the overall energy efficiency of the vehicle, but also directly affect the actual driving distance of the vehicle, requiring the driver to charge or refuel more frequently, further increasing the cost of use and time cost.

[0064] Therefore, the technical solution provided in the embodiment of the present application obtains the user instruction text, then recognizes the user instruction text to obtain the text recognition result, and then determines at least one user operation instruction and the execution time period information of at least one user operation instruction based on the text recognition result, and the above text recognition result includes at least one time operation text, at least one operation intention text, and the association between the operation intention text and the time operation text. Since the above technical solution uses a method combining deep learning and rule engine templates to recognize user operation instructions, it can better recognize user operation instructions containing delayed or continuous operations, so that users can preset operation content according to personal preferences and needs without distracting current attention, and enable the device to automatically execute future operations, thereby not only ensuring the user's driving safety, but also providing users with a more personalized and intelligent driving experience.

[0065] Figure 1 FIG. 1 is a flow chart of a method for identifying a user operation instruction in an embodiment of the present application. The method for identifying a user operation instruction can be implemented by a corresponding user operation instruction identification device, such as Figure 1 As shown, the method comprises the following steps:

[0066] Step 101: Obtain user instruction text;

[0067] The user instruction text in the technical solution provided in the embodiment of the present application refers to the text of the specific command or request entered by the user to interact with the system or device, and contains the operation information that the user wants the device to perform. The above user instructions can be provided by the user through keyboard input, touch screen operation, voice input and other input devices. For example, when the user issues an instruction by voice, the system will use automatic speech recognition technology to convert the above voice instructions into corresponding instruction text that can be understood by the machine.

[0068] Step 102: Perform text recognition on the user instruction text to obtain a text recognition result, wherein the text recognition result includes at least one time operation text, at least one operation intention text, and an association relationship between the operation intention text and the time operation text;

[0069] On the basis of step 101, the obtained user instruction text is recognized to obtain a text recognition result. Specifically, the user instruction text can be recognized by rule-based, machine learning-based, and deep learning-based methods to obtain text recognition results. The time operation text contains clear time information, which is used to guide the device to perform an operation at a certain time point or time period. For example, the "5 minutes later" in "turn on the air conditioner after 5 minutes" may include delayed operation text, continuous operation text, etc.; the operation intention text contains instructions to guide the device to perform specific operations, such as "turn on seat massage", "open the car window", etc.; the operation intention text and the time operation text are related, and the above-mentioned relationship includes positional relationship, logical relationship, etc. For example, the time information in the time operation text can provide a time frame for the operation intention text to be executed.

[0070] Step 103: Determine at least one user operation instruction and execution time period information of at least one user operation instruction based on the text recognition result.

[0071] This step is to determine at least one user operation instruction and at least one execution time period information of the user operation instruction based on the text recognition result obtained in step 102. The user operation instruction refers to the text that the user wants the device to perform a certain operation, for example, "Turn on the music for me"; the execution time period information includes the time information of executing a specific operation and the corresponding instruction, which refers to an instruction text given by the user, for example, "Close the car window after half an hour".

[0072] As described above, using this method to identify user operation instructions can better identify user operation instructions that include delayed or continuous operations, thereby allowing users to preset operation content according to personal preferences and needs without distracting current attention, and enable the device to automatically perform future operations, thereby not only ensuring the user's driving safety, but also providing the user with a more personalized and intelligent driving experience.

[0073] In some embodiments, the user operation instruction recognition result can be further generated based on at least one user operation instruction and the execution time period information of at least one user operation instruction. Specifically, the execution time period information of the at least one user operation instruction is integrated to obtain the user operation instruction recognition result. The above instruction recognition result is for multiple user instruction texts, that is, it contains multiple execution time period information, and each execution time period information comes from a different user instruction text, for example, "Turn on the air conditioner for me in half an hour", "Close the car windows", "Turn on seat massage in one hour", "Play music for 5 minutes for 10 minutes".

[0074] In some embodiments, Figure 2for Figure 1 The specific execution flow diagram of step 102 of the embodiment shown in FIG. Figure 2 As shown, the following steps are included:

[0075] Step 201: Segment the user instruction text to obtain at least two sub-instruction texts included in the user instruction text, and the positions of the sub-instruction texts in the user instruction text;

[0076] This step is to segment the user instruction text using a corresponding segmentation method after acquiring the user instruction text in step 101, so as to obtain at least two sub-instruction texts included in the user instruction text, and the positions of the sub-instruction texts in the user instruction text. The at least two sub-instruction texts may be an operation intention text and a time operation text; the segmentation methods include rule-based methods, machine learning-based methods, and deep learning-based methods.

[0077] In the embodiment of the present application, a method based on deep learning can be used to segment the user instruction text. Specifically, the user instruction text can be processed according to a pre-trained deep learning segmentation model to achieve the purpose of segmenting it. By applying the above deep learning segmentation model to segment the user instruction text, the user instruction text can be processed more efficiently and an accurate segmentation effect can be achieved.

[0078] In an embodiment of the present application, the above-mentioned deep learning sentence segmentation model includes at least two of a bidirectional encoder representation method, a convolutional neural network, and a conditional random field algorithm. Specifically, the deep learning sentence segmentation model can be obtained by any combination of at least two of a bidirectional encoder representation method, a convolutional neural network, and a conditional random field algorithm, for example, a bidirectional encoder representation method combined with a conditional random field algorithm, a convolutional neural network combined with a conditional random field algorithm, and a bidirectional encoder representation method combined with a convolutional neural network and a conditional random field algorithm. The bidirectional encoder representation method (Bidirectional Encoder Representations from Transformers, referred to as BERT) is a pre-trained language representation model, which can simultaneously consider the text information before and after the current word through bidirectional training, so as to more accurately understand the context of the language.

[0079] Convolutional Neural Network (CNN) is a type of feedforward neural network that includes convolution calculations and has a deep structure. It can automatically learn the deep features of input data based on its own multi-layer network structure. Networks at different levels can learn features at different levels.

[0080] The Conditional Random Field (CRF) algorithm is a discriminative probability model, a type of random field, commonly used to annotate or analyze sequence data, such as natural language text or biological sequences. Specifically, given a set of input variables, such as a sequence of words in a sentence, the model will learn the conditional probability distribution between these input variables and output variables, such as the part-of-speech tag of each word.

[0081] In the embodiment of the present application, the process of using the above-mentioned deep learning sentence segmentation model to segment the user instruction text is as follows: first use BERT to encode the above-mentioned user instruction text, that is, through the multi-layer neural network architecture of BERT, extract the deep-level instruction text features, these features can fully reflect the semantic information and contextual relationship in the text, and provide a solid foundation for the subsequent sentence segmentation processing. Then, the CNN layer uses the instruction text features output by BERT as input, and uses its convolution kernel to slide on the feature map to further extract and integrate sequence features, which help the model to understand the punctuation points in the text more accurately. Finally, the CRF layer decodes the output of CNN, optimizes the annotation sequence by considering the dependency between the labels, and outputs the final sentence segmentation processing result of the instruction text. The above-mentioned instruction text features also include the position information of each word in the user instruction text. By pre-training the deep learning sentence segmentation model to segment the user instruction text, the respective advantages of BERT, CNN and CRF can be fully utilized, not only can the accurate sentence segmentation of the user instruction text be achieved, but also the coherence and readability of the sentence segmentation processing result can be ensured.

[0082] Step 202: acquiring at least one time operation text and at least one operation intention text based on at least two sub-instruction texts;

[0083] This step is to further process the at least two sub-instruction texts obtained on the basis of step 201 to obtain at least one time operation text and at least one operation intention text. Specifically, the text type of the at least two sub-instruction texts is identified to obtain at least one operation intention text and at least one time operation text, that is, to clarify whether the above sub-instruction text belongs to the operation intention text or the time operation text type.

[0084] In an embodiment of the present application, the above-mentioned text type identification method includes a method based on a rule engine template. Specifically, the rule engine template can be used to identify the text type of at least two sub-instruction texts. When the matching degree of a sub-instruction text with a rule engine template reaches a preset threshold, it is considered that the sub-instruction text belongs to the instruction text type corresponding to the template, and a type label is given to it. For example, when the matching degree of the sub-instruction text "one hour later" and its corresponding rule engine template reaches a preset threshold, it is considered that the sub-instruction text belongs to the delayed operation text class in the time operation text type. The above-mentioned rule engine template is pre-set, which is a rule template summarized by manually analyzing each representative instruction text. For example, by analyzing the instruction text: 10 minutes later, 5 minutes later, 2 hours later, the rule engine template that can be summarized is [number] {seconds|minutes|hours|hours} [after], where [] represents an entity type or part of speech, {} represents a keyword, and | represents or; similarly, by analyzing the instruction text: turn on seat massage, turn on the air conditioner for me, close the window, the rule engine template that can be summarized is.*? {Open|On|Off}[Device Name], where .*? represents any character, [] represents entity type or part of speech, {} represents keyword, and | represents or. Using the rule engine template to perform text type recognition on at least two sub-command texts can effectively improve the accuracy and efficiency of text recognition.

[0085] Step 203: Determine the association relationship between the operation intention text and the time operation text based on the position of the sub-command text in the user command text.

[0086] This step is based on the position of the sub-instruction text obtained in step 201 in the user instruction text, and the operation intention text and time operation text obtained in step 202, to further determine the association between the operation intention and the time operation text.

[0087] Specifically, Figure 3 for Figure 2 The specific execution flow diagram of step 203 of the embodiment shown in FIG. Figure 3 As shown, the following steps are included:

[0088] Step 301: based on the position of the sub-command text in the user command text, obtain the position of the operation intention text in the user command text;

[0089] This step is to obtain the position of the sub-instruction text in the user instruction text and determine which type of text the sub-instruction text belongs to, that is, the sub-instruction text belongs to the operation intention text or the time operation text, and then obtain the position information of the operation intention text in the user instruction text to determine the position of the operation intention text in the user instruction text for subsequent processing.

[0090] Step 302: Determine the association between the operation intention text and the time operation text based on the position of the operation intention text in the user instruction text.

[0091] On the basis of step 301, the association between the operation intention and the time operation text is further determined according to the specific position of the operation intention text in the user instruction text. In this way, the logical relationship, positional relationship, etc. between the operation intention and the time element can be clarified, and more accurate information can be provided for subsequent processing. By accurately locating the operation intention text and the time operation text, the complete intention of the user instruction can be better understood, thereby ensuring that the system can perform the corresponding operation according to the user's instruction.

[0092] In the embodiment of the present application, after identifying the text types of at least two sub-command texts and obtaining the positions of the operation intention text and the time operation text in the user command text, the above sub-command texts need to be structured, that is, the format of the sub-command texts is unified, and the text type labels and the positions in the user command texts are recorded. Such processing can more efficiently manage and utilize the sub-command texts, and helps to quickly locate the specific positions of the sub-command texts in the user command text in subsequent processing.

[0093] In some embodiments, the execution time period information of the user operation instruction corresponding to the operation intention text can be determined based on the text recognition result. Specifically, Figure 4 for Figure 1 The specific execution flow diagram of step 103 of the embodiment shown in FIG. Figure 3 As shown, the following steps are included:

[0094] Step 401: determining at least one user operation instruction based on at least one operation intention text;

[0095] This step is to identify which sub-instruction texts belong to the operation intention text, and then determine at least one user operation instruction contained therein based on the above operation intention instruction text. For example, "Turn on the air conditioner" in the operation intention text "Turn on the air conditioner for me" is a user operation instruction.

[0096] In an embodiment of the present application, other algorithms or models in the system that can infer or predict text semantics can be called to process the above-mentioned operation intention text to determine at least one user operation instruction contained therein. Specifically, these operation intention texts are first input into the corresponding algorithms or models, which will then make in-depth predictions or inferences on the input texts based on their internal logic and algorithms, and finally output the results of their predictions or inferences, and then determine the user operation instructions contained in the operation intention instruction text based on these results, thereby laying the foundation for the subsequent instruction execution link.

[0097] Step 402: determining at least one operation time period on the time axis and the type of at least one operation time period based on at least one time operation text;

[0098] This step is to identify which sub-instruction texts belong to time operation texts, and then determine at least one operation time period on the time axis based on these identified time operation texts. At the same time, according to the text type label of the recorded sub-instruction text, further confirm the type of the above-mentioned at least one operation time period. The above-mentioned operation time period is clearly defined on the time axis and is used to guide or perform specific operations related to time, such as "5 minutes" in "turn on the air conditioner after 5 minutes"; the types of the above-mentioned operation time period include delay type and continuous type. The delay type refers to a time period containing delayed operations, that is, the operation is not executed immediately, but needs to be started after a specified time delay. The above-mentioned delayed operation can be clarified by time indicators such as "start after 5 minutes" or "execute after 10 seconds"; the continuous type refers to a time period containing continuous operations. In this type, the operation will continue for a specific time period, rather than starting and completing instantly after the delay like the delay type. For example, a continuous type of operation time period may be "play music for 30 minutes", which means that the music will continue to play for the next 30 minutes.

[0099] In the embodiment of the present application, a time operation text can determine one or more operation time periods, and the above operation time periods can be classified as either a delay type or a continuous type. For example, the "one hour" in "play music for 10 minutes after one hour" is a delay type operation time period, which indicates the start time of playing music; and "10 minutes" is a continuous type operation time period, which clarifies the duration of playing music. This setting enables the system to flexibly and accurately understand and process complex time requirements in user instructions, thereby ensuring that operations can be performed at the time and in the manner expected by the user.

[0100] Step 403: determining an index position of the user operation instruction in at least one operation time period based on the association relationship between the operation intention text and the time operation text;

[0101] After determining the association between the operation intention text and the time operation text in the above step 302, further determine the index position of the user operation instruction in at least one operation time period based on the logical relationship, positional relationship, etc. between the operation intention text and the time operation text. The index position is the precise positioning of the user operation instruction on the time axis, which can be a specific time point or a starting or ending position within a time period. The above index position can ensure that the operation can be performed accurately according to the time and order expected by the user. For example, in "half an hour later, turn on the air conditioner for me, close the windows, and one hour later, turn on the seat massage", the index positions of "half an hour later" and "turn on the air conditioner for me" are 0 and 1 respectively.

[0102] Step 404: Determine the execution time period information of each user operation instruction based on the index position of the user operation instruction in at least one operation time period and the type of the operation time period.

[0103] This step is based on step 403 and determines the execution time period information of each user operation instruction according to the index position of the user operation instruction in at least one operation time period and the type of the operation time period.

[0104] In some embodiments, according to the index position of the user operation instruction in at least one operation time period and the type of the operation time period, the manner of determining the execution time period information of each user operation instruction includes the following two methods:

[0105] First, if the user operation instruction is at the end position of the delay type operation time period, the execution time period of the user operation instruction is at least one operation time period after the end position. The above end position refers to the moment when the operation time period is about to end. Specifically, if the index position of the user operation instruction is immediately after the index position of the delay type operation time period, the execution time period of the user operation instruction is at least one operation time period between the end moment of the delay type operation time period and the start of the next delay type operation time period. For example, the user operation instruction "turn on the air conditioner" in "turn on the air conditioner after 5 minutes, start seat massage after 10 minutes" immediately follows the end of the delay type operation time period "after 5 minutes", then the execution time period of "turn on the air conditioner" is the operation time period between "after 5 minutes" and "after 10 minutes".

[0106] Second, if the user operation instruction is at the end position of the continuous type operation time period, the execution time period of the user operation instruction is the operation time period before the end position. Specifically, if the user operation instruction is at an index position before the index position of the continuous type operation time period, the execution time period of the user operation instruction is an operation time period before the end position of the continuous type operation time period. For example, if "play music for 10 minutes after one hour" is at the end position of "10 minutes", the execution time period of the user operation instruction "play music" is "10 minutes".

[0107] The present application provides a device control method, wherein the device may be a vehicle, specifically, Figure 5 FIG. 1 is a flow chart of a device control method in an embodiment of the present application. The device control method can be applied to a vehicle, such as Figure 5 As shown, the method comprises the following steps:

[0108] Step 501: Based on Figure 1-Figure 4 The user operation instruction recognition method described in any one of the embodiments obtains a user operation instruction recognition result;

[0109] Step 502: Control the device based on the user operation instruction recognition result.

[0110] This step is after obtaining the user operation instruction recognition result in step 501, using the operation time information and user operation instruction information contained in the multiple execution time period information in the user operation instruction recognition result, so as to control the vehicle to complete these user operation instructions in sequence.

[0111] As mentioned above, in Figure 1-Figure 5 In the described embodiment, user operation instructions are identified by a method that cooperates with deep learning and a rule engine, so that user operation instructions containing delayed or continuous operations can be better identified, and the efficiency and accuracy of instruction identification can be improved. Therefore, when the device is controlled based on the user operation instruction identification results obtained by the user operation instruction identification method, it can respond to the user's intention more accurately and optimize the operating efficiency of the device. While providing convenient operation, it also ensures a high level of security and intelligence, thereby improving the overall user experience.

[0112] With the above Figure 1-Figure 4 Corresponding to any of the provided user operation instruction recognition methods, the embodiments of the present application provide a corresponding device. Figure 6 This is a schematic diagram of the structure of a user operation instruction recognition device in an example of this application, such as Figure 6As shown, the user operation instruction recognition device includes a first data acquisition module 11, a data recognition module 12, and an information determination module 13, wherein the first data acquisition module 11 is used to obtain user instruction text; the data recognition module 12 is used to recognize the user instruction text to obtain a text recognition result, and the recognition result includes at least one time operation text, at least one operation intention text, and an association between the operation intention text and the time operation text; the information determination module 13 is used to determine at least one user operation instruction and the execution time period information of at least one user operation instruction based on the text recognition result.

[0113] In some embodiments, it also includes:

[0114] A user operation instruction recognition result is generated based on at least one user operation instruction and execution time period information of at least one user operation instruction.

[0115] In some embodiments, the above-mentioned performing text recognition on the user instruction text to obtain the text recognition result includes:

[0116] Segmenting the user instruction text to obtain at least two sub-instruction texts included in the user instruction text, and positions of the sub-instruction texts in the user instruction text;

[0117] Acquire at least one time operation text and at least one operation intention text based on at least two sub-instruction texts;

[0118] Based on the position of the sub-instruction text in the user instruction text, the association relationship between the operation intention text and the time operation text is determined.

[0119] In some embodiments, the acquiring of at least one time operation text and at least one operation intention text based on at least two sub-instruction texts includes:

[0120] Text type recognition is performed on at least two sub-instruction texts to obtain at least one time operation text and at least one operation intention text.

[0121] In some embodiments, the acquiring of at least one time operation text and at least one operation intention text based on at least two sub-instruction texts includes:

[0122] Text type recognition is performed on at least two sub-instruction texts to obtain at least one time operation text and at least one operation intention text.

[0123] In some embodiments, the above-mentioned sentence segmentation processing of the user instruction text includes:

[0124] The user instruction text is segmented through the deep learning segmentation model.

[0125] In some embodiments, the deep learning segmentation model includes at least two of a bidirectional encoder representation method, a convolutional neural network, and a conditional random field algorithm.

[0126] In some embodiments, the text type identification of at least two sub-instruction files includes:

[0127] Perform text type identification on at least two sub-instruction files based on the rule engine template.

[0128] In some embodiments, the determining of at least one user operation instruction and execution time period information of at least one user operation instruction based on the text recognition result includes:

[0129] Determine at least one user operation instruction based on at least one operation intention text;

[0130] Determine at least one operation time period on the time axis and the type of at least one operation time period based on at least one time operation text;

[0131] Based on the association relationship between the operation intention text and the time operation text, determine the index position of the user operation instruction in at least one operation time period;

[0132] Based on the index position of the user operation instruction in at least one operation time period and the type of the operation time period, the execution time period information of each user operation instruction is determined.

[0133] In some embodiments, the type of the above-mentioned operation time period includes a delay type or a continuous type.

[0134] In some embodiments, the above-mentioned determining the execution time period information of each user operation instruction based on the index position of the user operation instruction in at least one operation time period and the type of the operation time period includes:

[0135] If the user operation instruction is at the end position of the delay type operation time period, the execution time period of the user operation instruction is at least one operation time period after the end position; or,

[0136] If the user operation instruction is at the end position of the continuous type operation time period, the execution time period of the user operation instruction is the operation time period before the end position.

[0137] As described above, using the above-mentioned user operation instruction recognition device to recognize user operation instructions can allow users to preset operation content according to personal preferences and needs without distracting current attention, and enable the device to automatically perform future operations, thereby not only ensuring the user's driving safety, but also providing the user with a more personalized and intelligent driving experience.

[0138] With the above Figure 5 Corresponding to the provided device control method, the embodiments of the present application provide a corresponding apparatus. Figure 7 This is a schematic diagram of the structure of any device provided in the examples of this application, such as Figure 7 As shown, any of the provided devices includes a second data acquisition module 21 and a device control module 22, wherein the second data acquisition module 21 is used to obtain a user operation instruction recognition result through a user operation instruction recognition method; and the device control module 22 is used to control the device based on the user operation instruction recognition result.

[0139] In some embodiments, the above-mentioned device is a vehicle, and specifically can be used to control various electronic devices on the vehicle.

[0140] As described above, using the device control device to control the above-mentioned electronic equipment of the vehicle can more accurately respond to the user's intentions, optimize the operating efficiency of the equipment, and while providing convenient operation, it also ensures a high level of safety and intelligence, thereby improving the overall user experience.

[0141] The present application also provides an electronic device, such as Figure 8 As shown, the electronic device includes: a processor 31 and a memory 32, the memory 32 stores programs or instructions that can be run on the processor, and the above-mentioned programs or instructions are executed by the processor to achieve the following Figure 1-Figure 4 The steps of the user operation instruction recognition method; and / or, when the program or instruction is executed by the processor, the following is implemented Figure 5 The steps of the device control method.

[0142] The present application also provides a vehicle, such as Fig. 9 As shown, the vehicle includes Figure 8 The electronic device 41, wherein the memory of the electronic device stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the following is achieved: Figure 1-Figure 4 The steps of the user operation instruction recognition method; and / or, when the program or instruction is executed by the processor, the following is implemented Figure 5 The steps of the device control method.

[0143] The present application also provides a computer-readable storage medium, wherein a program or instruction is stored on the computer-readable storage medium, and when the program or instruction is executed by a processor, the following is implemented: Figure 1-Figure 4 The steps of the user operation instruction recognition method; and / or, when the program or instruction is executed by the processor, the following is implemented Figure 5 The steps of the device control method.

[0144] The present application also provides a computer program product, which, when executed by a processor of a vehicle, implements the following Figure 1-Figure 4 The steps of the user operation instruction recognition method; and / or, when the program or instruction is executed by the processor, the following is implemented Figure 5 The steps of the device control method.

[0145] It should be noted that, in this article, the terms "comprise", "include" or any other variant thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, an element defined by the sentence "comprises one..." does not exclude the presence of other identical elements in the process, method, article or device including the element. In addition, it should be pointed out that the scope of the method and device in the embodiment of the present application is not limited to performing functions in the order shown or discussed, and may also include performing functions in a substantially simultaneous manner or in reverse order according to the functions involved, for example, the described method may be performed in an order different from that described, and various steps may also be added, omitted or combined. In addition, the features described with reference to certain examples may be combined in other examples.

[0146] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of a computer software product plus a necessary general hardware platform, and of course, can also be implemented by hardware. The computer software product is stored in a storage medium (such as ROM, RAM, disk, CD, etc.), including several instructions to enable a terminal or a network-side device to execute the methods described in each embodiment of the present application.

[0147] The embodiments of the present application are described above in conjunction with the accompanying drawings, but the present application is not limited to the above-mentioned specific implementation methods. The above-mentioned specific implementation methods are merely illustrative and not restrictive. Under the guidance of the present application, ordinary technicians in this field can also make many forms of implementation methods without departing from the purpose of the present application and the scope of protection of the claims, and these implementation methods are all within the protection of the present application.

Claims

1. A method for identifying a user operation instruction, characterized in that: include: Get the user command text; Performing text recognition on the user instruction text to obtain a text recognition result, wherein the text recognition result includes at least one time operation text, at least one operation intention text, and an association relationship between the operation intention text and the time operation text; Based on the text recognition result, at least one user operation instruction and execution time period information of the at least one user operation instruction are determined.

2. The method according to claim 1, characterized in that Also includes: A user operation instruction recognition result is generated based on the at least one user operation instruction and the execution time period information of the at least one user operation instruction.

3. The method according to claim 1, characterized in that The performing text recognition on the user instruction text to obtain a text recognition result includes: Performing sentence segmentation processing on the user instruction text to obtain at least two sub-instruction texts included in the user instruction text, and positions of the sub-instruction texts in the user instruction text; Acquire at least one time operation text and at least one operation intention text based on the at least two sub-instruction texts; Based on the position of the sub-instruction text in the user instruction text, the association relationship between the operation intention text and the time operation text is determined.

4. The method according to claim 3, characterized in that The acquiring at least one time operation text and at least one operation intention text based on the at least two sub-instruction texts includes: The text type of the at least two sub-instruction texts is identified to obtain the at least one operation intention text and the at least one time operation text.

5. The method according to claim 4, characterized in that The determining, based on the position of the sub-instruction text in the user instruction text, the association relationship between the operation intention text and the time operation text includes: Based on the position of the sub-instruction text in the user instruction text, obtaining the position of the operation intention text in the user instruction text; Based on the position of the operation intention text in the user instruction text, the association relationship between the operation intention text and the time operation text is determined.

6. The method according to claim 5, characterized in that The sentence segmentation processing of the user instruction text includes: The user instruction text is segmented using a deep learning segmentation model.

7. The method according to claim 6, characterized in that The deep learning sentence segmentation model includes at least two of a bidirectional encoder representation method, a convolutional neural network, and a conditional random field algorithm.

8. The method according to claim 5, characterized in that The performing text type identification on the at least two sub-instruction files includes: The text type of the at least two sub-instruction files is identified based on the rule engine template.

9. The method according to any one of claims 1 to 8, characterized in that: The determining, based on the text recognition result, at least one user operation instruction and execution time period information of the at least one user operation instruction includes: Determine at least one user operation instruction based on the at least one operation intention text; Determine at least one operation time period on the time axis and the type of the at least one operation time period based on the at least one time operation text; Based on the association relationship between the operation intention text and the time operation text, determining the index position of the user operation instruction in the at least one operation time period; Based on the index position of the user operation instruction on the at least one operation time period and the type of the operation time period, the execution time period information of each user operation instruction is determined.

10. The method according to claim 9, characterized in that The type of the operation time period includes a delay type or a continuous type.

11. The method according to claim 10, characterized in that The determining the execution time period information of each user operation instruction based on the index position of the user operation instruction in the at least one operation time period and the type of the operation time period includes: If the user operation instruction is at the end position of the delay type operation time period, the execution time period of the user operation instruction is at least one operation time period after the end position; or, If the user operation instruction is at the end position of a continuous type operation time period, the execution time period of the user operation instruction is the operation time period before the end position.

12. A device control method, characterized in that: include: Obtaining a user operation instruction recognition result by using the user operation instruction method described in any one of claims 2 to 11; Device control is performed based on the user operation instruction recognition result.

13. The method according to claim 12, characterized in that The device is a vehicle.

14. A user operation instruction recognition device, characterized in that: include: A first data acquisition module, used to acquire the user instruction text; A data recognition module, used for recognizing the user instruction text to obtain the text recognition result, wherein the recognition result includes at least one time operation text, at least one operation intention text, and an association relationship between the operation intention text and the time operation text; The information determination module is used to determine at least one user operation instruction and execution time period information of the at least one user operation instruction based on the text recognition result.

15. A device control device, characterized in that: include: A second data acquisition module, used to obtain a user operation instruction recognition result through the user operation instruction recognition method described in any one of claims 2 to 11; The device control module is used to control the device based on the recognition result of the user operation instruction.

16. An electronic device, characterized in that: include: A processor and a memory, wherein the memory stores a program or instruction that can be run on the processor, and when the program or instruction is executed by the processor, the steps of the user operation instruction recognition method as described in any one of claims 1 to 11 are implemented; and / or when the program or instruction is executed by the processor, the steps of the device control method as described in claim 12 or 13 are implemented.

17. A vehicle, characterized in that: An electronic device comprising the electronic device of claim 16, wherein the memory of the electronic device stores a program or instruction that can be run on a processor, and the program or instruction, when executed by the processor, implements the steps of the user operation instruction recognition method as described in any one of claims 1 to 11; and / or, the program or instruction, when executed by the processor, implements the steps of the device control method as described in claim 12 or 13.

18. A computer-readable storage medium, characterized in that: The readable storage medium stores programs or instructions, which, when executed by a processor, implement the steps of the user operation instruction identification method as described in any one of claims 1 to 11; and / or, when executed by a processor, implement the steps of the device control method as described in claim 12 or 13.

Images