Voice instruction recognition method and device, storage medium and electronic equipment

Abstract

The invention provides a voice instruction recognition method and device, a storage medium and electronic equipment, and the method comprises the steps: obtaining a voice instruction library, and obtaining user voice in real time, a semantic coding mode of the voice instruction library being a target coding mode, and the user voice being voice for directly controlling target equipment to execute target operation; converting the user voice into a corresponding target semantic vector by adopting a target coding mode, and matching the target semantic vector with a basic semantic vector in a voice instruction library by adopting a semantic matching model to obtain a target instruction corresponding to the basic semantic vector which is the same as the target semantic vector; and controlling the target device to execute the target instruction to obtain an execution result, and adaptively updating the voice instruction library and the semantic matching model according to the execution result and the user voice. According to the method, the target of natural voice interaction without wakeup words in a home scene can be achieved, the user experience is further improved, and the development of the smart home technology is promoted.

Description

Technical Field

[0001] The present application relates to the field of intelligent voice interaction technology, and in particular to a method for recognizing voice commands, an intelligent device for recognizing voice commands, a computer-readable storage medium, and an electronic device. Background Art

[0002] With the development of intelligent voice interaction technology, voice control has gradually been applied to home scenes. However, the voice interaction method in the existing technology requires the user to say a preset "wake-up word" first, which increases the complexity of interaction, is affected by environmental noise, and cannot be personalized. Summary of the invention

[0003] The main purpose of the present application is to provide a method for recognizing voice commands, an intelligent device for recognizing voice commands, a computer-readable storage medium and an electronic device, so as to at least solve the problems in the prior art that the voice interaction method requires the user to say a preset wake-up word before interaction can be achieved, resulting in high interaction complexity, great influence of environmental noise, and inability to be personalized.

[0004] In order to achieve the above-mentioned purpose, according to one aspect of the present application, a method for recognizing voice commands is provided, comprising: obtaining a voice command library and obtaining user voice in real time, the semantic encoding method of the voice command library being a target encoding method, and the user voice being a voice that directly controls a target device to perform a target operation; using the target encoding method, converting the user voice into a corresponding target semantic vector, and using a semantic matching model to match the target semantic vector with a basic semantic vector in the voice command library to obtain a target command corresponding to the basic semantic vector that is the same as the target semantic vector; controlling the target device to execute the target command to obtain an execution result, and adaptively updating the voice command library and the semantic matching model according to the execution result and the user voice.

[0005] Optionally, the target encoding method is used to convert the user voice into a corresponding target semantic vector, including: converting the user voice into spoken text, and using natural language processing technology to extract characteristic semantic elements in the spoken text, the characteristic semantic elements including characteristic characters and / or characteristic words; obtaining the user's target image information and target sensor information; using the target encoding method, encoding the characteristic semantic elements based on the target image information and / or the target sensor information to obtain the target semantic vector corresponding to the characteristic semantic elements.

[0006] Optionally, obtaining the target image information of the user includes: using an image acquisition device to acquire user image data of the user, the user image data including at least facial information and behavioral information of the user; using image recognition technology to identify the user image data to obtain the facial data and behavioral data of the user, and determining the facial data and behavioral data of the user as the target image information of the user.

[0007] Optionally, obtaining the target sensor information of the user includes: receiving multiple sensor detections to obtain corresponding sub-sensor information, the multiple sensors are respectively a temperature sensor, a humidity sensor, and a human position sensor, the temperature sensor is used to detect the ambient temperature of the user's environment, the humidity sensor is used to detect the ambient humidity of the user's environment, and the human position sensor is used to detect the user's location information; all of the sub-sensor information is determined as target sensor information.

[0008] Optionally, the target encoding method is adopted to encode the characteristic semantic element based on the target image information and / or the target sensor information to obtain the target semantic vector corresponding to the characteristic semantic element, including: obtaining multiple groups of historical data, one group of historical data at least includes the historical voice data of the user at a historical moment and the execution record, historical environment information and historical execution time of the target device corresponding to the historical voice data; adopting the target encoding method to encode the characteristic semantic element based on at least one of the multiple groups of historical data, the target image information and the target sensor information to obtain the target semantic vector corresponding to the characteristic semantic element.

[0009] Optionally, a semantic matching model is used to match the target semantic vector with the basic semantic vector in the voice instruction library to obtain a target instruction corresponding to the basic semantic vector that is the same as the target semantic vector, including: using a matching algorithm to match the target semantic vector with the basic semantic vector in the voice instruction library to obtain multiple initial instructions corresponding to the basic semantic vector that is the same as the target semantic vector; determining the accuracy of each of the initial instructions based on multiple groups of historical data and current environmental information, and determining the initial instruction with the highest accuracy as the preliminary instruction; sending a confirmation message to the user, and receiving feedback information sent by the user based on the confirmation message, the confirmation message is confirmation information to the user to confirm whether the preliminary instruction is the target instruction; when the feedback information is to confirm that the preliminary instruction is the target instruction, the preliminary instruction is determined as the target instruction.

[0010] Optionally, obtaining a voice command library includes: obtaining historical voice data of users at multiple historical moments; classifying all the historical voice data based on the execution device, execution function and execution scenario corresponding to each of the historical voice data to obtain multiple groups of voice data sets, one group of the voice data sets corresponding to an execution device, an execution function and an execution scenario; removing redundant vocabulary from all the historical voice data in each group of the voice data sets to obtain the voice command library.

[0011] According to another aspect of the present application, there is provided an intelligent recognition device for voice commands, comprising: an acquisition unit, for acquiring a voice command library and acquiring user voice in real time, wherein the semantic encoding method of the voice command library is a target encoding method, and the user voice is a voice that directly controls a target device to perform a target operation; a first processing unit, for converting the user voice into a corresponding target semantic vector using the target encoding method, and matching the target semantic vector with a basic semantic vector in the voice command library using a semantic matching model to obtain a target instruction corresponding to the basic semantic vector that is the same as the target semantic vector; and a second processing unit, for controlling the target device to execute the target instruction, obtaining an execution result, and adaptively updating the voice command library and the semantic matching model according to the execution result and the user voice.

[0012] According to another aspect of the present application, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute any one of the voice instruction recognition methods.

[0013] According to another aspect of the present application, an electronic device is provided, comprising: one or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a recognition method for executing any one of the voice instructions.

[0014] Applying the technical solution of the present application, the above-mentioned voice command recognition method first obtains a voice command library and obtains user voice in real time. The semantic encoding method of the voice command library is a target encoding method, and the user voice is a voice that directly controls the target device to perform the target operation; then the target encoding method is used to convert the user voice into a corresponding target semantic vector, and the semantic matching model is used to match the target semantic vector with the basic semantic vector in the voice command library to obtain a target instruction corresponding to the basic semantic vector that is the same as the target semantic vector; finally, the target device is controlled to execute the target instruction to obtain the execution result, and the voice command library and the semantic matching model are adaptively updated according to the execution result and the user voice. This method can achieve the goal of naturally and efficiently completing voice control and interaction without a wake-up word by continuously monitoring and quickly matching and identifying the voice with the pre-built command semantics, combining context information and multimodal fusion technology, greatly improving the user experience, promoting the development and popularization of smart home technology, and solving the problem that the voice interaction method in the prior art requires the user to say a preset wake-up word before interaction can be achieved, resulting in high interaction complexity, greater impact from environmental noise, and inability to be personalized. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] The drawings constituting part of the present application are used to provide a further understanding of the present application. The exemplary embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0016] Figure 1 A hardware structure block diagram of a mobile terminal for executing a method for recognizing a voice command provided in an embodiment of the present application is shown;

[0017] Figure 2 A flow chart of a method for recognizing a voice command according to an embodiment of the present application is shown;

[0018] Figure 3 A schematic diagram showing a flow chart of another method for recognizing a voice command provided according to an embodiment of the present application;

[0019] Figure 4 A structural block diagram of an intelligent voice command recognition device provided according to an embodiment of the present application is shown.

[0020] The above drawings include the following reference numerals:

[0021] 102, processor; 104, memory; 106, transmission device; 108, input and output devices. DETAILED DESCRIPTION

[0022] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0023] In order to enable those skilled in the art to better understand the solution of the present application, the technical solution in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present application.

[0024] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present application described here. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0025] As introduced in the background technology, the mainstream voice interaction method requires the user to say a preset "wake-up word" first, which has the disadvantages of increasing the complexity of interaction, being affected by environmental noise, and being unable to be personalized. The continuous monitoring mode proposed in the prior art is difficult to promote due to energy consumption and privacy protection issues. In order to solve the problems in the prior art that the voice interaction method requires the user to say a preset wake-up word before interaction can be achieved, resulting in high interaction complexity, being greatly affected by environmental noise, and being unable to be personalized, the embodiments of the present application provide a voice command recognition method, a voice command intelligent recognition device, a computer-readable storage medium, and an electronic device.

[0026] The technical solutions in the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0027] The method embodiments provided in the embodiments of the present application can be executed in a mobile terminal, a computer terminal or a similar computing device. Taking running on a mobile terminal as an example, Figure 1 FIG. 1 is a hardware structure block diagram of a mobile terminal of a method for recognizing a voice command according to an embodiment of the present invention. Figure 1 As shown, the mobile terminal may include one or more ( Figure 1Only one is shown in the figure) a processor 102 (the processor 102 may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the mobile terminal may also include a transmission device 106 and an input / output device 108 for communication functions. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the mobile terminal. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations are shown.

[0028] The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as the computer program corresponding to the voice command recognition method in the embodiment of the present invention. The processor 102 executes various functional applications and data processing by running the computer program stored in the memory 104, that is, the above method is implemented. The memory 104 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some examples, the memory 104 may further include a memory remotely arranged relative to the processor 102, and these remote memories can be connected to the mobile terminal via a network. Examples of the above-mentioned network include but are not limited to the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof. The transmission device 106 is used to receive or send data via a network. The above-mentioned specific examples of the network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, referred to as NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0029] In this embodiment, a method for recognizing voice commands running on a mobile terminal, a computer terminal or a similar computing device is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0030] Figure 2 FIG. 1 is a flow chart of a method for recognizing a voice command according to an embodiment of the present application. Figure 2 As shown, the method comprises the following steps:

[0031] Step S201, obtaining a voice instruction library and obtaining user voice in real time, wherein the semantic encoding method of the voice instruction library is a target encoding method, and the user voice is a voice that directly controls the target device to perform a target operation;

[0032] Specifically, a pre-built voice command library commonly used in home scenarios can quickly and accurately identify the user's real-time voice command intentions. Through data analysis and semantic modeling technology, a voice command library commonly used in home scenarios is constructed. Each command contains a command semantic vector representation and the corresponding execution action or response content. The command semantic representation can be obtained through a pre-trained language model or other semantic representation learning methods. The voice command library can be constructed manually or automatically generated based on data and continuously updated.

[0033] Wherein, obtaining the voice command library includes the following steps:

[0034] Step S2011, obtaining historical voice data of the user at multiple historical moments;

[0035] Step S2012, classifying all the above historical voice data based on the execution device, execution function and execution scenario corresponding to each of the above historical voice data to obtain multiple groups of voice data sets, one group of the above voice data sets corresponding to one execution device, one execution function and one execution scenario;

[0036] Step S2013, removing all redundant words of the above historical voice data in each group of the above voice data sets to obtain the above voice instruction library.

[0037] Specifically, building a voice command library includes three steps: data collection, command classification, and command optimization. First, for specific home scenarios, a wide range of possible voice commands are collected. Some options include: investigating the commonly used commands of existing smart home products; conducting user interviews and questionnaires to understand the voice commands that users are accustomed to using in different scenarios; analyzing relevant smart home forums and communities to extract high-frequency commands that appear in user discussions. In addition to basic control commands, some complex scene commands should also be considered, such as "dim the lights and close the curtains while I watch a movie."

[0038] The collected instructions are classified according to different home appliances, functions and scenarios. For example: lighting control includes "turn on the light", "dim the light", "turn off the bedroom light", etc.; curtain control includes "open the curtains", "draw the curtains", "curtains half open", etc.; temperature control includes "raise the temperature", "lower the temperature", "set to 25 degrees", etc.; appliance control includes "turn on the TV", "turn off the stereo", "start the washing machine", etc. Classification can adopt a hierarchical structure to make subsequent semantic matching more efficient. In addition, optimizing each instruction and removing redundant words can make the instructions more concise and clear. For example, "please turn on the light" can be optimized to "turn on the light". Unifying the expression of instructions can effectively avoid recognition difficulties caused by multiple different statements for the same function.

[0039] Furthermore, a structured voice command library is constructed using database technology to store information such as the text content, category, and related parameters of each command, and an index is created for the command library for fast retrieval and matching.

[0040] Step S202, using the target encoding method, converting the user voice into a corresponding target semantic vector, and using a semantic matching model to match the target semantic vector with the basic semantic vector in the voice instruction library, to obtain a target instruction corresponding to the basic semantic vector that is the same as the target semantic vector;

[0041] Specifically, using an efficient semantic matching algorithm to analyze and understand multiple sets of historical data can improve the accuracy of instruction recognition and interaction efficiency. At the same time, database technology can be used to store the above historical data and establish corresponding indexes for rapid retrieval and analysis.

[0042] The target encoding method is used to convert the user speech into a corresponding target semantic vector, including the following steps:

[0043] Step S301, converting the user's voice into voice text, and extracting characteristic semantic elements from the voice text using natural language processing technology, wherein the characteristic semantic elements include characteristic characters and / or characteristic words;

[0044] Step S302, obtaining the user's target image information and target sensor information;

[0045] Step S303, using the target encoding method, based on the target image information and / or the target sensor information, encode the feature semantic elements to obtain the target semantic vector corresponding to the feature semantic elements.

[0046] Specifically, quickly matching the speech semantic vector with the instruction semantics in the instruction library can quickly identify the user's instruction intention and improve the user experience.

[0047] Among them, voice collection is required before encoding the characteristic semantic elements. Choose a suitable microphone device to ensure that the voice signal in the environment can be clearly collected. A microphone array can be used to improve the quality and directional sensitivity of voice collection. During the collection process, it is necessary to set a reasonable sampling frequency and quantization bit number to ensure the accuracy of the voice signal, and it is necessary to continuously monitor the sound in the environment. When voice input is detected, the subsequent processing flow is immediately started.

[0048] The collected speech signals need to be preprocessed, including noise reduction, echo removal, gain adjustment and other operations. We can use digital signal processing techniques, such as filters, adaptive algorithms, etc., to process the speech signals to improve the quality of the speech signals.

[0049] Speech recognition models based on deep learning, such as the combination of recurrent neural networks (RNN), convolutional neural networks (CNN) and long short-term memory networks (LSTM), recognize pre-processed speech signals and convert them into text versions, and train and optimize speech recognition models to improve recognition accuracy and speed. A large amount of annotated speech data can be used for training, and data enhancement, model fusion and other technologies can be used to improve model performance.

[0050] Wherein, obtaining the target image information of the user includes the following steps:

[0051] Step S3021, using an image acquisition device to acquire user image data of the user, wherein the user image data at least includes facial information and behavior information of the user;

[0052] Step S3022: Use image recognition technology to identify the user image data to obtain the user's facial data and behavior data, and determine the user's facial data and behavior data as the user's target image information.

[0053] Specifically, the visual information captured by the camera is combined for interaction to improve the accuracy and reliability of voice command understanding, thereby achieving a more natural human-computer interaction. In one embodiment, a user can point to an appliance with a gesture and then say "turn it on". The system can combine the gesture information and voice command to accurately identify the appliance that the user wants to turn on. When the system detects that the user enters a room, it can automatically adjust the voice interaction method and command library according to the layout of the room and the status of the equipment.

[0054] Among them, visual information fusion can be based on computational vision technology, such as target detection, gesture recognition, face recognition, etc., to extract key elements in visual information and integrate them with voice information.

[0055] Wherein, obtaining the user's target sensing information includes the following steps:

[0056] Step S3023, receiving corresponding sub-sensor information detected by multiple sensors, wherein the multiple sensors are respectively a temperature sensor, a humidity sensor, and a human position sensor, wherein the temperature sensor is used to detect the ambient temperature of the environment where the user is located, the humidity sensor is used to detect the ambient humidity of the environment where the user is located, and the human position sensor is used to detect the location information of the user;

[0057] Step S3024: determine all the above sub-sensing information as target sensing information.

[0058] Specifically, the use of sensor fusion technology, such as Kalman filtering, particle filtering, etc., and further integration of other modal information can effectively improve the accuracy and reliability of environmental perception, thereby improving the accuracy and stability of voice command understanding. In one embodiment, the information of the temperature sensor is received. When the temperature sensor detects that the indoor temperature is too high, the user says "turn on the air conditioner", and the system can automatically set the air conditioner to cooling mode and adjust it to a suitable temperature. In another embodiment, the information of the human position sensor is received. When the human sensor detects that the user leaves the room, the system can automatically turn off the lights and electrical appliances to save energy.

[0059] In addition, design the interface and process of multimodal interaction so that users can naturally switch and combine different interaction methods. For example, during voice interaction, visual feedback can be provided through indicator lights, screen displays, etc. to enhance the effect of interaction. Conduct user testing and evaluation on multimodal interaction, continuously optimize the interaction experience, and improve user satisfaction.

[0060] The target encoding method is adopted to encode the feature semantic elements based on the target image information and / or the target sensor information to obtain the target semantic vector corresponding to the feature semantic elements, including the following steps:

[0061] Step S3031, obtaining multiple groups of historical data, wherein one group of the historical data at least includes the historical voice data of the user at a historical moment and the execution record, historical environment information and historical execution time of the target device corresponding to the historical voice data;

[0062] Step S3032, adopting the above-mentioned target encoding method, based on the above-mentioned multiple groups of historical data, the above-mentioned target image information and at least one of the above-mentioned target sensor information, encodes the above-mentioned feature semantic elements to obtain the above-mentioned target semantic vector corresponding to the above-mentioned feature semantic elements.

[0063] Specifically, by continuously monitoring and collecting speech in the home environment, the acquired speech can be converted into a semantic vector representation in real time using the same semantic encoding method as the instruction library.

[0064] Furthermore, before semantic encoding, it is first necessary to perform semantic analysis on the text, that is, natural language processing techniques such as part-of-speech tagging, named entity recognition, syntactic analysis, etc. can be used for semantic analysis. In an optional scheme, for the input voice data "turn on the light in the living room", key information such as "turn on", "living room", and "light" can be extracted. The results of the semantic analysis are then encoded and converted into a form that is convenient for computer processing. For example, encoding can be performed using vector representation, semantic graphs, etc. The contextual information of the instruction and the needs of multimodal fusion should be considered during the encoding process to facilitate subsequent semantic matching and instruction recognition.

[0065] The method of using a semantic matching model to match the target semantic vector with the basic semantic vector in the voice instruction library to obtain a target instruction corresponding to the basic semantic vector that is the same as the target semantic vector comprises the following steps:

[0066] Step S401, using a matching algorithm to match the target semantic vector with the basic semantic vector in the voice instruction library, to obtain a plurality of initial instructions corresponding to the basic semantic vector that is the same as the target semantic vector;

[0067] Step S402, determining the accuracy of each of the initial instructions according to the plurality of sets of historical data and current environment information, and determining the initial instruction with the highest accuracy as the preliminary instruction;

[0068] Step S403, sending confirmation information to the user, and receiving feedback information sent by the user according to the confirmation information, wherein the confirmation information is confirmation information to the user to confirm whether the prepared instruction is the target instruction;

[0069] Step S404: when the feedback information confirms that the prepared instruction is the target instruction, determine the prepared instruction as the target instruction.

[0070] Specifically, the context information such as the recognized instructions, execution feedback, and related scene status is input into the context encoder to continuously update the speech semantic understanding model. By modeling historical interactions and environmental changes, the semantic representation can be adaptively adjusted to improve the ability to understand complex semantics.

[0071] A matching algorithm is used to quickly match the encoded semantics with the pre-built voice command library, and efficient retrieval algorithms such as hash tables and inverted indexes are used to improve the matching speed. For the matched commands, their accuracy and completeness are further confirmed. For example, check whether the parameters in the command are complete and whether they match the current environment.

[0072] If there are multiple possible matching results, ambiguity elimination is required. Ambiguity elimination can be performed in the following ways: combining contextual information, such as the user's historical operation records, current environmental status, etc., to determine the most likely instruction; asking questions to the user to confirm their intentions. For example, when "turn on the light" is identified but it is not sure which room the light is in, you can ask the user "Do you want to turn on the light in the living room or the bedroom?"; for some complex instructions, a step-by-step matching method can be used to gradually determine the specific content of the instruction.

[0073] After the final instruction is determined, the instruction is confirmed. The correctness of the instruction can be confirmed to the user through voice feedback, indicator lights, etc. If the user confirms that the instruction is correct, the system will perform the corresponding operation; if the user denies the instruction, the system will re-perform semantic matching and instruction recognition.

[0074] In addition, in order to improve the accuracy of command recognition and interaction efficiency, this application uses machine learning algorithms, such as decision trees, random forests, support vector machines, etc., to predict the user's next operation based on historical operation records. If the user turns on the lights in the living room for many consecutive times, the system can infer that the user may be active in the living room, and when the user issues a command again, the equipment in the living room can be given priority. In addition, the execution method of the command is adjusted in combination with the current environmental status, such as time, light intensity, temperature, etc. For example, when the light is dim at night, the user says "turn on the light", and the system can automatically brighten the light; when the light is sufficient during the day, the user says "turn on the light", and the system can ask the user if he is sure to turn on the light.

[0075] Step S203, controlling the target device to execute the target instruction, obtaining an execution result, and adaptively updating the voice instruction library and the semantic matching model according to the execution result and the user voice.

[0076] Specifically, the above steps realize natural and high-accuracy voice control and interaction without the need for wake-up words, and further enhance the interactive experience through contextual semantic modeling and multimodal fusion. Before executing the command, the confirmed command needs to be parsed to extract the operation object, operation method and related parameters. For example, for "turn on the lights in the living room", the operation object is parsed as "the lights in the living room" and the operation method is "turn on". According to the results of the command parsing, the corresponding operation is performed, and the control command can be sent through the communication interface with the home appliance to achieve control of the device. Among them, in the process of executing the operation, the status feedback and error handling of the device should also be considered. For example, if the device cannot respond to the control command, the system should promptly feedback the error information to the user, and try to resend the command or take other measures.

[0077] In addition, this application regularly updates the voice command library and semantic matching model based on user interaction feedback and new command data. Among them, the model is updated in real time using online learning to adapt to the ever-changing user needs and environment. When updating the model, attention should be paid to the quality and reliability of the data to avoid introducing erroneous data that may cause the model performance to deteriorate.

[0078] The above-mentioned voice command recognition method of the present application first obtains a voice command library and obtains user voice in real time. The semantic encoding method of the voice command library is a target encoding method, and the user voice is a voice that directly controls the target device to perform the target operation; then the target encoding method is used to convert the user voice into a corresponding target semantic vector, and the semantic matching model is used to match the target semantic vector with the basic semantic vector in the voice command library to obtain a target instruction corresponding to the basic semantic vector that is the same as the target semantic vector; finally, the target device is controlled to execute the target instruction to obtain the execution result, and the voice command library and the semantic matching model are adaptively updated according to the execution result and the user voice. This method can achieve the goal of naturally and efficiently completing voice control and interaction without a wake-up word by continuously monitoring and quickly matching and identifying the voice with the pre-built command semantics, combining context information and multimodal fusion technology, greatly improving the user experience, promoting the development and popularization of smart home technology, and solving the problem that the voice interaction method in the prior art requires the user to say a preset wake-up word before interaction can be achieved, resulting in high interaction complexity, greater impact from environmental noise, and inability to be personalized.

[0079] In some examples, a common voice command library for home scenes is first pre-built, that is, a command library containing common voice commands in home scenes is pre-built through data analysis and semantic modeling technology. Then, through real-time speech semantic encoding and efficient semantic matching, the voice in the home environment is continuously monitored and collected, and the acquired voice is converted into a semantic vector representation in real time using the same semantic encoding method as the command library. Then, an efficient semantic matching algorithm is used, such as approximate nearest neighbor search based on vector similarity, to quickly match the speech semantic vector with the command semantics in the command library to identify the user's command intention. Then, the context information such as the identified command, execution feedback, and related scene status is input into the context encoder to continuously update the speech semantic understanding model. By modeling historical interactions and environmental changes, the semantic representation can be adaptively adjusted to improve the ability to understand complex semantics. Finally, multimodal interaction fusion is performed, that is, combining other modal information such as speech and vision, which can further improve the accuracy and reliability of command intention recognition and achieve more natural human-computer interaction.

[0080] In order to enable those skilled in the art to more clearly understand the technical solution of the present application, the implementation process of the voice command recognition method of the present application will be described in detail below in conjunction with specific embodiments.

[0081] This embodiment relates to a specific method for recognizing voice commands, such as Figure 3 As shown, the following steps are included:

[0082] Step S1: Pre-build a common voice command library for home scenarios: Through data analysis and semantic modeling technology, build a command library containing common voice commands in home scenarios. Each command contains a command semantic vector representation and the corresponding execution action or response content. The command semantic representation can be obtained through a pre-trained language model or other semantic representation learning methods. The command library can be constructed manually or automatically generated based on data and continuously updated.

[0083] Step S2: Voice collection and recognition: Use the voice recognition model to continuously monitor and collect voice input in the home environment, recognize the acquired voice signals, and convert the voice into text form.

[0084] Step S3: Speech semantic coding: Use the same language model or algorithm as the instruction semantic representation coding method to perform semantic vector coding on the speech text to obtain the semantic vector representation of the current speech.

[0085] Step S4: semantic matching and instruction recognition: perform efficient semantic matching of the current speech semantic vector with the semantic vectors of all instructions in the instruction library, such as using an approximate nearest neighbor search algorithm based on vector similarity. If the matching degree with a certain instruction vector exceeds a preset threshold, the speech instruction is determined to be valid.

[0086] Step S5: Execute corresponding operations: For the recognized valid voice commands, search the command library for the corresponding execution actions or response contents, and execute related home control operations or multimodal interactive feedback.

[0087] Step S6: Context understanding and model updating: The recognized voice command and its execution feedback, relevant scene status and other context information are input into the context encoder, and the voice semantic understanding model is continuously updated to adaptively adjust the semantic representation.

[0088] Step S7: Multimodal fusion: combining voice with other modal information such as vision to improve the accuracy and reliability of voice command understanding and achieve more natural human-computer interaction.

[0089] The above steps are executed repeatedly to achieve continuous wake-up-free smart home voice interaction. The key to this method is the pre-built command library and semantic matching technology, which can efficiently recognize voice commands without wake-up words, and further enhance the interactive experience through contextual semantic modeling and multimodal fusion.

[0090] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0091] The embodiment of the present application also provides an intelligent recognition device for voice commands. It should be noted that the intelligent recognition device for voice commands of the embodiment of the present application can be used to execute the recognition method for voice commands provided by the embodiment of the present application. The device is used to implement the above-mentioned embodiments and preferred implementation modes, and the descriptions thereof will not be repeated. As used below, the term "module" can implement a combination of software and / or hardware for a predetermined function. Although the device described in the following embodiments is preferably implemented in software, the implementation of hardware, or a combination of software and hardware, is also possible and conceived.

[0092] The following is an introduction to the intelligent recognition device for voice commands provided in an embodiment of the present application.

[0093] Figure 4 is a schematic diagram of an intelligent recognition device for voice commands according to an embodiment of the present application. Figure 4As shown, the device includes: an acquisition unit 10, a first processing unit 20 and a second processing unit 30, the acquisition unit 10 is used to acquire a voice instruction library and acquire user voice in real time, the semantic encoding method of the above-mentioned voice instruction library is a target encoding method, and the above-mentioned user voice is a voice that directly controls the target device to perform a target operation; the first processing unit 20 is used to adopt the above-mentioned target encoding method to convert the above-mentioned user voice into a corresponding target semantic vector, and adopt a semantic matching model to match the above-mentioned target semantic vector with the basic semantic vector in the above-mentioned voice instruction library to obtain a target instruction corresponding to the above-mentioned basic semantic vector that is the same as the above-mentioned target semantic vector; the second processing unit 30 is used to control the above-mentioned target device to execute the above-mentioned target instruction, obtain an execution result, and adaptively update the above-mentioned voice instruction library and the above-mentioned semantic matching model according to the above-mentioned execution result and the above-mentioned user voice.

[0094] The above-mentioned intelligent recognition device of voice instructions of the present application includes an acquisition unit, a first processing unit and a second processing unit. The acquisition unit is used to acquire a voice instruction library and acquire user voice in real time. The semantic encoding method of the voice instruction library is a target encoding method, and the user voice is a voice that directly controls the target device to perform the target operation; the first processing unit is used to adopt the target encoding method to convert the user voice into a corresponding target semantic vector, and adopt a semantic matching model to match the target semantic vector with the basic semantic vector in the voice instruction library to obtain a target instruction corresponding to the basic semantic vector that is the same as the target semantic vector; the second processing unit is used to control the target device to execute the target instruction, obtain the execution result, and adaptively update the voice instruction library and the semantic matching model according to the execution result and the user voice. The device can achieve the goal of naturally and efficiently completing voice control and interaction without a wake-up word by continuously monitoring and quickly matching and identifying the voice with the pre-built instruction semantics, combining context information and multimodal fusion technology, greatly improving the user experience, promoting the development and popularization of smart home technology, and solving the problem that the voice interaction method in the prior art requires the user to say a preset wake-up word before interaction can be achieved, resulting in high interaction complexity, greater impact of environmental noise, and inability to be personalized.

[0095] In some examples, the acquisition unit includes a first acquisition module, a second acquisition module and a third acquisition module, wherein the first acquisition module is used to acquire the historical voice data of the user at multiple historical moments; the second acquisition module is used to classify all the above historical voice data based on the execution device, execution function and execution scenario corresponding to each of the above historical voice data, and obtain multiple groups of voice data sets, wherein one group of the above voice data sets corresponds to one execution device, one execution function and one execution scenario; the third acquisition module is used to remove all redundant words of the above historical voice data in each group of the above voice data sets, and obtain the above voice instruction library. Extensive collection of various possible voice instructions and optimization can build a more complete and accurate voice instruction library.

[0096] In some instances, the first processing unit includes a first processing module, a second processing module and a third processing module. The first processing module is used to convert the user's voice into voice text, and use natural language processing technology to extract characteristic semantic elements in the voice text, and the characteristic semantic elements include characteristic characters and / or characteristic words; the second processing module is used to obtain the user's target image information and target sensor information; the third processing module is used to use the target encoding method, based on the target image information and / or the target sensor information, to encode the characteristic semantic elements and obtain the target semantic vector corresponding to the characteristic semantic elements. Rapidly matching the voice semantic vector with the instruction semantics in the instruction library can quickly identify the user's instruction intention and improve the user experience.

[0097] In some examples, the second processing module includes a first processing submodule and a second processing submodule, wherein the first processing submodule is used to collect user image data of the user using an image acquisition device, wherein the user image data at least includes the face information and behavior information of the user; the second processing submodule is used to recognize the user image data using image recognition technology, obtain the face data and behavior data of the user, and determine the face data and behavior data of the user as the target image information of the user. The interaction is combined with the visual information captured by the camera to improve the accuracy and reliability of the understanding of voice commands, thereby achieving more natural human-computer interaction.

[0098] In some examples, the second processing module also includes a third processing submodule and a fourth processing submodule, the third processing submodule is used to receive multiple sensor detections to obtain corresponding sub-sensing information, the multiple sensors are respectively a temperature sensor, a humidity sensor, and a human position sensor, the temperature sensor is used to detect the ambient temperature of the user's environment, the humidity sensor is used to detect the ambient humidity of the user's environment, and the human position sensor is used to detect the location information of the user; the fourth processing submodule is used to determine all of the above sub-sensing information as target sensing information. Using sensor fusion technology, such as Kalman filtering, particle filtering, etc., and further integrating other modal information can effectively improve the accuracy and reliability of environmental perception, thereby improving the accuracy and stability of voice command understanding.

[0099] In some instances, the third processing module includes a fifth processing submodule and a sixth processing submodule, the fifth processing submodule is used to obtain multiple groups of historical data, one group of the above historical data at least includes the historical voice data of the user at the historical moment and the execution record, historical environment information and historical execution time of the above target device corresponding to the above historical voice data; the sixth processing submodule is used to use the above target encoding method, based on the above multiple groups of historical data, the above target image information and at least one of the above target sensor information, to encode the above feature semantic elements and obtain the above target semantic vector corresponding to the above feature semantic elements. Continuously monitor and collect the home environment voice, and use the same semantic encoding method as the instruction library to convert the acquired voice into a semantic vector representation in real time.

[0100] In some examples, the first processing unit further includes a fourth processing module, a fifth processing module, a sixth processing module and a seventh processing module, wherein the fourth processing module is used to match the target semantic vector with the basic semantic vector in the voice instruction library by using a matching algorithm to obtain a plurality of initial instructions corresponding to the basic semantic vector identical to the target semantic vector; the fifth processing module is used to determine the accuracy of each of the initial instructions according to the plurality of groups of historical data and current environmental information, and determine the initial instruction with the highest accuracy as the preliminary instruction; the sixth processing module is used to send a confirmation message to the user and receive feedback information sent by the user according to the confirmation message, wherein the confirmation message is confirmation information to the user whether the preliminary instruction is the target instruction; the seventh processing module is used to determine the preliminary instruction as the target instruction when the feedback message is confirmation that the preliminary instruction is the target instruction. The context information such as the identified instruction, execution feedback and related scene status is input into the context encoder to continuously update the speech semantic understanding model. By modeling historical interactions and environmental changes, the semantic representation can be adaptively adjusted to improve the ability to understand complex semantics.

[0101] The intelligent recognition device for voice commands includes a processor and a memory. The acquisition unit and the like are stored in the memory as program units, and the processor executes the program units stored in the memory to implement corresponding functions. The modules are all located in the same processor; or, the modules are located in different processors in any combination.

[0102] The processor contains a kernel, which retrieves the corresponding program unit from the memory. One or more kernels can be set, and the kernel parameters can be adjusted to solve the problems in the existing voice interaction method that requires the user to say a preset wake-up word before interaction can be achieved, resulting in high interaction complexity, greater impact from environmental noise, and inability to be personalized.

[0103] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM, and the memory includes at least one memory chip.

[0104] An embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the voice command recognition method.

[0105] Specifically, the voice command recognition method includes:

[0106] Step S201, obtaining a voice instruction library and obtaining user voice in real time, wherein the semantic encoding method of the voice instruction library is a target encoding method, and the user voice is a voice that directly controls the target device to perform a target operation;

[0107] Specifically, the pre-built voice command library commonly used in home scenarios can quickly and accurately identify the user's real-time voice command intentions.

[0108] Step S202, using the target encoding method, converting the user voice into a corresponding target semantic vector, and using a semantic matching model to match the target semantic vector with the basic semantic vector in the voice instruction library, to obtain a target instruction corresponding to the basic semantic vector that is the same as the target semantic vector;

[0109] Specifically, using an efficient semantic matching algorithm to analyze and understand multiple sets of historical data can improve the accuracy of instruction recognition and interaction efficiency. At the same time, database technology can be used to store the above historical data and establish corresponding indexes for rapid retrieval and analysis.

[0110] Step S203, controlling the target device to execute the target instruction, obtaining an execution result, and adaptively updating the voice instruction library and the semantic matching model according to the execution result and the user voice.

[0111] Specifically, it enables natural and highly accurate voice control and interaction without the need for wake-up words, and further enhances the interactive experience through contextual semantic modeling and multimodal fusion.

[0112] Optionally, the target encoding method is used to convert the user voice into a corresponding target semantic vector, including: converting the user voice into speech text, and using natural language processing technology to extract characteristic semantic elements in the speech text, the characteristic semantic elements including characteristic characters and / or characteristic words; obtaining the user's target image information and target sensor information; using the target encoding method, encoding the characteristic semantic elements based on the target image information and / or the target sensor information to obtain the target semantic vector corresponding to the characteristic semantic elements.

[0113] Optionally, obtaining the target image information of the user includes: using an image acquisition device to acquire user image data of the user, the user image data at least including facial information and behavioral information of the user; using image recognition technology to identify the user image data to obtain the facial data and behavioral data of the user, and determining the facial data and behavioral data of the user as the target image information of the user.

[0114] Optionally, obtaining the target sensor information of the user includes: receiving multiple sensor detections to obtain corresponding sub-sensor information, the multiple sensors are respectively a temperature sensor, a humidity sensor, and a human position sensor, the temperature sensor is used to detect the ambient temperature of the user's environment, the humidity sensor is used to detect the ambient humidity of the user's environment, and the human position sensor is used to detect the position information of the user; and all of the above sub-sensor information are determined as target sensor information.

[0115] Optionally, the target encoding method is used to encode the characteristic semantic elements based on the target image information and / or the target sensor information to obtain the target semantic vector corresponding to the characteristic semantic elements, including: obtaining multiple groups of historical data, one group of historical data at least includes the historical voice data of the user at a historical moment and the execution record, historical environment information and historical execution time of the target device corresponding to the historical voice data; the target encoding method is used to encode the characteristic semantic elements based on at least one of the multiple groups of historical data, the target image information and the target sensor information to obtain the target semantic vector corresponding to the characteristic semantic elements.

[0116] Optionally, a semantic matching model is used to match the target semantic vector with the basic semantic vector in the voice instruction library to obtain a target instruction corresponding to the basic semantic vector that is the same as the target semantic vector, including: using a matching algorithm to match the target semantic vector with the basic semantic vector in the voice instruction library to obtain multiple initial instructions corresponding to the basic semantic vector that is the same as the target semantic vector; determining the accuracy of each of the initial instructions based on multiple groups of historical data and current environmental information, and determining the initial instruction with the highest accuracy as the preliminary instruction; sending a confirmation message to the user, and receiving feedback information sent by the user based on the confirmation message, the confirmation message is confirmation information to the user whether the preliminary instruction is the target instruction; when the feedback information is to confirm that the preliminary instruction is the target instruction, the preliminary instruction is determined as the target instruction.

[0117] Optionally, obtaining a voice command library includes: obtaining historical voice data of users at multiple historical moments; classifying all of the above historical voice data based on the execution devices, execution functions and execution scenarios corresponding to each of the above historical voice data to obtain multiple groups of voice data sets, one group of the above voice data sets corresponding to an execution device, an execution function and an execution scenario; removing redundant vocabulary from all of the above historical voice data in each group of the above voice data sets to obtain the above voice command library.

[0118] An embodiment of the present invention provides a processor, which is used to run a program, wherein the program executes the voice instruction recognition method when running.

[0119] Specifically, the voice command recognition method includes:

[0120] Step S201, obtaining a voice instruction library and obtaining user voice in real time, wherein the semantic encoding method of the voice instruction library is a target encoding method, and the user voice is a voice that directly controls the target device to perform a target operation;

[0121] Specifically, the pre-built voice command library commonly used in home scenarios can quickly and accurately identify the user's real-time voice command intentions.

[0122] Step S202, using the target encoding method, converting the user voice into a corresponding target semantic vector, and using a semantic matching model to match the target semantic vector with the basic semantic vector in the voice instruction library, to obtain a target instruction corresponding to the basic semantic vector that is the same as the target semantic vector;

[0123] Specifically, using an efficient semantic matching algorithm to analyze and understand multiple sets of historical data can improve the accuracy of instruction recognition and interaction efficiency. At the same time, database technology can be used to store the above historical data and establish corresponding indexes for rapid retrieval and analysis.

[0124] Step S203, controlling the target device to execute the target instruction, obtaining an execution result, and adaptively updating the voice instruction library and the semantic matching model according to the execution result and the user voice.

[0125] Specifically, it enables natural and highly accurate voice control and interaction without the need for wake-up words, and further enhances the interactive experience through contextual semantic modeling and multimodal fusion.

[0126] An embodiment of the present invention provides a device, the device including a processor, a memory, and a program stored in the memory and executable on the processor, and when the processor executes the program, at least the following steps are implemented:

[0127] Step S201, obtaining a voice instruction library and obtaining user voice in real time, wherein the semantic encoding method of the voice instruction library is a target encoding method, and the user voice is a voice that directly controls the target device to perform a target operation;

[0128] Specifically, the pre-built voice command library commonly used in home scenarios can quickly and accurately identify the user's real-time voice command intentions.

[0129] Step S202, using the target encoding method, converting the user voice into a corresponding target semantic vector, and using a semantic matching model to match the target semantic vector with the basic semantic vector in the voice instruction library, to obtain a target instruction corresponding to the basic semantic vector that is the same as the target semantic vector;

[0130] Specifically, using an efficient semantic matching algorithm to analyze and understand multiple sets of historical data can improve the accuracy of instruction recognition and interaction efficiency. At the same time, database technology can be used to store the above historical data and establish corresponding indexes for rapid retrieval and analysis.

[0131] Step S203, controlling the target device to execute the target instruction, obtaining an execution result, and adaptively updating the voice instruction library and the semantic matching model according to the execution result and the user voice.

[0132] Specifically, it enables natural and highly accurate voice control and interaction without the need for wake-up words, and further enhances the interactive experience through contextual semantic modeling and multimodal fusion.

[0133] The devices in this article can be servers, PCs, PADs, mobile phones, etc.

[0134] The present application also provides a computer program product, which, when executed on a data processing device, is suitable for executing a program for initializing at least the following method steps:

[0135] Step S201, obtaining a voice instruction library and obtaining user voice in real time, wherein the semantic encoding method of the voice instruction library is a target encoding method, and the user voice is a voice that directly controls the target device to perform a target operation;

[0136] Specifically, the pre-built voice command library commonly used in home scenarios can quickly and accurately identify the user's real-time voice command intentions.

[0137] Step S202, using the target encoding method, converting the user voice into a corresponding target semantic vector, and using a semantic matching model to match the target semantic vector with the basic semantic vector in the voice instruction library, to obtain a target instruction corresponding to the basic semantic vector that is the same as the target semantic vector;

[0138] Specifically, using an efficient semantic matching algorithm to analyze and understand multiple sets of historical data can improve the accuracy of instruction recognition and interaction efficiency. At the same time, database technology can be used to store the above historical data and establish corresponding indexes for rapid retrieval and analysis.

[0139] Step S203, controlling the target device to execute the target instruction, obtaining an execution result, and adaptively updating the voice instruction library and the semantic matching model according to the execution result and the user voice.

[0140] Specifically, it enables natural and highly accurate voice control and interaction without the need for wake-up words, and further enhances the interactive experience through contextual semantic modeling and multimodal fusion.

[0141] Obviously, those skilled in the art should understand that the above modules or steps of the present invention can be implemented by a general computing device, they can be concentrated on a single computing device, or distributed on a network composed of multiple computing devices, they can be implemented by a program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, the steps shown or described can be executed in a different order than here, or they can be made into individual integrated circuit modules, or multiple modules or steps therein can be made into a single integrated circuit module for implementation. Thus, the present invention is not limited to any specific combination of hardware and software.

[0142] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that contain computer-usable program code.

[0143] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0144] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0145] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0146] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0147] The memory may include non-permanent memory in a computer-readable medium, random access memory (RAM) and / or non-volatile memory in the form of read-only memory (ROM) or flash RAM. The memory is an example of a computer-readable medium.

[0148] Computer readable media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, magnetic disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer readable media does not include temporary computer readable media (transitory media), such as modulated data signals and carrier waves.

[0149] It should also be noted that the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, commodity or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, commodity or device. In the absence of more restrictions, the elements defined by the sentence "comprises a ..." do not exclude the existence of other identical elements in the process, method, commodity or device including the elements.

[0150] From the above description, it can be seen that the above embodiments of the present application achieve the following technical effects:

[0151] 1) The above-mentioned voice command recognition method of the present application first obtains a voice command library and obtains user voice in real time. The semantic encoding method of the voice command library is the target encoding method, and the user voice is the voice that directly controls the target device to perform the target operation; then the target encoding method is used to convert the user voice into the corresponding target semantic vector, and the semantic matching model is used to match the target semantic vector with the basic semantic vector in the voice command library to obtain the target instruction corresponding to the basic semantic vector that is the same as the target semantic vector; finally, the target device is controlled to execute the target instruction to obtain the execution result, and the voice command library and the semantic matching model are adaptively updated according to the execution result and the user voice. This method can achieve the goal of naturally and efficiently completing voice control and interaction without a wake-up word by continuously monitoring and quickly matching and identifying the voice with the pre-built command semantics, combining context information and multimodal fusion technology, greatly improving the user experience, promoting the development and popularization of smart home technology, and solving the problem that the voice interaction method in the prior art requires the user to say the preset wake-up word before interaction can be achieved, resulting in high interaction complexity, greater impact of environmental noise, and inability to be personalized.

[0152] 2) The intelligent recognition device for the above-mentioned voice instructions of the present application includes an acquisition unit, a first processing unit and a second processing unit. The acquisition unit is used to acquire a voice instruction library and acquire user voice in real time. The semantic encoding method of the voice instruction library is a target encoding method, and the user voice is a voice that directly controls the target device to perform the target operation; the first processing unit is used to adopt the target encoding method to convert the user voice into a corresponding target semantic vector, and adopt a semantic matching model to match the target semantic vector with the basic semantic vector in the voice instruction library to obtain a target instruction corresponding to the basic semantic vector that is the same as the target semantic vector; the second processing unit is used to control the target device to execute the target instruction, obtain the execution result, and adaptively update the voice instruction library and the semantic matching model according to the execution result and the user voice. The device can achieve the goal of naturally and efficiently completing voice control and interaction without a wake-up word by continuously monitoring and quickly matching and identifying the voice with the pre-built instruction semantics, combining context information and multimodal fusion technology, greatly improving the user experience, promoting the development and popularization of smart home technology, and solving the problem that the voice interaction method in the prior art requires the user to say a preset wake-up word before interaction can be achieved, resulting in high interaction complexity, greater impact of environmental noise, and inability to be personalized.

[0153] The above description is only the preferred embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application shall be included in the protection scope of the present application.

Claims

1. A method for recognizing a voice command, characterized in that: include: Acquire a voice instruction library and acquire user voice in real time, wherein the semantic encoding mode of the voice instruction library is the target encoding mode, and the user voice is the voice that directly controls the target device to perform the target operation; The target encoding method is used to convert the user speech into a corresponding target semantic vector, and the target semantic vector is matched with the basic semantic vector in the speech instruction library by using a semantic matching model to obtain a target instruction corresponding to the basic semantic vector that is the same as the target semantic vector; The target device is controlled to execute the target instruction, and an execution result is obtained, and the voice instruction library and the semantic matching model are adaptively updated according to the execution result and the user voice.

2. The method according to claim 1, characterized in that The target encoding method is used to convert the user speech into a corresponding target semantic vector, including: Convert the user's speech into voice text, and extract characteristic semantic elements from the voice text by using natural language processing technology, wherein the characteristic semantic elements include characteristic characters and / or characteristic words; Obtain target image information and target sensor information of the user; The target encoding method is adopted to encode the characteristic semantic elements based on the target image information and / or the target sensor information to obtain the target semantic vector corresponding to the characteristic semantic elements.

3. The method according to claim 2, characterized in that Get the user's target image information, including: Using an image acquisition device to acquire user image data of a user, the user image data at least including facial information and behavior information of the user; The user image data is identified by using image recognition technology to obtain the user's facial data and behavior data, and the user's facial data and behavior data are determined as the user's target image information.

4. The method according to claim 2, characterized in that: Obtain the user's target sensor information, including: Receiving corresponding sub-sensing information detected by multiple sensors, wherein the multiple sensors are respectively a temperature sensor, a humidity sensor, and a human position sensor, wherein the temperature sensor is used to detect the ambient temperature of the environment where the user is located, the humidity sensor is used to detect the ambient humidity of the environment where the user is located, and the human position sensor is used to detect the location information of the user; All of the sub-sensing information are determined as target sensing information.

5. The method according to claim 2, characterized in that: The target encoding method is adopted to encode the characteristic semantic element based on the target image information and / or the target sensor information to obtain the target semantic vector corresponding to the characteristic semantic element, including: Acquire multiple groups of historical data, where one group of historical data at least includes historical voice data of the user at a historical moment and execution records, historical environment information, and historical execution time of the target device corresponding to the historical voice data; The target encoding method is adopted to encode the characteristic semantic elements based on the multiple groups of historical data, the target image information and at least one of the target sensor information to obtain the target semantic vector corresponding to the characteristic semantic element.

6. The method according to claim 5, characterized in that The target semantic vector is matched with the basic semantic vector in the voice instruction library by using a semantic matching model to obtain a target instruction corresponding to the basic semantic vector which is the same as the target semantic vector, including: Using a matching algorithm to match the target semantic vector with the basic semantic vectors in the voice instruction library, and obtaining a plurality of initial instructions corresponding to the basic semantic vectors that are the same as the target semantic vector; Determining the accuracy of each of the initial instructions according to the multiple groups of historical data and current environmental information, and determining the initial instruction with the highest accuracy as the preliminary instruction; Sending confirmation information to the user, and receiving feedback information sent by the user according to the confirmation information, wherein the confirmation information is confirmation information for confirming to the user whether the preparatory instruction is the target instruction; When the feedback information confirms that the preliminary instruction is the target instruction, the preliminary instruction is determined as the target instruction.

7. The method according to claim 1, characterized in that Get the voice command library, including: Obtain the historical voice data of the user at multiple historical moments; Based on the execution device, execution function and execution scenario corresponding to each of the historical voice data, all the historical voice data are classified to obtain multiple groups of voice data sets, where one group of the voice data sets corresponds to one execution device, one execution function and one execution scenario; All redundant words of the historical voice data in each group of the voice data sets are removed to obtain the voice instruction library.

8. A voice command recognition device, characterized in that: include: An acquisition unit, used to acquire a voice instruction library and acquire user voice in real time, wherein the semantic encoding mode of the voice instruction library is a target encoding mode, and the user voice is a voice that directly controls the target device to perform a target operation; A first processing unit is used to convert the user speech into a corresponding target semantic vector by adopting the target encoding method, and match the target semantic vector with a basic semantic vector in the speech instruction library by adopting a semantic matching model to obtain a target instruction corresponding to the basic semantic vector that is the same as the target semantic vector; The second processing unit is used to control the target device to execute the target instruction, obtain an execution result, and adaptively update the voice instruction library and the semantic matching model according to the execution result and the user voice.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the method for recognizing a voice instruction according to any one of claims 1 to 7.

10. An electronic device, characterized in that: include: One or more processors, a memory, and one or more programs, wherein the one or more programs are stored in the memory and are configured to be executed by the one or more processors, and the one or more programs include a recognition method for executing the voice instruction described in any one of claims 1 to 7.

Images