Method, system and equipment for voice control equipment based on large language model

Abstract

The embodiment of the invention provides a method, a system and equipment for voice control of equipment based on a large language model. The method is applied to the field of electric data processing, and comprises the following steps: collecting a user voice signal, and preprocessing the voice signal to obtain a corresponding text through voice recognition; performing entropy calculation on the equipment control instruction set, the attribute set and the attribute value set to obtain one or more key attributes, and constructing a cue word example; and constructing a complete cue word in combination with the corresponding text and the cue word example, and inputting the complete cue word into the large language model to obtain an equipment control instruction. Through the scheme disclosed by the invention, a voice control task with long context, multiple languages and generalization requirements can be completed.

Description

Technical Field

[0001] The present disclosure relates to the field of electrical data processing, and in particular to a method, system and device for voice control of a device based on a large language model. Background Art

[0002] Voice control devices have become an indispensable part of the modern smart home and IoT ecosystem. Their deep integration with artificial intelligence and natural language processing technologies provides users with a convenient and intuitive way of interaction. In the application scenarios of voice control devices, the collaborative working mode of three major technical modules, STT (Speech to Text), TTS (Text to Speech) and NLP (Natural Language Processing), is traditionally often used. The workflow of this method is usually: first, record the voice command on the user-end device; then, upload the recorded audio data to the cloud server; then, the STT module converts the voice into text; then, the NLP module analyzes the text content, identifies the user's intention, determines the device to be controlled and the specific operation; the system executes the corresponding device control instructions; finally, feedback text is generated according to the operation results, converted into voice through the TTS module, and transmitted back to the user's device for playback. This technical route has laid the foundation for the development of voice control devices in the past, enabling users to control various smart devices through simple voice commands, greatly improving the convenience of life and work efficiency.

[0003] However, with the increasing complexity and personalization of user needs, traditional voice control technology faces many challenges. First, the serial processing of the three independent modules of STT, NLP and TTS increases the system latency and affects the user experience; second, this method has high requirements on the quality of network connection and is prone to slow response or failure when the network is unstable; third, the traditional NLP model has limited accuracy when processing complex, ambiguous or context-related instructions, and it is difficult to meet the diverse needs of users; finally, this method is not satisfactory when processing multiple languages, dialects or accents. In order to solve these problems, researchers have disclosed a series of improvement plans, such as optimizing speech recognition algorithms, introducing edge computing technology, and using more advanced natural language understanding models. However, although these methods have improved in some aspects, it is still difficult to fundamentally solve the all-round challenges faced by voice control devices, especially in terms of long-context understanding, multi-language support, generalization ability and response speed. There are still obvious deficiencies.

[0004] Therefore, there is an urgent need for a technical solution that can complete speech control tasks with long context, multi-language and generalization requirements. Summary of the invention

[0005] In order to solve the deficiencies of the prior art, the present disclosure provides a method, system and device for voice control of a device based on a large language model. The present disclosure solves the technical problems that the prior art performs poorly when processing long conversations or complex contexts and requires training for specific tasks.

[0006] According to a first aspect of the present disclosure, a method for voice controlling a device based on a large language model is provided, comprising: collecting a user voice signal and preprocessing the voice signal to obtain a corresponding text through voice recognition; performing entropy calculation on a device control instruction set, an attribute set, and an attribute value set to obtain one or more key attributes and construct a prompt word example; constructing a complete prompt word in combination with the corresponding text and the prompt word example, and inputting the complete prompt word into the large language model to obtain the device control instruction.

[0007] In a possible implementation, a user voice signal is collected and the voice signal is preprocessed to obtain the corresponding text through voice recognition, including: collecting a user voice signal, the sampling rate of the voice signal is 16kHz, and the bit depth is 16bit; preprocessing the voice signal, including: using a high-pass filter to filter out low-frequency noise below 50Hz; using an adaptive noise elimination algorithm to eliminate background noise; performing voice activity detection to extract valid voice segments; and inputting the preprocessed voice signal into a deep learning-based voice recognition model to obtain a corresponding text recognition result.

[0008] In a possible implementation, entropy calculation is performed on a set of device control instructions, a set of attributes, and a set of attribute values ​​to obtain one or more key attributes and construct a prompt word example, including: obtaining a set of device control instructions, wherein each device control instruction includes multiple attributes; determining a target value range for each attribute; calculating the entropy value of the device control instruction; calculating the entropy value of each attribute; calculating the entropy value of each attribute value; selecting at least one key attribute based on the information gain principle; constructing a prompt word in JSON format containing the selected key attributes; and adding examples to the constructed JSON format prompt word.

[0009] In a possible implementation, it also includes: regularly updating the usage frequency of device control instructions, attributes and attribute values; recalculating the entropy value and information gain according to the updated usage frequency, and dynamically adjusting the key attributes and examples in the JSON format prompt words.

[0010] In a possible implementation, a complete prompt word is constructed in combination with corresponding text and prompt word examples, and is input into a large language model to obtain device control instructions, including: obtaining a text result output by a speech recognition module, and using the text result as user input information; constructing a multi-round dialogue structure including system role definition, user input information, assistant response and instruction set, and generating a complete prompt word for semantic parsing; transmitting the complete prompt word to a preset large language model, triggering the large language model to perform a semantic parsing operation; receiving output data of the large language model, wherein the output data is a standardized device control instruction in JSON format.

[0011] In a possible implementation, the step of calculating the entropy value of the device control instruction includes: Where H(D) represents the entropy of the instruction set D, d i represents the ith instruction in the instruction set D, n represents the total number of instructions, p(d i ) indicates instruction d i frequency of use.

[0012] In a possible implementation, the entropy value of each attribute is calculated, including: Among them, H(A i ) represents the attribute set A of the i-th instruction i The entropy of ij represents the jth attribute of the ith instruction, m represents the total number of attributes, and p(a ij ) represents attribute a ij frequency of use.

[0013] In a possible implementation, calculating the entropy value of each attribute value includes: Among them, H(V ij ) represents the attribute value set V of the jth attribute of the ith instruction ij The entropy of v ijk represents the kth value of the jth attribute of the ith instruction, where k represents the total number of attribute values, p(v ijk ) indicates the attribute value v ijk frequency of use.

[0014] According to a second aspect of the present disclosure, a system for voice control devices based on a large language model is provided, which is used to execute the method as described in the first aspect, and includes: a collection unit, a recognition unit and an instruction unit; wherein the collection unit is used to collect user voice signals; the recognition unit is used to preprocess the voice signals to obtain corresponding texts through voice recognition; the instruction unit is used to obtain one or more key attributes by performing entropy calculation on a device control instruction set, an attribute set and an attribute value set, and construct a prompt word example; a complete prompt word is constructed in combination with the corresponding text and the prompt word example, and is input into the large language model to obtain the device control instruction.

[0015] According to a third aspect of the present disclosure, an electronic device is provided, comprising: a memory and a processor, wherein a computer program is stored in the memory, and when the processor executes the program, the method described above is implemented.

[0016] In the method, system and device for voice control devices based on a large language model as provided above, the disclosed embodiment calculates entropy and information gain, and the method can automatically adjust the key attributes and examples in the prompt words to adapt to changes in different scenarios and user habits. Furthermore, due to the use of a large language model as the core processing unit, the method can support multi-language input without the need to train a model separately for each language, and can understand and process voice instructions with long contexts, fuzzy expressions and complex logic, overcoming the limitations of traditional NLP methods. Furthermore, by pre-building structured prompt words, the amount of computation at runtime is reduced and the response speed is improved. Furthermore, by updating the instruction set and attribute definitions, it is easy to add new device types and control instructions. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the embodiments of the present disclosure or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present disclosure. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0018] Figure 1 A schematic diagram of a method flow of a voice control device based on a large language model according to an embodiment of the present disclosure is shown;

[0019] Figure 2 A system schematic block diagram of a voice control device based on a large language model according to an embodiment of the present disclosure is shown;

[0020] Figure 3 A schematic block diagram of a voice control device system architecture based on a large language model according to an embodiment of the present disclosure is shown;

[0021] Figure 4 A block diagram of an exemplary electronic device according to an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0022] Various exemplary embodiments of the present disclosure will now be described in detail with reference to the accompanying drawings. It should be noted that the relative arrangement of components and steps, numerical expressions and numerical values ​​set forth in these embodiments do not limit the scope of the present disclosure unless otherwise specifically stated.

[0023] Those skilled in the art can understand that the terms "first", "second" and the like in the embodiments of the present disclosure are only used to distinguish different steps, devices or modules, etc., and neither represent any specific technical meaning nor represent the necessary logical order between them. It should also be understood that in the embodiments of the present disclosure, "multiple" can refer to two or more, and "at least one" can refer to one, two or more. It should also be understood that for any component, data or structure mentioned in the embodiments of the present disclosure, in the absence of explicit limitation or contrary revelation given in the context, it can generally be understood as one or more. In addition, the term "and / or" in the present disclosure is only a description of the association relationship of the associated objects, indicating that there can be three relationships, for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in the present disclosure generally indicates that the associated objects before and after are an "or" relationship. It should also be understood that the description of each embodiment in the present disclosure emphasizes the differences between the embodiments, and the same or similar parts can refer to each other. For the sake of brevity, they will not be repeated one by one.

[0024] At the same time, it should be understood that, for ease of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship. The following description of at least one exemplary embodiment is actually only illustrative and is by no means intended to limit the present disclosure and its application or use. The techniques, methods and devices known to ordinary technicians in the relevant fields may not be discussed in detail, but where appropriate, the techniques, methods and devices should be considered part of the specification. It should be noted that similar numbers and letters represent similar items in the following drawings, so once an item is defined in one drawing, it does not need to be further discussed in subsequent drawings.

[0025] In order to make the purpose, technical solution and advantages of the embodiments of the present disclosure clearer, the technical solution in the embodiments of the present disclosure will be clearly and completely described below in conjunction with the drawings in the embodiments of the present disclosure. Obviously, the described embodiments are part of the embodiments of the present disclosure, not all of the embodiments. Based on the embodiments in the present disclosure, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present disclosure.

[0026] Figure 1 A flow chart of a method for voice control of a device based on a large language model provided in an embodiment of the present disclosure. The method of the embodiment of the present disclosure aims to achieve long-context, multi-language, generalized and fast-response voice control. It should be understood that a voice control device refers to a technology that operates and manages various electronic devices or systems through human voice commands. Voice control devices generally include but are not limited to smart home devices, vehicle-mounted systems, mobile devices, etc.

[0027] Traditional voice control methods often rely on predefined command sets and fixed grammatical structures, and are difficult to handle complex natural language expressions and emerging control requirements. The disclosed embodiments overcome the limitations of traditional natural language processing (NLP) in processing complex language patterns, do not need to be trained for specific tasks, and can handle new or irregular language patterns. The core of this method is to use the powerful ability of a large language model to directly understand the user's intention, without involving methods such as intention recognition in traditional NLP. The user can enter some vague instructions or device aliases or aliases for a group of devices. For example, if there are "air conditioners in the living room on the first floor", "air conditioners in the living room on the second floor", and "air conditioners in the bedroom", the user only needs to give the instruction: "Turn off all air conditioners in the living room", and the system can infer from the context that the user's actual intention is to turn off "air conditioners in the living room on the first floor" and "air conditioners in the living room on the second floor", without actually writing code analysis statements, parsing the subject, predicate, and object of the statements, and without the user actually customizing the scene to actively set the relevant equipment. In a multi-language scenario, there is no need to train for a specific language. The user can directly give instructions in the corresponding language, such as "Open all ACs in livingrooms", and the devices and locations that have been entered in the system can also be identified. A large language model is a deep learning model trained with massive text data that has the ability to understand and generate human language. It can capture the complex semantics and contextual relationships of language, thereby better understanding the user's natural language input.

[0028] The implementation process of the disclosed embodiment includes three main steps: voice collection and conversion, dynamic prompt word example construction, and input text to fill in the prompt word. These steps together constitute a complete voice control process, from user voice input to the final device control instruction generation.

[0029] like Figure 1 As shown, at step S101, the user voice signal is collected, and the voice signal is preprocessed to obtain the corresponding text through voice recognition. This includes: collecting the user voice signal, the sampling rate of the voice signal is 16kHz, and the bit depth is 16bit; preprocessing the voice signal, including: using a high-pass filter to filter out low-frequency noise below 50Hz; using an adaptive noise elimination algorithm to eliminate background noise; performing voice activity detection to extract valid voice segments; inputting the preprocessed voice signal into a deep learning-based voice recognition model to obtain the corresponding text recognition result.

[0030] Specifically, the sampling rate of the collected speech signal is 16kHz and the bit depth is 16bit. This sampling parameter setting can ensure that the collected speech signal has sufficient quality and details to support the subsequent speech recognition process. Next, the collected speech signal is preprocessed. The preprocessing process includes using a high-pass filter to filter out low-frequency noise below 50Hz, which helps to remove low-frequency interference in the environment; using an adaptive noise removal algorithm to eliminate background noise and improve the clarity of the speech signal; performing voice activity detection to extract valid speech segments and avoid processing invalid silent parts. The purpose of these preprocessing steps is to improve the quality of the speech signal and provide better input for subsequent speech recognition. After the preprocessing is completed, the processed speech signal is input into a speech recognition model based on deep learning. This model usually adopts a deep neural network structure, such as a recurrent neural network (RNN) or a convolutional neural network (CNN), which can effectively convert speech signals into corresponding text. The output of the speech recognition model is the text representation of the speech content, which will serve as the input for the subsequent steps.

[0031] For example, in one implementation scenario, the user says: "Turn up the temperature in the living room a little bit", the preprocessed speech signal: [0.11, -0.2, 0.17, ...] (16kHz sampled floating point array), the speech recognition result text: "Turn up the temperature in the living room a little bit".

[0032] At step S102, entropy calculation is performed on the device control instruction set, the attribute set and the attribute value set to obtain one or more key attributes and construct a prompt word example. The purpose of this step is to dynamically generate prompt word examples that adapt to the current control scenario and improve the accuracy of the large language model in understanding and processing user instructions. It includes: obtaining a set of device control instructions, each of which includes multiple attributes; determining a target value range for each attribute; calculating the entropy value of the device control instruction; calculating the entropy value of each attribute; calculating the entropy value of each attribute value; selecting at least one key attribute based on the information gain principle; constructing a prompt word in JSON format containing the selected key attribute; adding examples to the constructed JSON format prompt word.

[0033] Specifically, first, a set of device control instructions is obtained, where each device control instruction includes multiple attributes. These attributes can be the function, state, parameters, etc. of the device. For example, for air conditioning control, the attributes may include temperature, mode, wind speed, etc. Then, for each attribute, its target value range is determined, which helps the subsequent entropy calculation and information gain analysis.

[0034] Then, the method calculates the entropy of the device control instruction, the entropy of each attribute, and the entropy of each attribute value. Entropy is an indicator of uncertainty in information theory. By calculating these entropy values, the information content of different instructions, attributes, and attribute values ​​can be quantified. Based on the information gain principle, at least one key attribute is selected. Information gain indicates the contribution of a certain attribute to distinguishing different control instructions. Selecting the attribute with the largest information gain as the key attribute can minimize the uncertainty of instruction recognition.

[0035] Further, the device control instruction set D = {d1, d2, ..., dn}, where di represents the ith device control instruction. The attribute set Ai of each instruction di = {ai1, ai2, ..., aim}, where aij represents the jth attribute of the ith instruction. The attribute value set Vij = {vij1, vij2, ..., vijk}, where vijk represents the kth possible value of the jth attribute of the ith instruction.

[0036] The entropy value of the device control instruction is calculated, including: Where H(D) represents the entropy of the instruction set D, d i represents the ith instruction in the instruction set D, n represents the total number of instructions, p(d i ) indicates instruction d i frequency of use.

[0037] Among them, the entropy value of each attribute is calculated, including: Among them, H(A i) represents the attribute set A of the i-th instruction i The entropy of ij represents the jth attribute of the ith instruction, m represents the total number of attributes, and p(a ij ) represents attribute a ij frequency of use.

[0038] The entropy value of each attribute value is calculated, including: Among them, H(V ij ) represents the attribute value set V of the jth attribute of the ith instruction ij The entropy of v ijk represents the kth value of the jth attribute of the ith instruction, where k represents the total number of attribute values, p(v ijk ) indicates the attribute value v ijk frequency of use.

[0039] Next, calculate the information gain: IG(D,Ai) = H(D) - H(D|Ai), where H(D|Ai) is the conditional entropy given the attribute Ai. Sort the attributes according to the information gain and select the top-k attributes as the key attributes in the prompt word example.

[0040] Based on the selected key attributes, JSON format prompts are constructed. JSON (JavaScript Object Notation) is a lightweight data exchange format that is easy for humans to read and write, and easy for machines to parse and generate. Using JSON format, the various attributes and values ​​of control instructions can be structured, which is convenient for large language models to process. Examples are also added to the constructed JSON format prompts. These examples can help the large language model better understand the context of the current control scenario and the expected output format.

[0041] The JSON structure for constructing the dynamic prompt word example may include but is not limited to the following text:

[0042]

[0043]

[0044] In addition, in order to adapt to changes in user habits and device functions, the disclosed embodiment also includes: regularly updating the usage frequency of device control instructions, attributes, and attribute values; recalculating entropy and information gain based on the updated usage frequency, and dynamically adjusting key attributes and examples in the JSON format prompt words. This dynamic adjustment mechanism enables the system to continuously optimize and improve the recognition efficiency of commonly used user instructions.

[0045] The construction process will be further described below in conjunction with a specific embodiment. In an implementation scenario, there is an air conditioning control instruction set D = {temperature control, mode control, wind speed control};

[0046] The attribute set of each instruction: A1 (temperature control) = {temperature, room}, A2 (mode control) = {mode, room}, A3 (wind speed control) = {wind speed, room};

[0047] Attribute value set: V11 (temperature) = {16, 17, ..., 30}, V12 (room) = {living room, bedroom, study}, V21 (mode) = {cooling, heating, dehumidification, automatic}, V31 (wind speed) = {low speed, medium speed, high speed, automatic}.

[0048] Calculate the instruction entropy and set the usage frequency to: p(temperature control)=0.5, p(mode control)=0.3, p(wind speed control)=0.2, H(D)=-(0.5*log2(0.5)+0.3*log2(0.3)+0.2*log2(0.2))≈1.485bits.

[0049] Calculate the attribute entropy, taking the temperature control instruction as an example: Assume that p(temperature) = 0.7, p(room) = 0.3, H(A1) = -(0.7*log2(0.7)+0.3*log2(0.3))≈0.881 bits.

[0050] Calculate the attribute value entropy, taking the temperature attribute as an example: Assume that the temperature value usage frequency is uniformly distributed, and the probability of each temperature value is 1 / 15, H(V11) = -15*(1 / 15*log2(1 / 15))≈3.907 bits.

[0051] Calculate the information gain, taking the temperature attribute as an example: assuming that after a given temperature, the conditional entropy of the instruction H(D|temperature)=0.5, IG(D, temperature)=H(D)-H(D|temperature)=1.485-0.5=0.985 bits.

[0052] The attributes are sorted according to information gain, and the top-k attributes are selected as the key attributes in the prompt word example. The sorting results are: 1. Temperature (0.985 bits), 2. Mode (0.8 bits), 3. Room (0.6 bits), 4. Wind speed (0.4 bits).

[0053] Then construct the JSON structure of the dynamic prompt word example:

[0054]

[0055]

[0056] For dynamically updated prompt word examples, the user command usage can be counted every day, the usage frequency of each command, attribute and attribute value can be updated, the entropy and information gain can be recalculated every week, and the key attribute order and examples in the prompt word example can be updated according to the new calculation results. For example, if the frequency of wind speed control is found to increase significantly after one week: p (temperature control) = 0.4, p (mode control) = 0.3, p (wind speed control) = 0.3. After recalculation, the information gain ranking can be changed to: 1. Temperature (0.95 bits), 2. Wind speed (0.85 bits), 3. Mode (0.75 bits), 4. Room (0.6 bits). The attribute order and examples in the prompt word example are updated accordingly.

[0057] It should be noted that the dynamic construction of the prompt word example in the embodiment of the present disclosure lies in the selection of its instructions, attributes and attribute values, and is not limited to the structure of the JSON text itself. The structure containing dynamically selected instructions, attributes and attribute values ​​can be constructed according to the needs and application scenarios. In another embodiment, the JSON text can also be as follows:

[0058]

[0059]

[0060]

[0061] In step S103, a complete prompt word is constructed by combining the corresponding text and the prompt word example, and input into the large language model to obtain the device control instruction. This includes: obtaining the text result output by the speech recognition module, and using the text result as the user input information; constructing a multi-round dialogue structure including the system role definition, user input information, assistant response and instruction set, and generating a complete prompt word for semantic analysis; transmitting the complete prompt word to the preset large language model, triggering the large language model to perform semantic analysis operations; receiving the output data of the large language model, wherein the output data is a standardized device control instruction in JSON format.

[0062] Specifically, first, obtain the text result output by the speech recognition module and use it as user input information. This text result is the output of the first step of speech recognition and represents the user's original voice command. Next, build a multi-round dialogue structure that includes system role definition, user input information, assistant response, and command set to generate complete prompt words for semantic parsing. The multi-round dialogue structure can simulate the context of human dialogue and help the large language model understand user intent more accurately.

[0063] The constructed complete prompt words are transmitted to the preset large language model, triggering the model to perform semantic parsing operations. Based on its trained massive data and deep learning algorithm, the large language model can understand the complex semantics and contextual relationships of natural language, thereby accurately parsing the user's control intentions. Finally, the output data of the large language model is received, where the output data is a standardized device control instruction in JSON format. This standardized JSON format output facilitates subsequent device control system parsing and execution.

[0064] In one embodiment, the user input may be converted into a JSON structure as shown below:

[0065]

[0066] Then, the voice command JSON text is merged with the prompt word example JSON text described in the previous embodiment. After the merger, a multi-round dialogue structure can be constructed based on it, for example:

[0067]

[0068] Next, the constructed prompt word is sent to the large language model (such as GPT-3.5 or GPT-4) for processing. The output result of the large language model is received, which can be in standard JSON format and contains the parsed device control instructions. For example, if device_ac_ctrl is set to air conditioning, mode1 is cooling, mode2 is heating, mode4 is ventilation, switch1 is on, switch2 is off, and the temperature is a number, in one embodiment, the return of GPT can be as follows:

[0069]

[0070] It can be obtained that the user wants to control the device_ac_ctrl air conditioner, located in the living room on the first floor and the living room on the second floor, the mode is 4, that is, ventilation, the switch is on, and the temperature is 30 degrees Celsius. These instructions can be sent to the device later.

[0071] In summary, the disclosed embodiment constructs a general JSON format prompt word example through entropy calculation, and realizes dynamic adaptive voice control. Compared with the traditional method, the disclosed embodiment does not need to be trained for a specific task, and can flexibly handle various language patterns, including new or irregular expressions. The advantage of this method is its versatility and scalability. It can adapt to different types of devices and control scenarios, and supports multi-language input to meet the needs of global applications.

[0072] In addition, the dynamic prompt word example construction mechanism of the embodiment of the present disclosure can automatically adjust according to user habits and device characteristics, improving the adaptability and accuracy of the system. By utilizing the powerful capabilities of the large language model, the embodiment of the present disclosure can directly understand the user's intention, avoiding the complex intention recognition and slot filling process in the traditional NLP method, thereby improving processing efficiency and response speed.

[0073] Figure 2 A system schematic block diagram of a voice control device based on a large language model provided in an embodiment of the present disclosure. Specifically, an embodiment of the present disclosure discloses a system of a voice control device based on a large language model, which is used to execute any of the methods described above, and the system includes: a collection unit 201, a recognition unit 202, and an instruction unit 203; wherein the collection unit 201 is used to collect user voice signals; the recognition unit 202 is used to pre-process the voice signals to obtain corresponding texts through voice recognition; the instruction unit 203 is used to obtain one or more key attributes by performing entropy calculation on a device control instruction set, an attribute set, and an attribute value set, and construct a prompt word example; a complete prompt word is constructed in combination with the corresponding text and the prompt word example, and the large language model is input to obtain the device control instruction.

[0074] It should be understood that the systems shown in the figure are exemplary and not restrictive. This means that the system architecture involved is not limited to a specific form or design, but is presented as an example. In other words, the architecture shown in the figure can be regarded as a way of expression to clearly describe related concepts and relationships, and does not exclude other forms of architecture. Therefore, when interpreting the architecture in the picture, it should be understood that the model is flexible and diverse, and its purpose is to provide an exemplary description rather than a restrictive provision on a specific form. In one embodiment (such as Figure 3 As shown in Figure 3, GPT can be used as the core of parsing intent to build a control architecture.

[0075] Figure 3 FIG. 1 is a schematic block diagram of a system architecture of a voice control device based on a large language model according to an embodiment of the present disclosure. Figure 3 As shown in the figure, the user's voice is first recorded from the console, and then converted into text through the STT service. The middle-end service generates prompt words and then gives the text to GPT to identify the user's true intention, and finally sends the device command to the device control service. After completing the control of the device, the text template is returned based on the results of the device control and given to the TTS service to generate voice and play it by the console.

[0076] Due to its pre-trained nature, the GPT model can better understand and generate diverse language patterns, while traditional NLP may rely more on rules and templates. The GPT model is able to capture long-distance dependencies and contextual information, while some traditional NLP methods may not perform well when dealing with long conversations or complex contexts. The GPT model can perform well on a variety of language tasks through pre-training, while traditional NLP systems may need to be trained for specific tasks. Due to its large-scale pre-training, the GPT model has good generalization capabilities and can handle unseen language patterns, while traditional NLP systems may not perform well when faced with new or irregular language patterns. The GPT model can generate responses quickly, while some complex NLP pipelines may require more computing resources and time.

[0077] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device and a readable storage medium.

[0078] Figure 4 A schematic block diagram of an electronic device 300 that can be used to implement an embodiment of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present disclosure described and / or required herein.

[0079] The electronic device 300 includes a computing unit 301, which can perform various appropriate actions and processes according to a computer program stored in a ROM 302 or a computer program loaded from a storage unit 308 into a RAM 303. In the RAM 303, various programs and data required for the operation of the electronic device 300 can also be stored. The computing unit 301, the ROM 302, and the RAM 303 are connected to each other via a bus 304. An I / O interface 305 is also connected to the bus 304.

[0080] A number of components in the electronic device 300 are connected to the I / O interface 305, including: an input unit 306, such as a keyboard, a mouse, etc.; an output unit 307, such as various types of displays, speakers, etc.; a storage unit 308, such as a disk, an optical disk, etc.; and a communication unit 309, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 309 allows the electronic device 300 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0081] The computing unit 301 may be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 301 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units running machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 301 performs the various methods and processes described above, such as the method of a voice control device based on a large language model. For example, in some embodiments, the method of a voice control device based on a large language model may be implemented as a computer software program, which is tangibly contained in a machine-readable medium, such as a storage unit 308. In some embodiments, part or all of the computer program may be loaded and / or installed on the electronic device 300 via the ROM 302 and / or the communication unit 309. When the computer program is loaded into the RAM 303 and executed by the computing unit 301, one or more steps of the method of a voice control device based on a large language model described above may be performed. Alternatively, in other embodiments, the computing unit 301 may be configured to execute the method for controlling a device by voice based on a large language model in any other appropriate manner (for example, by means of firmware).

[0082] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0083] The program code for implementing the method of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that the program code, when executed by the processor or controller, enables the functions / operations specified in the flow chart and / or block diagram to be implemented. The program code may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

[0084] In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, device, or equipment. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0085] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0086] The systems and techniques described herein may be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), and the Internet.

[0087] A computer system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The relationship of client and server is generated by computer programs running on respective computers and having a client-server relationship with each other. The server may be a cloud server, a server of a distributed system, or a server combined with a blockchain.

[0088] It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps recorded in this disclosure can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved, and this document does not limit this.

[0089] The above specific implementations do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present disclosure shall be included in the protection scope of the present disclosure.

Claims

1. A method for voice control of a device based on a large language model, characterized in that: include: Collecting user voice signals and preprocessing the voice signals to obtain corresponding text through voice recognition; By performing entropy calculation on the device control instruction set, the attribute set and the attribute value set, one or more key attributes are obtained, and a prompt word example is constructed; The corresponding text and prompt word examples are combined to build a complete prompt word, and then input into the large language model to obtain device control instructions.

2. The method according to claim 1, characterized in that in, Collecting user voice signals and preprocessing the voice signals to obtain corresponding text through voice recognition, including: Collecting user voice signals, wherein the sampling rate of the voice signals is 16kHz and the bit depth is 16bit; Preprocessing the speech signal includes: using a high-pass filter to filter out low-frequency noise below 50 Hz; using an adaptive noise elimination algorithm to eliminate background noise; performing speech activity detection to extract valid speech segments; The preprocessed speech signal is input into the speech recognition model based on deep learning to obtain the corresponding text recognition result.

3. The method according to claim 1, characterized in that in, By performing entropy calculation on the device control instruction set, attribute set, and attribute value set, one or more key attributes are obtained and prompt word examples are constructed, including: Obtain a device control instruction set, where each device control instruction includes multiple attributes; Determine the target value range for each attribute; Calculate the entropy value of the device control instructions; Calculate the entropy value of each attribute; Calculate the entropy value of each attribute value; Select at least one key attribute based on the information gain principle; Construct a prompt word in JSON format containing the selected key attributes; Add examples to the built JSON format prompt.

4. The method according to claim 3, characterized in that Among them, it also includes: Regularly update the usage frequency of device control instructions, attributes, and attribute values; The entropy value and information gain are recalculated according to the updated usage frequency, and the key attributes and examples in the JSON format prompt words are dynamically adjusted.

5. The method according to claim 1, characterized in that: in, Combine the corresponding text and prompt word examples to build a complete prompt word and input it into the large language model to obtain device control instructions, including: Obtaining the text result output by the speech recognition module, and using the text result as user input information; Construct a multi-round dialogue structure that includes system role definition, user input information, assistant response and command set, and generate complete prompt words for semantic parsing; Transmitting the complete prompt word to a preset large language model, triggering the large language model to perform a semantic parsing operation; Output data of the large language model is received, wherein the output data is a standardized device control instruction in a JSON format.

6. The method according to claim 3, characterized in that in, Calculate the entropy value of device control instructions, including: Where H(D) represents the entropy of the instruction set D, d i represents the ith instruction in the instruction set D, n represents the total number of instructions, p(d i ) indicates instruction d i frequency of use.

7. The method according to claim 3, characterized in that in, Calculate the entropy value of each attribute, including: Among them, H(A i ) represents the attribute set A of the i-th instruction i The entropy of ij represents the jth attribute of the ith instruction, m represents the total number of attributes, and p(a ij ) represents attribute a ij frequency of use.

8. The method according to claim 3, characterized in that in, Calculate the entropy value for each attribute value, including: Among them, H(V ij ) represents the attribute value set V of the jth attribute of the ith instruction ij The entropy of v ijk represents the kth value of the jth attribute of the ith instruction, where k represents the total number of attribute values, p(v ijk ) indicates the attribute value v ijk frequency of use.

9. A system for voice control of a device based on a large language model, for executing the method according to any one of claims 1 to 8, characterized in that: include: Collection unit, recognition unit and instruction unit; wherein, The collection unit is used to collect user voice signals; The recognition unit is used to pre-process the speech signal to obtain the corresponding text through speech recognition; The instruction unit is used to obtain one or more key attributes and construct prompt word examples by performing entropy calculation on a device control instruction set, an attribute set and an attribute value set; construct a complete prompt word in combination with the corresponding text and the prompt word example, and input the complete prompt word into a large language model to obtain the device control instruction.

10. An electronic device, comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1 to 8.

Images