Home control method and device based on large model knowledge enhancement and electronic equipment

Abstract

The invention discloses a home control method and device based on large model knowledge enhancement and electronic equipment. The scheme comprises the following steps: acquiring an initial audio; performing conversion operation on the initial audio to obtain a target text; according to the target text, calling a large language model to obtain a control intention judgment result; when the control intention judgment result is a smart home control intention, obtaining a control instruction according to the target text; and controlling the target smart home according to the control instruction to obtain a control result. The method can improve the accuracy of hit of the smart home control intention, and can be widely applied to the technical field of data processing.

Description

Technical Field

[0001] The present invention relates to the field of data processing technology, and in particular to a home control method, device and electronic equipment based on large model knowledge enhancement. Background Art

[0002] The voice AI platform uses smart remote controllers and speakers as the main application terminals, with voice search for live broadcasts, on-demand content, and playback, and voice control of smart home devices as the core business scenarios. At the same time, it realizes voice interaction of Tianyi high-definition value-added applications and skills, creating a nationwide intensive voice AI platform to provide users with a more efficient and natural interactive experience, and support the nationwide implementation and large-scale development of smart remote controllers and smart speakers. However, the accuracy of users' intelligent intent depends on the size of the pre-annotated home field data set. The current AI platform is in the construction and large-scale development stage, with fewer annotated data sets, and the data set source is still supplemented by manual enumeration, resulting in the fact that the voice commands that users can successfully hit to control smart home devices are relatively fixed and less flexible. Summary of the invention

[0003] In view of this, the main purpose of the embodiments of the present invention is to provide a home control method, device and electronic device based on large model knowledge enhancement, in order to solve at least one problem of the prior art and improve the accuracy of control intention hits.

[0004] To achieve the above object, an embodiment of the present invention provides a home control method based on large model knowledge enhancement, the method comprising the following steps:

[0005] Collect initial audio;

[0006] Performing a conversion operation on the initial audio to obtain a target text;

[0007] According to the target text, calling a large language model to obtain a control intention discrimination result;

[0008] When the control intention determination result is a smart home control intention, obtaining a control instruction according to the target text;

[0009] According to the control instruction, the target smart home is controlled to obtain a control result.

[0010] In some embodiments, converting the initial audio to obtain the target text comprises the following steps:

[0011] Converting the initial audio into a target audio in the form of binary stream bytes;

[0012] The target audio is converted into the target text through speech recognition technology.

[0013] In some embodiments, the step of calling a large language model to obtain a control intention identification result according to the target text includes the following steps:

[0014] constructing an initial prompt word according to the target text;

[0015] Sending an inference request to the large language model according to the initial prompt word;

[0016] In response to the inference request, performing a vectorization operation on the initial prompt word through the large language model to obtain a target prompt word;

[0017] The control intention discrimination result is obtained according to the target prompt word.

[0018] In some embodiments, obtaining the control intention identification result according to the target prompt word includes the following steps:

[0019] According to the target prompt word, similarity judgment is performed on the smart home control intention to obtain the control intention judgment result;

[0020] The control intention identification result includes that the target prompt word is the smart home control intention and the target prompt word is not the smart home control intention.

[0021] In some embodiments, when the control intention determination result is a smart home control intention, obtaining a control instruction according to the target text includes the following steps:

[0022] Performing entity extraction operation on the target text to obtain entity extraction result;

[0023] The entity extraction result is normalized to obtain the control instruction.

[0024] In some embodiments, controlling the target smart home according to the control instruction to obtain a control result includes the following steps:

[0025] Sending the control instruction to the target smart home;

[0026] In response to the control instruction, the control task of the target smart home is executed, and the control result is obtained and displayed.

[0027] To achieve the above object, another aspect of an embodiment of the present invention provides a home control device based on large model knowledge enhancement, the device comprising:

[0028] The first module is used to collect initial audio;

[0029] The second module is used to convert the initial audio to obtain a target text;

[0030] The third module is used to call the large language model to obtain the control intention discrimination result according to the target text;

[0031] The fourth module is used to obtain a control instruction according to the target text when the control intention determination result is a smart home control intention;

[0032] The fifth module is used to control the target smart home according to the control instruction to obtain the control result.

[0033] To achieve the above-mentioned purpose, another aspect of an embodiment of the present invention provides an electronic device, which includes a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, it implements the aforementioned home control method based on large model knowledge enhancement.

[0034] To achieve the above-mentioned purpose, another aspect of an embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the home control method based on large model knowledge enhancement as described above is implemented.

[0035] To achieve the above object, another aspect of an embodiment of the present invention provides a computer program product or a computer program, the computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the aforementioned home control method based on large model knowledge enhancement.

[0036] The embodiments of the present invention include at least the following beneficial effects: The present invention provides a home control method, device and electronic device based on large model knowledge enhancement, the scheme collects initial audio; converts the initial audio to obtain a target text; calls a large language model to obtain a control intention discrimination result according to the target text; when the control intention discrimination result is a smart home control intention, obtains a control instruction according to the target text; controls the target smart home according to the control instruction to obtain a control result, thereby improving the accuracy of the smart home control intention hit. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0038] Figure 1 It is a schematic diagram of an implementation environment of home control based on large model knowledge enhancement provided by an embodiment of the present invention;

[0039] Figure 2 is a flow chart of a home control method based on large model knowledge enhancement provided by an embodiment of the present invention;

[0040] Figure 3 The embodiment of the present invention provides Figure 1 Flow chart of step S200 in FIG.

[0041] Figure 4 The embodiment of the present invention provides Figure 1 Flow chart of step S300 in FIG.

[0042] Figure 5 The embodiment of the present invention provides Figure 4 Flow chart of step S340 in FIG.

[0043] Figure 6 The embodiment of the present invention provides Figure 4 Flow chart of step S400 in FIG.

[0044] Figure 7 The embodiment of the present invention provides Figure 4 Flow chart of step S500 in FIG.

[0045] Figure 8 This is a flow chart of a user's voice control of a smart home device provided by an embodiment of the present invention;

[0046] Fig. 9 is a structural schematic diagram of a home control device based on large model knowledge enhancement provided by an embodiment of the present invention;

[0047] Fig.10 It is a schematic diagram of the hardware structure of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0048] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the embodiments of the present invention, and they are only examples of devices and methods consistent with some aspects of the embodiments of the present invention as detailed in the attached claims.

[0049] It should be noted that, although the functional modules are divided in the system schematic diagram and the logical order is shown in the flow chart, in some cases, the steps shown or described may be performed in a different order than the module division in the system or the flow chart. The terms "first / S100" and "second / S200" in the specification and claims and the above-mentioned drawings may be used to describe various concepts in this article, but unless otherwise specified, these concepts are not limited by these terms. These terms are only used to distinguish one concept from another. For example, without departing from the scope of the embodiment of the present invention, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the words "if" and "if" as used herein may be interpreted as "at the time of" or "when" or "in response to determination".

[0050] The terms "at least one", "multiple", "each", "any", etc. used in the present invention, at least one includes one, two or more, multiple includes two or more, each refers to each of the corresponding multiple, and any refers to any one of the multiple.

[0051] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as those commonly understood by those skilled in the art to which the present invention belongs. The terms used herein are only for the purpose of describing the embodiments of the present invention and are not intended to limit the present invention.

[0052] Before describing the embodiments of the present invention in detail, some nouns and terms involved in the embodiments of the present invention are first described. The nouns and terms involved in the embodiments of the present invention are subject to the following explanations.

[0053] Speech recognition technology (Automatic Speech Recognition, ASR): Its goal is to convert the vocabulary content in human speech into computer-readable input, such as keystrokes, binary codes or character sequences. It is different from speaker recognition and speaker verification, which try to identify or confirm the speaker of the speech rather than the vocabulary content contained therein.

[0054] Large Language Model (LLM): refers to a deep learning model trained with a large amount of text data that can generate natural language text or understand the meaning of language text. Large language models can handle a variety of natural language tasks, such as text classification, question answering, and dialogue.

[0055] At present, the accuracy of users' voice control of smart homes depends on the size of the pre-labeled home-related data set, and the judgment of smart home control intentions depends on the pre-labeling of the home control data set. The larger the data set, the stronger the generalization ability of home control intentions. However, the current AI platform is in the construction and large-scale development stage. It is difficult to obtain such data in vertical industries. The sorted training sets rely on manual enumeration and the data sets are small. This requires that the user's voice control commands must be limited to the pre-sorted training set range in order to successfully control, resulting in relatively fixed voice commands that users can use to successfully control smart home devices, and low flexibility.

[0056] In view of this, an embodiment of the present invention provides a home control method, device and electronic device based on big model knowledge enhancement. The solution provides smart home intent classification capability through a home knowledge graph formed by big model pre-training, and achieves correct home control intent hits on user audio texts without the need for post-operation personnel to annotate data, thereby achieving control of smart home devices, enhancing the generalization capability of user home control audio, improving the accuracy of intent hits, and reducing the workload of manual intervention in operations.

[0057] It is understandable that the home control method based on large model knowledge enhancement provided in the embodiment of the present invention can be applied to any computer device with data processing and computing capabilities, and this computer device can be various terminals or servers. When the computer device in the embodiment of the present invention is a server, the server is an independent physical server, or a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN (Content De l ivery Network, content distribution network), and big data and artificial intelligence platforms. Optionally, the terminal is a smart phone, a tablet computer, a laptop computer, a desktop computer, etc., but is not limited to this.

[0058] It should be further explained that the terminals involved in the embodiments of the present invention include but are not limited to smart phones, computers, intelligent voice interaction devices, smart home appliances, vehicle terminals, aircraft, etc. The embodiments of the present invention can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc.

[0059] In some possible implementations, a computer program capable of implementing the home control method based on large model knowledge enhancement provided by an embodiment of the present invention may be deployed for execution on a computer device, or on multiple computer devices located at one location, or on multiple computer devices distributed at multiple locations and interconnected through a communication network. Multiple computer devices distributed at multiple locations and interconnected through a communication network may form a blockchain system.

[0060] Based on the fact that multiple computer devices can form a blockchain system, the computer device that implements the home control method based on large model knowledge enhancement in the embodiment of the present invention can be a node in the blockchain system. The node or the node corresponding to other devices in the blockchain can also store control results, target texts, and other intermediate data obtained in the home control process based on large model knowledge enhancement, etc.

[0061] like Figure 1 FIG. 1 is a schematic diagram of an implementation environment provided by an embodiment of the present invention. Figure 1 , the implementation environment includes at least one terminal 101 and a server 102. The terminal 101 and the server 102 can be connected to the network wirelessly or wired to complete data transmission and exchange. The terminal 101 can be provided with functions such as large model training and supervised adjustment of large model training parameters to adapt to downstream task requirements, or the terminal 101 communicates with the server 102 interactively, and implements relevant functions adapted to downstream task requirements based on the corresponding program logic provided by the server 102. Optionally, the server 102 is a background server of the program logic or a cloud server that provides cloud computing and cloud storage services. The terminal 101 can form a home knowledge graph through large model pre-training, provide smart home intention classification capabilities, can be applied to various downstream tasks of home control, and can further perform corresponding data processing based on the control results according to the needs of downstream tasks (including but not limited to data display or storage of relevant control results, control intention discrimination results, target texts, etc.). Among them, the terminal 101 is a computer, mobile phone, tablet computer, smart speaker or other terminal.

[0062] In some embodiments, the server can obtain the initial audio and the target text of the initial audio from the terminal, and obtain the smart home control intent corresponding to the target text to train the big model. The server can send the pre-trained big model to the terminal, and the terminal performs various downstream tasks of home control based on the pre-trained big model. Optionally, the model training method can be applied to various scenarios.

[0063] Exemplarily, in the scenario of smart home control using initial audio: after obtaining authorization from the target object, the initial audio information provided by the target object is obtained and sent to the server 102 through the terminal 101. The server 102 uses the aforementioned method to obtain the pre-trained large model and sends it to the terminal 101. In this way, the terminal 101 can use the large model to control the target smart home and generate control results. It can be understood that the above application scenarios are only examples. In actual applications, in addition to using the server to pre-train the large model, other devices with large model training capabilities, such as terminals, can also be used to pre-train and construct a large model for smart home control; in addition, other devices other than terminal devices can also be used to carry the pre-trained large model for smart home control. No limitation is made to the application scenarios of the model training method provided in the embodiments of the present invention.

[0064] In some embodiments, the terminal can store the control results, control intention determination results, target text, and data generated by the intermediate processing process, and can also display them in a visual manner, or the terminal can upload the control results, control intention determination results, target text, and data generated by the intermediate processing process to the server for storage.

[0065] It should be noted that in each specific implementation of the present invention, when it comes to obtaining the feature data, behavior data, historical data, location information and other data related to the identity or characteristics of the object, the permission or authorization of the object will be obtained first, and the collection, use and processing of these data will comply with the relevant laws, regulations and standards of the relevant countries and regions. In addition, when the embodiment of the present invention needs to obtain the sensitive personal information of the object, it will obtain the separate permission or consent of the object through a pop-up window or jump to a confirmation page, voice command confirmation, etc. After clearly obtaining the separate permission or consent of the object, the necessary object-related data for the normal operation of the embodiment of the present invention will be obtained.

[0066] like Figure 2 As shown, it is a flowchart of a home control method based on large model knowledge enhancement provided by an embodiment of the present invention. The execution subject of the home control method based on large model knowledge enhancement can be any of the aforementioned computer devices. Figure 2The method may include but is not limited to the following steps:

[0067] Step S100: collecting initial audio.

[0068] In the embodiment of the present invention, the initial audio refers to the audio that needs to be identified for control intent, and the corresponding control instructions are obtained through the initial audio, thereby realizing the control of the target smart home. The collected initial audio can be in different languages, dialects, etc. When processing audio data, it is necessary to ensure that the user's privacy is protected to avoid unauthorized data access.

[0069] In some embodiments, the initial audio input by the user in real time through a microphone or recorded by other audio input devices can be collected, or the initial audio input by the user through text-to-speech can be collected. Exemplarily, the initial audio is collected in real time from the microphone installed on the terminal through the target application installed on the terminal, or the initial audio recorded and input by the user is read through other audio input devices, or the initial audio input through text-to-speech is read, etc., which is not limited here.

[0070] Step S200, converting the initial audio to obtain a target text.

[0071] In the embodiment of the present invention, the target text is obtained by converting the initial audio, and the target text may include but is not limited to speech transcription text, intonation features, speech rate features and emotional features. It is understandable that speech transcription text is the process of converting an audio signal into written language, which involves converting a sound waveform into a readable text form, which is usually achieved through automatic speech recognition (ASR) technology. Intonation features describe the intonation and pitch changes of speech, which can reflect the speaker's emotions and emphasis, for example, high notes may indicate surprise or excitement, and low notes may indicate seriousness or sadness. The speech rate feature involves the speed of speaking, that is, the number of words spoken per unit time, which can provide clues about the speaker's nervousness or excitement. The emotional feature is to identify the speaker's emotional state based on the pitch, rhythm and intensity of the voice, such as happiness, sadness, anger or neutrality. Emotional features are very important for understanding the speaker's intentions and emotional background, especially in interactive voice applications. By analyzing these target text features, voice data can be understood and processed more accurately, thereby improving the accuracy of speech recognition and the user experience of the interactive system. For example, by identifying intonation features, the system can distinguish whether the user is asking a question or issuing a command. By analyzing speech rate and emotional features, the system can better understand the user's urgency or emotional state and make a more appropriate response. The comprehensive analysis of these features helps build a smarter and more natural voice interaction system.

[0072] In some embodiments, audio in multiple languages ​​or dialects can be correctly converted and recognized, and audio can be accurately transcribed in multi-person conversations or background noise. For example, in a multi-language or dialect environment, an automatic speech recognition (ASR) system pre-trained with different language and dialect features is used to achieve multi-language and multi-dialect mixed recognition, so that users do not need to define dialects in advance, and the model can be automatically adapted to improve the accuracy and applicability of speech recognition.

[0073] Step S300: Based on the target text, a large language model is called to obtain a control intention identification result.

[0074] In some embodiments, the control intention discrimination result may be that the discrimination object is a smart home control intention, or that the discrimination object is not a smart home control intention. Optionally, the possible smart home control intention corresponding to the discrimination object may be one or more, and these smart home control intentions usually have a corresponding confidence or similarity score, indicating the degree of match between the discrimination object and the smart home control intention. If the large model returns a single intention with a high enough confidence or multiple non-contradictory intentions with a high enough confidence, the corresponding smart home control command can be executed in subsequent operations. If the confidence is not high or there are multiple possible contradictory intentions (such as turning lights on and off, turning up and down the volume, etc.), the user can be requested to provide more information, or a set of predefined commands can be provided for the user to choose, to ensure that the user can quickly and accurately control the smart home device.

[0075] Step S400: When the control intention determination result is a smart home control intention, a control instruction is obtained according to the target text.

[0076] In some embodiments, the control intention discrimination result is obtained by calling a large language model, and the control intention discrimination result can be that the discrimination object is a smart home control intention, or that the discrimination object is not a smart home control intention. When the control intention discrimination result is a smart home control intention, an operation of obtaining control instructions according to the target text is performed. Optionally, it is also possible to determine whether to perform an operation of obtaining control instructions according to the target text in combination with the corresponding confidence or similarity score of the smart home control intention. Exemplarily, the large model returns a control intention discrimination result that the discrimination object is a single intention with a sufficiently high confidence or multiple non-contradictory intentions with a sufficiently high confidence, then an operation of obtaining control instructions according to the target text is performed.

[0077] Step S500: Control the target smart home according to the control instruction to obtain a control result.

[0078] In some embodiments, after obtaining the control instruction, it can be sent to the target smart home device through a wireless network (such as Wi-Fi, ZigBee, etc.), so as to control the target smart home and return the control result. After receiving the control instruction, the target smart home device will perform the corresponding operation according to the instruction. For example, if the instruction is to turn on the light in the living room, the smart light bulb will receive and execute this command and change its state to on. After the device executes the control instruction, in order to ensure that the user can understand the state change of the device in time, the system will design an effective result feedback mechanism, which may include real-time push notifications, instant updates on the interface, etc. The feedback content may include the new state of the device, such as the light is turned on, the air conditioner has been adjusted to a specified temperature, etc. The embodiment of the present invention can display the control result through a visual interface, so that the user can intuitively see the current state of the device. This can be achieved through the supporting application or Web interface of the smart home system, providing a user-friendly interactive experience. During the control process, if the device fails to respond as expected or an error occurs, the system needs to be able to handle these abnormal situations and prompt the user through the user interface, which helps to improve the reliability of the system and user experience.

[0079] like Figure 3 As shown, in order to obtain a more accurate target text, improve the accuracy of the control intention hit and accurately control the target smart home, in some feasible embodiments, in the step of converting the initial audio to obtain the target text, reference is made to Figure 3 , which may include but is not limited to steps S110 to S120:

[0080] Step S110, converting the initial audio into a target audio in the form of binary stream bytes.

[0081] In some embodiments, the initial audio input by the user is acquired and converted into an analog signal, which needs to be converted into a digital signal before it can be processed by the computer system. This process is called analog-to-digital conversion, which is performed by an analog-to-digital converter (ADC). The ADC samples the analog signal at a fixed sampling rate and quantizes it into a digital value. Digital signals are usually encoded in a specific audio format, such as PCM (pulse code modulation), MP3, WAV, etc. The encoding process involves compressing the digital signal into a binary stream byte form for easy storage and transmission. The encoded audio data exists in the form of a binary stream, and the target audio can be obtained. This series of bytes can be transmitted to different devices through various interfaces (such as USB, Bluetooth, etc.), or stored on a digital storage medium.

[0082] Step S120: converting the target audio into the target text through speech recognition technology.

[0083] In some embodiments, after receiving the audio data in the form of binary stream bytes, it is first necessary to decode it to restore it to the original digital audio signal. The decoding process is the opposite of the encoding process, and it converts the compressed audio format back to the original audio data format such as PCM. The decoded audio data is fed into the automatic speech recognition system. The ASR system uses complex algorithms and models, such as deep learning networks, to analyze the audio signal and identify the speech patterns therein. The ASR system extracts key features from the audio signal, such as Mel-frequency cepstral coefficients (MFCC), which can represent the key information in the speech signal. Using the extracted speech features, the ASR system converts the speech into text. This process includes recognizing words, phrases, and even entire sentences and converting them into their corresponding text forms. In order to improve the accuracy of recognition, the ASR system can also perform post-processing, such as grammar checking, error correction, etc., to ensure that the converted text is as accurate as possible.

[0084] In some possible embodiments, such as Figure 4 As shown, in step S300, based on the target text, the large language model is called to obtain the control intention discrimination result, which may include but is not limited to the following steps:

[0085] Step S310, constructing an initial prompt word according to the target text;

[0086] In some embodiments, based on the target text converted from the audio, an initial prompt is constructed. The prompt is a specially designed text that can call the large model LLM dialogue interface to request reasoning, and is used to guide the large language model (LLM) to understand and perform specific tasks or generate specific outputs. The initial prompt usually contains key information and context so that the model can accurately understand the user's intentions. The designed prompt can improve the accuracy of the model's understanding of the user's intentions, reduce the time required for model reasoning, and improve overall processing efficiency.

[0087] Step S320, sending an inference request to the large language model according to the initial prompt word;

[0088] In some embodiments, the initial prompt word is sent as input to the large language model. The prompt word, as the core content of the reasoning request, can be sent to the large language model by encapsulating the initial prompt word in an reasoning request using a computer network API. The reasoning request contains all the information of the initial prompt word, as well as other parameters or context data that may be required to ensure that the large language model can correctly understand and process the request. Requests sent through the network API usually use HTTP or HTTPS protocols to ensure the security and reliability of data transmission. By encapsulating the user's intention in the initial prompt word, the system can provide clear and unambiguous instructions to the large language model through the reasoning request. This request-response mode makes the interaction between the system and the large language model more efficient and facilitates the implementation of complex language processing tasks. In addition, various complex language tasks can be processed by the large language model, which facilitates system expansion and improves the flexibility of the system.

[0089] Step S330, in response to the inference request, performing a vectorization operation on the initial prompt word through the large language model to obtain a target prompt word;

[0090] In some embodiments, the target prompt word is the prompt word after the initial prompt word is vectorized, which is used for further processing and analysis within the model. After receiving the reasoning request, the large language model converts the initial prompt word into a vector form. This process is called vectorization. Vectorization is the conversion of text into a numerical representation that the model can process. Vectorization can help the model capture the semantic information of the text and improve the depth of understanding, while the numerical representation allows the model to calculate and compare more efficiently.

[0091] Step S340: obtaining the control intention identification result according to the target prompt word.

[0092] In some embodiments, the target prompt word is used for processing within the large language model to identify and extract the user's control intent. The control intent discrimination result refers to the final judgment of the user's intent returned by the model, which can be used to trigger the corresponding action or response. The advanced large model can provide more accurate intent recognition and reduce erroneous operations. Fast intent recognition can improve user experience and make the system respond more quickly.

[0093] In some embodiments, Figure 5 As shown, in the above step S340, obtaining the control intention discrimination result according to the target prompt word includes the following steps:

[0094] Step S341, performing similarity judgment on the smart home control intention according to the target prompt word to obtain the control intention judgment result.

[0095] In some embodiments, the large language model uses the target prompt word to perform a similarity comparison with the representation of the known smart home control intent. Exemplarily, the similarity or distance between the target prompt word and each smart home control intent is calculated. By comparison, the model can determine which smart home control intent the target prompt word is most similar to, or whether it does not match any known intent. Based on the similarity judgment, the model generates a control intent discrimination result, clearly indicating whether the target prompt word matches the smart home control intent (that is, the target prompt word is the smart home control intent or the target prompt word is not the smart home control intent). Optionally, when the control intent discrimination result is that the target prompt word is the smart home control intent, the target prompt word may correspond to one or more possible smart home control intents, and the degree of matching can be indicated by the corresponding confidence or similarity score of the smart home control intent. The control intent discrimination result will be sent back to the requesting system or device for subsequent processing.

[0096] In some embodiments, if the large model returns a single intent with a high enough confidence or multiple non-contradictory intents with a high enough confidence, the corresponding smart home control command can be executed in subsequent operations. If the confidence is not high or there are multiple possible conflicting intents (such as turning lights on and off, turning up and down the volume, etc.), the user can be requested to provide more information, or a set of predefined commands can be provided for the user to choose, so that the user's voice commands can be understood and responded to more intelligently, providing the user with a more convenient and personalized smart life experience.

[0097] By using a large language model for similarity judgment, the system can more accurately and quickly identify the user's smart home control intentions, respond to the user's instructions in real time, reduce misoperations, and provide timely feedback.

[0098] In some possible embodiments, such as Figure 6 As shown, in step S400, when the control intention determination result is a smart home control intention, a control instruction is obtained according to the target text, which may include but is not limited to the following steps:

[0099] Step S410, performing entity extraction operation on the target text to obtain entity extraction results;

[0100] In some embodiments, after the intention of smart home control is determined, specific entities are extracted from the target text, which may include but are not limited to device names (such as "living room lights"), operating actions (such as "turn on", "turn off"), related parameter settings (such as time settings, temperature settings, brightness levels), etc.

[0101] Step S420, normalizing the entity extraction result to obtain the control instruction.

[0102] In some embodiments, after the entity extraction step, the entity extraction results need to be normalized, the purpose of which is to convert the user's natural language instructions into a format that the system can understand and execute. For example, the user's instruction may be "Please dim the lights in the living room." In the entity extraction step, the system will identify "living room lights" as a device entity and "dim" as an action. In the normalization step, this information will be converted into a clear control instruction, such as {"device":"living_room_l ight","action":"d im","level":"lower"}. Through normalization, it can be ensured that different user expressions can be uniformly understood and can be executed unambiguously by the smart home system.

[0103] In some possible embodiments, such as Figure 7 As shown, in step S500, according to the control instruction, the target smart home is controlled to obtain a control result, which may include but is not limited to the following steps:

[0104] Step S510, sending the control instruction to the target smart home;

[0105] In some embodiments, the normalized control instructions will be sent to the device end, that is, to the target smart home. This step usually involves a network communication protocol, such as MQTT or HTTP, to ensure that the instructions can be transmitted from the central control system to the target device.

[0106] Step S520, in response to the control instruction, executing the control task of the target smart home, and acquiring and displaying the control result.

[0107] In some embodiments, after receiving the control instruction, the target smart home performs corresponding operations according to the instruction (which may include but are not limited to turning on or off the target smart home, adjusting settings, etc.). After the target smart home executes the instruction, it returns a response to confirm that the instruction has been successfully executed. The response may contain detailed information about the execution result, which may indicate that the device has successfully executed the instruction, or may contain error information if the execution fails. And the result of the device control can be displayed on the user's voice terminal or related front-end interface, or voice feedback can be provided to the user. For example, the user issues an instruction to turn on the light in the living room and asks "Is the light in the living room turned on?" The system will not only perform the operation of turning on the light, but also inform the user at the front end that the state of the light has changed or voice feedback "The light in the living room is turned on". By controlling the target smart home through control instructions and feeding back the control results, the accuracy of the instructions, the reliability of execution and the timeliness of user feedback can be ensured, which improves the user experience and ensures the efficient and stable operation of the smart home system.

[0108] The embodiment of the present invention also provides a home control system based on large model knowledge enhancement, which can include three modules: voice terminal, voice AI platform, and large model capability platform. Among them, the voice terminal mainly provides a voice entry for sampling user audio and interacting with the voice AI platform; the voice AI platform is a platform that provides voice recognition, natural language processing, and device control instruction distribution, and mainly interacts with the voice terminal and the large model capability platform; the large model capability platform supports data vectorization and intent similarity retrieval capabilities, and provides the voice AI platform with smart home control intent classification functions.

[0109] like Figure 8 As shown in the figure, taking the home control system based on large model knowledge enhancement including three modules of voice terminal, voice AI platform and large model capability platform as an example, an optional step for users to voice control smart home devices is as follows:

[0110] Step 1: The voice terminal collects user audio in real time and calls the audio recognition interface of the voice AI platform;

[0111] Step 2: The voice AI platform receives audio in the form of binary stream bytes and translates the audio into text using speech recognition (ASR) technology;

[0112] Step 3: The voice AI platform constructs a prompt for the prompt word engineering and calls the large model LLM dialogue interface to request reasoning;

[0113] Step 4: After receiving the inference request, the large model vectorizes the request and performs a similarity search for smart home control intentions, and returns the identification intention hit result;

[0114] Step 5: The voice AI platform receives the response result of the large model. If it is a smart home control intention, the home entity is extracted from the text;

[0115] Step 6: The voice AI platform internally normalizes the household entity extraction results and normalizes the user's actions in the device operation text into control command values;

[0116] Step 7: The voice AI platform sends the device control command to the device, executes the device control and returns the control result;

[0117] Step 8: The voice terminal receives the device control result and the front end displays it to the user.

[0118] like Fig. 9 As shown, an embodiment of the present invention further provides a home control device 600 based on large model knowledge enhancement, which can implement the above-mentioned home control method based on large model knowledge enhancement, and the device includes:

[0119] The first module 601 is used to collect initial audio;

[0120] The second module 602 is used to convert the initial audio to obtain a target text;

[0121] The third module 603 is used to call the large language model to obtain the control intention discrimination result according to the target text;

[0122] The fourth module 604 is used to obtain a control instruction according to the target text when the control intention determination result is a smart home control intention;

[0123] The fifth module 605 is used to control the target smart home according to the control instruction to obtain a control result.

[0124] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0125] The embodiment of the present invention further provides an electronic device, which includes a processor and a memory, wherein the memory stores a computer program, and when the processor executes the computer program, the above-mentioned home control method based on large model knowledge enhancement is implemented. The electronic device can be any intelligent terminal including a tablet computer, a car computer, etc.

[0126] It can be understood that the contents of the above method embodiments are all applicable to the present device embodiments, the functions specifically implemented by the present device embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0127] refer to Fig.10 , Fig.10 The hardware structure of an electronic device of another embodiment is illustrated, and the electronic device includes:

[0128] The processor 701 may be implemented by a general-purpose CPU (Central Processing Unit), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits, and is used to execute relevant programs to implement the technical solutions provided in the embodiments of the present invention.

[0129] The memory 702 can be implemented in the form of a read-only memory (ROM), a static storage device, a dynamic storage device, or a random access memory (RAM). The memory 702 can store an operating system and other application programs. When the technical solution provided in the embodiment of this specification is implemented by software or firmware, the relevant program code is stored in the memory 702, and the processor 701 calls and executes a home control method based on large model knowledge enhancement according to an embodiment of the present invention;

[0130] Input / output interface 703, used to implement information input and output;

[0131] Communication interface 704, used to realize communication interaction between the device and other devices, which can be realized through wired mode (such as USB, network cable, etc.) or wireless mode (such as mobile network, WI FI, Bluetooth, etc.);

[0132] A bus 705 that transmits information between the various components of the device (e.g., the processor 701, the memory 702, the input / output interface 703, and the communication interface 704);

[0133] The processor 701 , the memory 702 , the input / output interface 703 and the communication interface 704 are connected to each other in communication within the device via a bus 705 .

[0134] An embodiment of the present invention further provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, it implements the above-mentioned home control method based on large model knowledge enhancement.

[0135] It can be understood that the contents of the above method embodiments are all applicable to the present storage medium embodiments, the functions specifically implemented by the present storage medium embodiments are the same as those of the above method embodiments, and the beneficial effects achieved are also the same as those achieved by the above method embodiments.

[0136] The embodiment of the present invention further provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device can read the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the aforementioned home control method based on large model knowledge enhancement.

[0137] In summary, a home control method, device and electronic device based on large model knowledge enhancement according to an embodiment of the present invention have the following advantages:

[0138] 1. The embodiment of the present invention provides smart home intent classification capabilities through a home knowledge graph formed by pre-training of a large model, so as to achieve correct home control intent matching of user audio texts without the need for later operators to annotate data, thereby achieving control of smart home devices, enhancing the generalization capability of user home control audio, improving the accuracy of intent matching, and reducing the workload of manual intervention in operations.

[0139] 2. In the embodiment of the present invention, the sound pickup device, the voice AI platform and the big model are integrated and interacted with each other. After the user initiates voice control, the solution of the embodiment of the present invention can hit the smart home control intention and control the device execution without exhausting the user's possible smart device control statements by enumeration in advance.

[0140] 3. The embodiment of the present invention constructs a home control intention classification prompt project prompt to achieve correct judgment and classification of user text intention without the need for business or operation personnel to organize zero-sample or small-sample tasks of the home control training set in advance, thereby reducing human intervention and lowering operating costs.

[0141] 4. The embodiment of the present invention has strong generalization ability. Based on the knowledge graph and reasoning ability generated by the large model, it can quickly retrieve the intention of the user text in a vectorized form and determine whether it hits the home control, so that the user control text can be complex and flexible, thereby improving the user experience quality.

[0142] In some selectable embodiments, the function / operation mentioned in the block diagram may not occur in the order mentioned in the operation diagram. For example, depending on the function / operation involved, the two boxes shown in succession can actually be executed substantially simultaneously or the boxes can sometimes be executed in reverse order. In addition, the embodiment presented and described in the flow chart of the present invention is provided by way of example, for the purpose of providing a more comprehensive understanding of technology. The disclosed method is not limited to the operation and logic flow presented herein. Selectable embodiments are expected, wherein the order of various operations is changed and the sub-operation of a part for which is described as a larger operation is performed independently.

[0143] In addition, although the present invention is described in the context of functional modules, it should be understood that, unless otherwise specified, one or more of the functions and / or features described may be integrated into a single physical device and / or software module, or one or more functions and / or features may be implemented in separate physical devices or software modules. It is also understood that a detailed discussion of the actual implementation of each module is unnecessary for understanding the present invention. More specifically, in view of the properties, functions, and internal relationships of the various functional modules in the device disclosed herein, the actual implementation of the module will be understood within the conventional skills of the engineer. Therefore, those skilled in the art can implement the present invention set forth in the claims without excessive experimentation using ordinary techniques. It is also understood that the specific concepts disclosed are merely illustrative and are not intended to limit the scope of the present invention, which is determined by the full scope of the appended claims and their equivalents.

[0144] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0145] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0146] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0147] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, a plurality of steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0148] In the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "examples", "specific examples", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described may be combined in any one or more embodiments or examples in a suitable manner.

[0149] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

[0150] The above is a specific description of the preferred implementation of the present invention, but the present invention is not limited to the described embodiments. Those skilled in the art may make various equivalent modifications or substitutions without violating the spirit of the present invention. These equivalent modifications or substitutions are all included in the scope defined by the claims of the present invention.

Claims

1. A home control method based on large model knowledge enhancement, characterized in that: The following steps are involved: Collect initial audio; Performing a conversion operation on the initial audio to obtain a target text; According to the target text, calling a large language model to obtain a control intention discrimination result; When the control intention determination result is a smart home control intention, obtaining a control instruction according to the target text; According to the control instruction, the target smart home is controlled to obtain a control result.

2. A home control method based on large model knowledge enhancement according to claim 1, characterized in that: The converting operation on the initial audio to obtain the target text comprises the following steps: Converting the initial audio into a target audio in the form of binary stream bytes; The target audio is converted into the target text through speech recognition technology.

3. A home control method based on large model knowledge enhancement according to claim 1, characterized in that: The method of calling a large language model to obtain a control intention discrimination result according to the target text includes the following steps: constructing an initial prompt word according to the target text; Sending an inference request to the large language model according to the initial prompt word; In response to the inference request, performing a vectorization operation on the initial prompt word through the large language model to obtain a target prompt word; The control intention discrimination result is obtained according to the target prompt word.

4. A home control method based on large model knowledge enhancement according to claim 3, characterized in that: The step of obtaining the control intention identification result according to the target prompt word comprises the following steps: According to the target prompt word, similarity judgment is performed on the smart home control intention to obtain the control intention judgment result; The control intention identification result includes that the target prompt word is the smart home control intention and the target prompt word is not the smart home control intention.

5. The home control method based on large model knowledge enhancement according to claim 1 is characterized in that: When the control intention determination result is a smart home control intention, obtaining a control instruction according to the target text includes the following steps: Performing entity extraction operation on the target text to obtain entity extraction result; The entity extraction result is normalized to obtain the control instruction.

6. The home control method based on large model knowledge enhancement according to claim 1 is characterized in that: According to the control instruction, controlling the target smart home to obtain a control result includes the following steps: Sending the control instruction to the target smart home; In response to the control instruction, the control task of the target smart home is executed, and the control result is obtained and displayed.

7. A home control device based on large model knowledge enhancement, characterized in that: include: The first module is used to collect initial audio; The second module is used to convert the initial audio to obtain a target text; The third module is used to call the large language model to obtain the control intention discrimination result according to the target text; The fourth module is used to obtain a control instruction according to the target text when the control intention determination result is a smart home control intention; The fifth module is used to control the target smart home according to the control instruction to obtain the control result.

8. An electronic device, characterized in that: including a processor and a memory; The memory is used to store programs; The processor executes the program to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium, characterized in that: The storage medium stores a program, and the program is executed by a processor to implement the method according to any one of claims 1 to 6.

10. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images