Equipment control method, device, storage medium and electronic device based on digital twin
By adjusting the text characteristics of the device control instructions, more accurate control instructions are generated, which solves the problem of low device control accuracy and improves the device's response accuracy.
Patent Information
- Application Number
- CN202210690264.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2022-06-17
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2042-06-17
AI Technical Summary
In the prior art, the control accuracy of the equipment is low, especially in voice control scenarios, when the equipment does not have accurate speech recognition, resulting in omissions or errors in the command response, reducing the control accuracy of the equipment.
By acquiring the first control command information, adjusting the characters according to its text characteristics, and generating the second control command information to improve the control accuracy of the device. The method includes using adjustment module, replacement submodule, insertion submodule, determination module, input module and identification module, and using neural network model to perform text recognition and encoding processing, and generating a second control instruction that can trigger more control instructions.
The accuracy of equipment control is improved. By adjusting the character of the original control command information, additional supplements to the control command information and improvement of the information dimension are achieved, thereby enhancing the control accuracy of the equipment.
Smart Images

Figure CN115097738B_ABST
Abstract
Description
Technical Field
[0001] The present application relates to the field of computers, and more specifically, to a device control method, storage medium, and electronic device based on digital twins. Background Art
[0002] With the rapid development of smart devices, users can control their devices in more convenient ways, such as voice control and gesture control. However, this improvement in control efficiency often comes with some problems. For example, if the device's voice recognition is not accurate enough, the response to the command will naturally be flawed or erroneous, thereby reducing the device's control accuracy. Therefore, the related art has a technical problem of low device control accuracy.
[0003] To address the above-mentioned problems, no effective solutions have been proposed so far. Summary of the Invention
[0004] The embodiments of the present application provide a device control method, apparatus, storage medium, and electronic device based on digital twins to at least solve the technical problem of low control accuracy of the device.
[0005] According to one aspect of an embodiment of the present application, a device control method based on digital twins is provided, including: obtaining first control instruction information, wherein the first control instruction information is used to trigger a first control instruction for a device to be controlled; adjusting N characters in the first control instruction information according to text features corresponding to the first control instruction information to obtain second control instruction information, wherein the second control instruction information is used to trigger the first control instruction and the second control instruction for the device to be controlled; and controlling the device to be controlled in response to the first control instruction and the second control instruction triggered by the second control instruction information.
[0006] According to another aspect of an embodiment of the present application, a device control device based on digital twins is also provided, including: an acquisition unit for acquiring first control instruction information, wherein the above-mentioned first control instruction information is used to trigger a first control instruction for the device to be controlled; an adjustment unit for adjusting N characters in the above-mentioned first control instruction information according to text features corresponding to the above-mentioned first control instruction information to obtain second control instruction information, wherein the above-mentioned second control instruction information is used to trigger the above-mentioned first control instruction and the second control instruction for the above-mentioned device to be controlled; a control unit for controlling the above-mentioned device to be controlled in response to the above-mentioned first control instruction and the above-mentioned second control instruction triggered by the above-mentioned second control instruction information.
[0007] As an optional solution, the above-mentioned adjustment unit includes: an adjustment module, which is used to adjust the above-mentioned N characters according to the text features corresponding to the above-mentioned first control instruction information based on the above-mentioned N characters to obtain the above-mentioned second control instruction information, wherein the above-mentioned M characters include the above-mentioned N characters and at least one auxiliary adjustment character.
[0008] As an optional solution, the above-mentioned adjustment module includes at least one of the following: a replacement submodule, used to replace the second character of the above-mentioned N characters with the first character of the above-mentioned M characters; an insertion submodule, used to insert the third character of the above-mentioned M characters into the associated position of the above-mentioned N characters, wherein the above-mentioned associated position is the position associated with the fourth character of the above-mentioned N characters.
[0009] As an optional solution, the above-mentioned device includes: a first determination module, which is used to determine the configuration account associated with the above-mentioned first control instruction information and obtain historical control instruction information of the above-mentioned configuration account before adjusting the above-mentioned N characters using the above-mentioned M characters according to the text features corresponding to the above-mentioned first control instruction information to obtain the above-mentioned second control instruction information, wherein the above-mentioned historical control instruction information is the control instruction information associated with the above-mentioned configuration account; a second determination module, which is used to determine from the above-mentioned historical control instruction information the characters whose association frequency with the above-mentioned configuration account is greater than or equal to a preset threshold before adjusting the above-mentioned N characters using the above-mentioned M characters according to the text features corresponding to the above-mentioned first control instruction information to obtain the above-mentioned second control instruction information, and determine the characters whose association frequency is greater than or equal to the preset threshold as the above-mentioned auxiliary adjustment characters.
[0010] As an optional solution, the above-mentioned adjustment unit includes: an input module for inputting the above-mentioned first control instruction information into a text recognition model, wherein the above-mentioned information recognition model is a neural network model trained using sample information and used to recognize text information; a recognition module for using the above-mentioned text recognition model to perform text recognition on the above-mentioned first control instruction information to obtain a recognition result, wherein the above-mentioned recognition result includes the above-mentioned second control instruction information.
[0011] As an optional solution, the above-mentioned recognition module includes at least one of the following: an extraction module, which is used to use the extraction module in the above-mentioned text recognition model to extract the text features corresponding to the above-mentioned first control instruction information; an encoding module, which is used to use the encoding module in the above-mentioned text recognition model to encode the above-mentioned N characters to obtain a first encoding result; a prediction module, which is used to use the network module in the above-mentioned text recognition model to predict the above-mentioned first encoding result based on the text features corresponding to the above-mentioned first control instruction information to obtain a second encoding result; a decoding module, which is used to use the decoding module of the above-mentioned text recognition model to decode the above-mentioned second encoding result to obtain the above-mentioned recognition result.
[0012] As an optional solution, the above-mentioned control unit includes at least one of the following: a first control module, used to control the above-mentioned first device according to the above-mentioned first control instruction when the above-mentioned first control instruction is used to instruct the control of the first device and the above-mentioned second control instruction is used to instruct the control of the second device, and to control the above-mentioned second device according to the above-mentioned second control instruction, wherein the above-mentioned device to be controlled includes the above-mentioned first device and the above-mentioned second device; a second control module, used to control the above-mentioned device to be controlled according to the above-mentioned first control instruction when the instruction priority of the above-mentioned first control instruction is greater than the instruction priority of the above-mentioned second control instruction; a third control module, used to control the above-mentioned device to be controlled according to the above-mentioned second control instruction when the instruction priority of the above-mentioned second control instruction is greater than the instruction priority of the above-mentioned first control instruction.
[0013] According to another aspect of the embodiments of the present application, a computer program product or computer program is provided, the computer program product or computer program including computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the above-described device control method based on digital twins.
[0014] According to another aspect of an embodiment of the present application, an electronic device is also provided, including a memory, a processor, and a computer program stored in the memory and runnable on the processor, wherein the processor executes the above-mentioned digital twin-based device control method through the computer program.
[0015] In an embodiment of the present application, first control instruction information is obtained, wherein the first control instruction information is used to trigger a first control instruction for the device to be controlled; according to the text features corresponding to the first control instruction information, N characters in the first control instruction information are adjusted to obtain second control instruction information, wherein the second control instruction information is used to trigger the first control instruction and the second control instruction for the device to be controlled; in response to the first control instruction and the second control instruction triggered by the second control instruction information, the device to be controlled is controlled, and control instruction information that can trigger more control instructions is obtained by adjusting the characters in the original control instruction information, thereby achieving the purpose of additional supplement to the control instruction information and improving the information dimension of the control instruction information, thereby achieving the technical effect of improving the control accuracy of the device, and thus solving the technical problem of low control accuracy of the device. BRIEF DESCRIPTION OF THE DRAWINGS
[0016] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0017] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.
[0018] Figure 1 Schematic diagram of a hardware environment for an interactive method of a smart device according to an embodiment of the present application;
[0019] Figure 2 This is a schematic diagram of a process of an optional device control method based on digital twins according to an embodiment of the present application;
[0020] Figure 3 is a schematic diagram of an optional device control method based on digital twins according to an embodiment of the present application;
[0021] Figure 4 is a schematic diagram of another optional device control method based on digital twins according to an embodiment of the present application;
[0022] Figure 5 is a schematic diagram of another optional device control method based on digital twins according to an embodiment of the present application;
[0023] Figure 6 is a schematic diagram of another optional device control method based on digital twins according to an embodiment of the present application;
[0024] Figure 7 is a schematic diagram of another optional device control method based on digital twins according to an embodiment of the present application;
[0025] Figure 8 is a schematic diagram of an optional device control apparatus based on digital twins according to an embodiment of the present application;
[0026] Figure 9 It is a schematic structural diagram of an optional electronic device according to an embodiment of the present application. DETAILED DESCRIPTION
[0027] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts should fall within the scope of protection of this application.
[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or sequential order. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in a sequence other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.
[0029] According to one aspect of the embodiment of the present application, a method for interacting with a smart home device is provided. The method for interacting with the smart home device is widely used in smart home (Smart Home), smart home, smart home device ecology, smart home (IntelligenceHouse) ecology and other whole-house intelligent digital control application scenarios. Optionally, in this embodiment, the method for interacting with the smart home device can be applied to Figure 1 In the hardware environment shown in FIG. 1 , which is composed of a terminal device 102 and a server 104. Figure 1As shown, the server 104 is connected to the terminal device 102 via a network, and can be used to provide services (such as application services, etc.) for the terminal or the client installed on the terminal. A database can be set up on the server or independently of the server to provide data storage services for the server 104. Cloud computing and / or edge computing services can be configured on the server or independently of the server to provide data computing services for the server 104.
[0030] The aforementioned network may include, but is not limited to, at least one of the following: a wired network and a wireless network. The aforementioned wired network may include, but is not limited to, at least one of the following: a wide area network, a metropolitan area network, and a local area network. The aforementioned wireless network may include, but is not limited to, at least one of the following: Wi-Fi (Wireless Fidelity) and Bluetooth. The terminal device 102 may be, but is not limited to, a PC, a mobile phone, a tablet computer, a smart air conditioner, a smart range hood, a smart refrigerator, a smart oven, a smart stove, a smart washing machine, a smart water heater, a smart washing machine, a smart dishwasher, a smart projection device, a smart TV, a smart clothes drying rack, smart curtains, smart audio and video, a smart socket, a smart speaker, a smart fresh air device, smart kitchen and bathroom equipment, smart bathroom equipment, a smart sweeping robot, a smart window cleaning robot, a smart mopping robot, a smart air purifier, a smart steamer, a smart microwave oven, a smart kitchen treasure, a smart purifier, a smart water dispenser, a smart door lock, etc.
[0031] Alternatively, as an optional implementation, Figure 2 As shown in Figure 2, the device control method based on digital twins includes:
[0032] S202, obtaining first control instruction information, where the first control instruction information is used to trigger a first control instruction for the device to be controlled;
[0033] S204: Adjust N characters in the first control instruction information according to text features corresponding to the first control instruction information to obtain second control instruction information, wherein the second control instruction information is used to trigger the first control instruction and the second control instruction for the device to be controlled;
[0034] S206 , controlling the device to be controlled in response to the first control instruction and the second control instruction triggered by the second control instruction information.
[0035] Optionally, in this embodiment, the above device control method based on digital twin can be but is not limited to being applied to a multi-intent recognition scenario. Among them, multi-intent recognition is a technology of natural language understanding (NLU for short), and natural language understanding is a technology of natural language processing (NLP for short). It is currently widely used in the fields of dialogue systems and intelligent assistants in various industries; in addition, multi-intent recognition is also an important part of a machine to achieve artificial intelligence.
[0036] Optionally, in this embodiment, the first control instruction information can be but is not limited to the control instruction information obtained by recognizing the collected audio information using speech recognition technology, such as Figure 3 As shown, the controlled device 302 collects a piece of audio initiated by the user 304 and performs speech recognition on the piece of audio to obtain the first control instruction information; in addition, the collection of audio and the execution of speech recognition can be but is not limited to being performed by a third-party device to reduce the burden and energy consumption of the controlled device 302; and the execution of speech recognition can be but is not limited to being performed by a server to improve the execution efficiency of speech recognition.
[0037] Optionally, in this embodiment, the text features corresponding to the first control instruction information can be but is not limited to the basic features of the text corresponding to the first control instruction information, such as lexical features, context features, etc.
[0038] Optionally, in this embodiment, the N characters in the first control instruction information are based on Figure 3 As shown in the scenario, continuing for example Figure 4 As shown, the controlled device 302 collects a piece of audio initiated by the user 304 and performs speech recognition on the piece of audio to obtain the first control instruction information 402. Then, the N characters can be but is not limited to including "turn", "on", "air", "conditioner" in the first control instruction information 402.
[0039] Optionally, in this embodiment, the first control instruction information for triggering the first control instruction for the controlled device can be but is not limited to being understood as recognizing the first key information for triggering the first control instruction for the controlled device; the second control instruction information for triggering the first control instruction and the second control instruction for the controlled device can be but is not limited to being understood as recognizing the above first key information and the second key information for triggering the second control instruction for the controlled device;
[0040] To further illustrate, optionally, for example, the first control instruction information is "turn on the air conditioner and water heater", but the first key information of the first control instruction information used to trigger the first control instruction of the controlled device is identified as "turn on the air conditioner"; and the second control instruction information is "turn on the air conditioner, turn on the water heater", and then the first key information of the second control instruction information used to trigger the first control instruction of the controlled device is identified as "turn on the air conditioner", and the second key information of the second control instruction information used to trigger the second control instruction of the controlled device is identified as "turn on the water heater".
[0041] It should be noted that first control instruction information is obtained, wherein the first control instruction information is used to trigger a first control instruction for the controlled device; according to the text features corresponding to the first control instruction information, N characters in the first control instruction information are adjusted to obtain second control instruction information, wherein the second control instruction information is used to trigger the first control instruction and the second control instruction for the controlled device; in response to the first control instruction and the second control instruction triggered by the second control instruction information, the controlled device is controlled.
[0042] To further illustrate, the optional Figure 3 The scenario shown, continuing with e.g. Figure 5 As shown, the first control instruction information 502 is obtained, such as Figure 5 As shown in (a), the first control instruction information 502 is used to trigger the first control instruction of the controlled device 302; further as shown in Figure 5 As shown in (b), according to the text features corresponding to the first control instruction information 502, N characters in the first control instruction information 502 are adjusted to obtain the second control instruction information 504, wherein the second control instruction information 504 is used to trigger the first control instruction and the second control instruction for the controlled device 302; in response to the first control instruction and the second control instruction triggered by the second control instruction information 504, the controlled device 302 is controlled.
[0043] Through the embodiments provided by the present application, first control instruction information is obtained, wherein the first control instruction information is used to trigger a first control instruction for a controlled device; according to text features corresponding to the first control instruction information, N characters in the first control instruction information are adjusted to obtain second control instruction information, wherein the second control instruction information is used to trigger the first control instruction and the second control instruction for the controlled device; in response to the first control instruction and the second control instruction triggered by the second control instruction information, the controlled device is controlled, and control instruction information that can trigger more control instructions is obtained by adjusting the characters in the original control instruction information, thereby achieving the purpose of additional supplementation to the control instruction information and improving the information dimension of the control instruction information, thereby realizing the technical effect of improving the accuracy of device control.
[0044] As an alternative solution, according to the text features corresponding to the first control instruction information, N characters in the first control instruction information are adjusted to obtain the second control instruction information, including:
[0045] When M characters associated with the first control instruction information are determined based on the N characters, according to the text features corresponding to the first control instruction information, the N characters are adjusted using the M characters to obtain the second control instruction information, where the M characters include the N characters and at least one auxiliary adjustment character.
[0046] Optionally, in this embodiment, the auxiliary adjustment character can be but is not limited to being used to supplement the characters in the first control instruction information. For example, the auxiliary adjustment character can be but is not limited to being a symbol character (such as ",", "!", "?", etc.), an auxiliary word character (such as "le", "de", "di", etc.), a high-frequency character (such as a character with a relatively high usage frequency), etc.
[0047] Optionally, in this embodiment, adjusting the N characters using the M characters can be but is not limited to including adjusting the character content, character position, character punctuation, etc. of the N characters.
[0048] As an alternative solution, adjusting the N characters using the M characters includes at least one of the following:
[0049] S1, replacing the second character in the N characters with the first character in the M characters;
[0050] S2, inserting the third character in the M characters at the associated position of the N characters, where the associated position is the position associated with the fourth character in the N characters.
[0051] It should be noted that the second character in the N characters is replaced with the first character in the M characters; the third character in the M characters is inserted at the associated position of the N characters, where the associated position is the position associated with the fourth character in the N characters.
[0052] For further illustration, optionally, for example Figure 6 As shown, the second character "he" in the N characters of the first control instruction information 604 is replaced with the first character "da" in the M characters 602; the third character "," in the M characters 602 is inserted at the first associated position of the N characters of the first control instruction information 604, and the third character "kai" in the M characters 602 is inserted at the second associated position of the N characters of the first control instruction information 604, to obtain the second control instruction information 606, where the first associated position is the position after the fourth character "qi" in the N characters, and the second associated position is the position before the fourth character "kong" in the N characters.
[0053] Through the embodiment provided by the present application, the first character among M characters is used to replace the second character among N characters; the third character among M characters is inserted into the associated position of N characters, wherein the associated position is the position associated with the fourth character among N characters, thereby achieving the technical effect of improving the efficiency of character adjustment.
[0054] As an optional solution, before adjusting N characters using M characters according to the text features corresponding to the first control instruction information to obtain the second control instruction information, the method includes:
[0055] S1, determining a configuration account associated with the first control instruction information, and obtaining historical control instruction information of the configuration account, wherein the historical control instruction information is control instruction information associated with the configuration account;
[0056] S2: Determine characters whose association frequency with the configuration account is greater than or equal to a preset threshold from the historical control instruction information, and determine the characters whose association frequency is greater than or equal to the preset threshold as auxiliary adjustment characters.
[0057] Optionally, in this embodiment, in order to improve the accuracy of the supplement to the first control instruction information, it is possible but not limited to referring to the historical control instruction information of the configuration account, such as determining from the historical control instruction information the characters whose association frequency with the configuration account is greater than or equal to a preset threshold, and determining the characters whose association frequency is greater than or equal to the preset threshold as auxiliary adjustment characters.
[0058] Through the embodiments provided in the present application, the configuration account associated with the first control instruction information is determined, and the historical control instruction information of the configuration account is obtained, wherein the historical control instruction information is the control instruction information associated with the configuration account; characters whose association frequency with the configuration account is greater than or equal to a preset threshold are determined from the historical control instruction information, and characters whose association frequency is greater than or equal to the preset threshold are determined as auxiliary adjustment characters, thereby achieving the technical effect of improving the accuracy of the supplement to the first control instruction information.
[0059] As an optional solution, N characters in the first control instruction information are adjusted according to the text features corresponding to the first control instruction information to obtain the second control instruction information, including:
[0060] S1, inputting the first control instruction information into a text recognition model, wherein the information recognition model is a neural network model trained using sample information and used to recognize text information;
[0061] S2. Perform text recognition on the first control instruction information using a text recognition model to obtain a recognition result, wherein the recognition result includes the second control instruction information.
[0062] Optionally, in this embodiment, the text recognition model can be, but is not limited to, a deep learning model, such as a deep learning network with functions of sentence segmentation and sentence completion. On the one hand, it solves the problem of semantic recognition in multi-intent recognition and also solves the problem of inaccurate prediction of multi-intent sentence component completion, providing a new direction for multi-intent recognition.
[0063] Optionally, in this embodiment, before using the text recognition model to perform text recognition on the first control instruction information to obtain the recognition result, the initial text recognition model is trained using the sample information;
[0064] For further illustration, optionally, for example, the format of the training data (sample information) is the original sentence and the rewritten sentence, such as "Turn on the water heater and raise the temperature by 20 degrees" and "Turn on the water heater, and the water heater raises the temperature by 20 degrees", "Turn on the water heater and air conditioner" and "Turn on the water heater, turn on the air conditioner", etc.;
[0065] Optionally, the extra words after the training data is rewritten are extracted from the original sentence. Therefore, an external character source (auxiliary adjustment characters) that is relatively commonly used needs to be added to the data input into the model network (text recognition model), such as "de", "le", commas, etc. Therefore, the data after processing the training data and inputting it into the network is as follows: "De le, turn on the water heater and raise the temperature by 20 degrees" and "Turn on the water heater, and the water heater raises the temperature by 20 degrees", "De le, turn on the water heater and air conditioner" and "Turn on the water heater, turn on the air conditioner";
[0066] In this embodiment, an encoder is used to encode the training data. Taking the matrix encoded by the example sentences ("De le, turn on the water heater and air conditioner" and "Turn on the water heater, turn on the air conditioner") as an example, the encoding rule is as Figure 7 shown. The vertical example sentences represent the original sentences, and the horizontal sentences represent the rewritten sentences. -1 in the matrix indicates that the character in the rewritten sentence is discarded, 0 indicates that the character is retained, 1 indicates insertion, and 2 indicates replacement. It can be seen from Figure 7 that the position corresponding to "and" in the rewritten sentence with 2 is the comma ",". It shows that "and" in the original sentence is replaced by the comma ",". And the empty space in the rewritten sentence corresponds to two "1"s which are "turn on", so the two words "turn on" are inserted before the empty space;
[0067] Furthermore, the subsequent network results, such as the attention-unet network structure and loss results, are trained using the matrix data shown in Figure 7 where the attention-unet network structure can be, but is not limited to, the albert model, the Unet structure can be, but is not limited to, configured with downsampling and two upsamplings, and the Loss structure can be, but is not limited to, the sparse_cross structure;
[0068] Furthermore, in this embodiment, the attention-unet network structure is used to predict a matrix, which is then decoded into a rewritten sentence for output, and it is determined whether the outputs have met the convergence conditions. If so, the training is terminated; if not, the next training data is selected to continue the model training.
[0069] Through the embodiments provided in the present application, the first control instruction information is input into a text recognition model, wherein the information recognition model is a neural network model trained using sample information and used to recognize text information; the text recognition model is used to perform text recognition on the first control instruction information to obtain a recognition result, wherein the recognition result includes the second control instruction information, thereby achieving the purpose of efficiently completing the efficient adjustment of the control instruction information using the model, thereby realizing the technical effect of improving the adjustment efficiency of the control instruction information.
[0070] As an optional solution, the text recognition model is used to perform text recognition on the first control instruction information to obtain a recognition result, including at least one of the following:
[0071] S1, using an extraction module in a text recognition model to extract text features corresponding to the first control instruction information;
[0072] S2, using the encoding module in the text recognition model to encode the N characters to obtain a first encoding result;
[0073] S3, based on the text features corresponding to the first control instruction information, using a network module in the text recognition model to predict the first encoding result to obtain a second encoding result;
[0074] S4, using the decoding module of the text recognition model to decode the second encoding result to obtain a recognition result.
[0075] As an optional solution, controlling the device to be controlled in response to the first control instruction and the second control instruction triggered by the second control instruction information includes at least one of the following:
[0076] S1, when a first control instruction is used to instruct control of a first device and a second control instruction is used to instruct control of a second device, controlling the first device according to the first control instruction and controlling the second device according to the second control instruction, wherein the devices to be controlled include the first device and the second device;
[0077] S2, when the instruction priority of the first control instruction is greater than the instruction priority of the second control instruction, controlling the device to be controlled according to the first control instruction;
[0078] S3: When the instruction priority of the second control instruction is greater than the instruction priority of the first control instruction, control the device to be controlled according to the second control instruction.
[0079] Optionally, in this embodiment, the first control instruction and the second control instruction can be used but are not limited to controlling different devices. When the first control instruction and the second control instruction control the same device, it is necessary to further determine whether there is a conflict between the first control instruction and the second control instruction. If there is a conflict, the priority of the first control instruction and the second control instruction is determined. When the instruction priority of the first control instruction is greater than the instruction priority of the second control instruction, the controlled device is controlled according to the first control instruction; when the instruction priority of the second control instruction is greater than the instruction priority of the first control instruction, the controlled device is controlled according to the second control instruction.
[0080] It should be noted that, when the first control instruction is used to instruct the control of the first device and the second control instruction is used to instruct the control of the second device, the first device is controlled according to the first control instruction, and the second device is controlled according to the second control instruction, wherein the devices to be controlled include the first device and the second device; when the instruction priority of the first control instruction is greater than the instruction priority of the second control instruction, the device to be controlled is controlled according to the first control instruction; when the instruction priority of the second control instruction is greater than the instruction priority of the first control instruction, the device to be controlled is controlled according to the second control instruction.
[0081] Through the embodiments provided by the present application, when the first control instruction is used to instruct the control of the first device and the second control instruction is used to instruct the control of the second device, the first device is controlled according to the first control instruction, and the second device is controlled according to the second control instruction, wherein the devices to be controlled include the first device and the second device; when the instruction priority of the first control instruction is greater than the instruction priority of the second control instruction, the device to be controlled is controlled according to the first control instruction; when the instruction priority of the second control instruction is greater than the instruction priority of the first control instruction, the device to be controlled is controlled according to the second control instruction, thereby achieving the technical effect of improving the comprehensiveness of control over the devices.
[0082] It is understandable that in the specific implementation of this application, related data such as user information is involved. When the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data must comply with relevant laws, regulations and standards of relevant countries and regions.
[0083] It should be noted that for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that this application is not limited by the order of the actions described, because according to this application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by this application.
[0084] According to another aspect of the embodiment of the present application, a device control apparatus based on digital twins for implementing the above-mentioned device control method based on digital twins is also provided. Figure 8 As shown, the device includes:
[0085] An acquiring unit 802 is configured to acquire first control instruction information, wherein the first control instruction information is used to trigger a first control instruction to a device to be controlled;
[0086] An adjusting unit 804 is configured to adjust N characters in the first control instruction information according to text features corresponding to the first control instruction information to obtain second control instruction information, wherein the second control instruction information is used to trigger the first control instruction and the second control instruction for the device to be controlled;
[0087] The control unit 806 is configured to control the device to be controlled in response to the first control instruction and the second control instruction triggered by the second control instruction information.
[0088] Optionally, in this embodiment, the above-mentioned digital twin-based device control device can be but is not limited to being used in multi-intent recognition scenarios, where multi-intent recognition is a technology of natural language understanding (NLU), and natural language understanding is a technology of natural language processing (NLP), which is currently widely used in dialogue systems and intelligent assistants in various industries; in addition, multi-intent recognition is also an important part of machines realizing artificial intelligence.
[0089] Optionally, in this embodiment, the text feature corresponding to the first control instruction information may be, but is not limited to, a basic feature of the text corresponding to the first control instruction information, such as a vocabulary feature, a context feature, and the like.
[0090] It should be noted that first control instruction information is obtained, wherein the first control instruction information is used to trigger a first control instruction for the controlled device; according to the text features corresponding to the first control instruction information, N characters in the first control instruction information are adjusted to obtain second control instruction information, wherein the second control instruction information is used to trigger the first control instruction and the second control instruction for the controlled device; in response to the first control instruction and the second control instruction triggered by the second control instruction information, the controlled device is controlled.
[0091] For specific embodiments, reference can be made to the examples shown in the above-mentioned device control apparatus based on digital twins, which will not be repeated here.
[0092] Through the embodiments provided by the present application, first control instruction information is obtained, wherein the first control instruction information is used to trigger a first control instruction for a controlled device; according to text features corresponding to the first control instruction information, N characters in the first control instruction information are adjusted to obtain second control instruction information, wherein the second control instruction information is used to trigger the first control instruction and the second control instruction for the controlled device; in response to the first control instruction and the second control instruction triggered by the second control instruction information, the controlled device is controlled, and control instruction information that can trigger more control instructions is obtained by adjusting the characters in the original control instruction information, thereby achieving the purpose of additional supplementation to the control instruction information and improving the information dimension of the control instruction information, thereby realizing the technical effect of improving the accuracy of device control.
[0093] As an optional solution, the adjustment unit 804 includes:
[0094] An adjustment module is used to adjust the N characters using M characters according to text features corresponding to the first control instruction information when M characters associated with the first control instruction information are determined based on N characters to obtain second control instruction information, wherein the M characters include N characters and at least one auxiliary adjustment character.
[0095] For specific embodiments, please refer to the examples shown in the above-mentioned device control method based on digital twins, which will not be repeated here.
[0096] As an optional solution, the adjustment module includes at least one of the following:
[0097] A replacement submodule, for replacing the second character in N characters with the first character in M characters;
[0098] The inserting submodule is used to insert the third character among the M characters into the associated position of the N characters, wherein the associated position is the position associated with the fourth character among the N characters.
[0099] For specific embodiments, please refer to the examples shown in the above-mentioned device control method based on digital twins, which will not be repeated here.
[0100] As an optional solution, the device includes:
[0101] a first determining module configured to, before obtaining second control instruction information by adjusting N characters using M characters based on text features corresponding to the first control instruction information, determine a configuration account associated with the first control instruction information and obtain historical control instruction information of the configuration account, wherein the historical control instruction information is control instruction information that has been associated with the configuration account;
[0102] The second determination module is used to adjust N characters using M characters according to the text features corresponding to the first control instruction information, before obtaining the second control instruction information, to determine from the historical control instruction information the characters whose association frequency with the configured account is greater than or equal to a preset threshold, and to determine the characters whose association frequency is greater than or equal to the preset threshold as auxiliary adjustment characters.
[0103] For specific embodiments, please refer to the examples shown in the above-mentioned device control method based on digital twins, which will not be repeated here.
[0104] As an optional solution, the adjustment unit 804 includes:
[0105] An input module, configured to input the first control instruction information into a text recognition model, wherein the information recognition model is a neural network model trained using sample information and used to recognize text information;
[0106] The recognition module is used to perform text recognition on the first control instruction information using a text recognition model to obtain a recognition result, wherein the recognition result includes the second control instruction information.
[0107] For specific embodiments, please refer to the examples shown in the above-mentioned device control method based on digital twins, which will not be repeated here.
[0108] As an optional solution, the identification module includes at least one of the following:
[0109] An extraction module, configured to extract text features corresponding to the first control instruction information using an extraction module in a text recognition model;
[0110] An encoding module, configured to encode the N characters using the encoding module in the text recognition model to obtain a first encoding result;
[0111] A prediction module, configured to predict the first encoding result using a network module in a text recognition model based on text features corresponding to the first control instruction information to obtain a second encoding result;
[0112] The decoding module is used to use the decoding module of the text recognition model to decode the second encoding result to obtain a recognition result.
[0113] For specific embodiments, please refer to the examples shown in the above-mentioned device control method based on digital twins, which will not be repeated here.
[0114] As an optional solution, the control unit 806 includes at least one of the following:
[0115] a first control module, configured to control the first device according to the first control instruction and the second device according to the second control instruction when the first control instruction is used to instruct control of the first device and the second control instruction is used to instruct control of the second device, wherein the devices to be controlled include the first device and the second device;
[0116] a second control module, configured to control the device to be controlled according to the first control instruction when the instruction priority of the first control instruction is greater than the instruction priority of the second control instruction;
[0117] The third control module is configured to control the device to be controlled according to the second control instruction when the instruction priority of the second control instruction is greater than the instruction priority of the first control instruction.
[0118] For specific embodiments, please refer to the examples shown in the above-mentioned device control method based on digital twins, which will not be repeated here.
[0119] According to another aspect of the embodiment of the present application, an electronic device for implementing the above-mentioned device control method based on digital twins is also provided, such as Figure 9 As shown, the electronic device includes a memory 902 and a processor 904. The memory 902 stores a computer program, and the processor 904 is configured to execute the steps in any of the above method embodiments through the computer program.
[0120] Optionally, in this embodiment, the electronic device may be located in at least one network device among a plurality of network devices of a computer network.
[0121] Optionally, in this embodiment, the processor may be configured to execute the following steps through a computer program:
[0122] S1, obtaining first control instruction information, wherein the first control instruction information is used to trigger a first control instruction for a device to be controlled;
[0123] S2, adjusting N characters in the first control instruction information according to text features corresponding to the first control instruction information to obtain second control instruction information, wherein the second control instruction information is used to trigger the first control instruction and the second control instruction for the device to be controlled;
[0124] S3, controlling the device to be controlled in response to the first control instruction and the second control instruction triggered by the second control instruction information.
[0125] Alternatively, those skilled in the art will appreciate that Figure 9 The structure shown is for illustration only, and the electronic device may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a PDA, a mobile Internet device (MID), a PAD, or other terminal devices. Figure 9 It does not limit the structure of the above electronic device. For example, the electronic device may also include Figure 9 More or fewer components (such as network interfaces, etc.) as shown in, or with Figure 9 Different configurations shown.
[0126] Among them, the memory 902 can be used to store software programs and modules, such as the program instructions / modules corresponding to the device control method and apparatus based on digital twins in the embodiments of the present application. The processor 904 executes various functional applications and data processing by running the software programs and modules stored in the memory 902, that is, realizing the above-mentioned device control method based on digital twins. The memory 902 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 902 may further include a memory remotely located relative to the processor 904, and these remote memories may be connected to the terminal via a network. Examples of the above-mentioned networks include, but are not limited to, the Internet, corporate intranets, local area networks, mobile communication networks, and combinations thereof. Among them, the memory 902 can be used specifically, but not limited to, to store information such as the first control instruction information and the second control instruction information. As an example, such as Figure 9 As shown, the memory 902 may include, but is not limited to, the acquisition unit 802, adjustment unit 804, and control unit 806 in the device control apparatus based on digital twins. In addition, it may also include, but is not limited to, other module units in the device control apparatus based on digital twins, which will not be repeated in this example.
[0127] Optionally, the transmission device 906 is used to receive or send data via a network. Specific examples of the network may include wired networks and wireless networks. In one embodiment, the transmission device 906 includes a network interface controller (NIC), which can be connected to other network devices and a router via a network cable to communicate with the Internet or a local area network. In one embodiment, the transmission device 906 is a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.
[0128] In addition, the electronic device further includes: a display 908 for displaying information such as the first control instruction information and the second control instruction information; and a connection bus 910 for connecting various module components in the electronic device.
[0129] In other embodiments, the terminal device or server may be a node in a distributed system, wherein the distributed system may be a blockchain system, and the blockchain system may be a distributed system formed by connecting multiple nodes via network communication. The nodes may form a peer-to-peer (P2P) network, and any computing device, such as a server, terminal, or other electronic device, may become a node in the blockchain system by joining the peer-to-peer network.
[0130] According to one aspect of the present application, a computer program product is provided, comprising a computer program / instructions containing program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network via a communication component and / or installed from a removable medium. When the computer program is executed by a central processing unit, the various functions provided in the embodiments of the present application are performed.
[0131] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0132] It should be noted that the computer system of the electronic device is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.
[0133] A computer system includes a central processing unit (CPU), which can perform various appropriate actions and processes based on programs stored in read-only memory (ROM) or programs loaded from the storage unit into random access memory (RAM). The RAM also stores various programs and data required for system operation. The CPU, the read-only memory, and the RAM are connected to each other via a bus. Input / output interfaces (I / O interfaces) are also connected to the bus.
[0134] The following components are connected to the input / output interface: an input section including a keyboard, mouse, etc.; an output section including a cathode ray tube (CRT), a liquid crystal display (LCD), and a speaker; a storage section including a hard disk; and a communication section including a network interface card such as a local area network card and a modem. The communication section performs communication processing via a network such as the Internet. A drive is also connected to the input / output interface as needed. Removable media such as magnetic disks, optical disks, magneto-optical disks, semiconductor memories, etc. are installed in the drive as needed so that computer programs read from them can be installed into the storage section as needed.
[0135] In particular, according to an embodiment of the present application, the processes described in the various method flow charts can be implemented as computer software programs. For example, an embodiment of the present application includes a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for executing the methods shown in the flow charts. In such an embodiment, the computer program can be downloaded and installed from a network via a communication portion, and / or installed from a removable medium. When the computer program is executed by a central processing unit, the various functions defined in the system of the present application are performed.
[0136] According to one aspect of the present application, a computer-readable storage medium is provided, and a processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the methods provided in the various optional implementations described above.
[0137] Optionally, in this embodiment, the computer-readable storage medium may be configured to store a computer program for performing the following steps:
[0138] S1, obtaining first control instruction information, wherein the first control instruction information is used to trigger a first control instruction for a device to be controlled;
[0139] S2, adjusting N characters in the first control instruction information according to text features corresponding to the first control instruction information to obtain second control instruction information, wherein the second control instruction information is used to trigger the first control instruction and the second control instruction for the device to be controlled;
[0140] S3, controlling the device to be controlled in response to the first control instruction and the second control instruction triggered by the second control instruction information.
[0141] Optionally, in this embodiment, a person of ordinary skill in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing the hardware related to the terminal device through a program, and the program may be stored in a computer-readable storage medium, which may include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, etc.
[0142] The serial numbers of the above embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.
[0143] If the integrated units in the above embodiments are implemented in the form of software functional units and sold or used as independent products, they can be stored in the above-mentioned computer-readable storage medium. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for enabling one or more computer devices (which can be personal computers, servers, or network devices, etc.) to execute all or part of the steps of the methods described in each embodiment of the present application.
[0144] In the above embodiments of the present application, the description of each embodiment has its own focus. For parts that are not described in detail in a certain embodiment, please refer to the relevant description of other embodiments.
[0145] In the several embodiments provided in this application, it should be understood that the disclosed client can be implemented in other ways. Among them, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, and can be electrical or other forms.
[0146] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.
[0147] In addition, the functional units in the various embodiments of the present application may be integrated into a single processing unit, or each unit may exist physically separately, or two or more units may be integrated into a single unit. The aforementioned integrated units may be implemented in the form of hardware or software functional units.
[0148] The above is only a preferred embodiment of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.
Claims
1. A device control method based on digital twins, It is characterized in that include: Acquire first control instruction information, wherein the first control instruction information is used to trigger a first control instruction of the device to be controlled; Determine the configuration account associated with the first control instruction information, and obtain historical control instruction information of the configuration account, wherein the historical control instruction information is control instruction information associated with the configuration account; Determine an auxiliary adjustment character from the historical control instruction information, wherein the auxiliary adjustment character is a character whose association frequency with the configuration account is greater than or equal to a preset threshold, and the auxiliary adjustment character is used to adjust the character content, character position, and character segmentation of the N characters in the first control instruction; In a case where M characters associated with the first control instruction information are determined based on the N characters, the N characters are adjusted using the M characters according to text features corresponding to the first control instruction information to obtain second control instruction information, wherein the M characters include the N characters and at least one of the auxiliary adjustment characters, and the second control instruction information is used to trigger the first control instruction and a second control instruction for the device to be controlled; In response to the first control instruction and the second control instruction triggered by the second control instruction information, the device to be controlled is controlled.
2. The method according to claim 1, It is characterized in that The adjusting the N characters by using the M characters includes at least one of the following: Using the first character of the M characters to replace the second character of the N characters; Inserting a third character of the M characters into an associated position of the N characters, wherein the associated position is a position associated with a fourth character of the N characters.
3. The method according to any one of claims 1 to 2, It is characterized in that The controlling of the device to be controlled in response to the first control instruction and the second control instruction triggered by the second control instruction information includes at least one of the following: In a case where the first control instruction is used to instruct control of a first device and the second control instruction is used to instruct control of a second device, the first device is controlled according to the first control instruction, and the second device is controlled according to the second control instruction, wherein the devices to be controlled include the first device and the second device; When the instruction priority of the first control instruction is greater than the instruction priority of the second control instruction, controlling the device to be controlled according to the first control instruction; When the instruction priority of the second control instruction is greater than the instruction priority of the first control instruction, the device to be controlled is controlled according to the second control instruction.
4. A device control device based on digital twins, It is characterized in that include: An acquiring unit, configured to acquire first control instruction information, wherein the first control instruction information is used to trigger a first control instruction for a device to be controlled; Determine the configuration account associated with the first control instruction information, and obtain historical control instruction information of the configuration account, wherein the historical control instruction information is control instruction information associated with the configuration account; Determine an auxiliary adjustment character from the historical control instruction information, wherein the auxiliary adjustment character is a character whose association frequency with the configuration account is greater than or equal to a preset threshold, and the auxiliary adjustment character is used to adjust the character content, character position, and character segmentation of the N characters in the first control instruction; In a case where M characters associated with the first control instruction information are determined based on the N characters, the N characters are adjusted using the M characters according to text features corresponding to the first control instruction information to obtain second control instruction information, wherein the M characters include the N characters and at least one of the auxiliary adjustment characters, and the second control instruction information is used to trigger the first control instruction and a second control instruction for the device to be controlled; A control unit is used to control the device to be controlled in response to the first control instruction and the second control instruction triggered by the second control instruction information.
5. A computer-readable storage medium, It is characterized in that The computer-readable storage medium includes a stored program, wherein the program executes the method according to any one of claims 1 to 3 when executed.
6. An electronic device comprising a memory and a processor, It is characterized in that A computer program is stored in the memory, and the processor is configured to execute the method according to any one of claims 1 to 3 through the computer program.
Citation Information
Patent Citations
Intention recognition and execution method and device, vehicle-mounted voice dialogue system and computer storage medium
CN110556102A
User multi-intention identification method and device, storage medium and vehicle
CN111400438A
Voice control method and device, electronic equipment and readable storage medium
CN112151035A
Multi-intention identification method and display equipment
CN112163086A
Multi-intention recognition method and device, electronic equipment and readable storage medium
CN112256864A
