Device control method, device, system and storage medium

Abstract

The present disclosure provides a device control method, apparatus, computer system, computer-readable storage medium, and computer program product, relating to the fields of artificial intelligence technology and the Internet of Things technology. The method comprises: in response to receiving control information from a user, performing semantic feature extraction on the control information to obtain semantic feature information; using target convolution kernels corresponding to different tag categories, respectively extracting information associated with the tag category from the semantic feature information to obtain N category-associated feature information corresponding to the tag category, wherein the tag category includes N, and N≥2; for each tag category, determining a target tag from a tag set corresponding to the tag category based on the semantic feature information and the category-associated feature information corresponding to the tag category; and controlling a target device to perform an operation based on the N target tags.

Description

Technical Field

[0001] The present disclosure relates to the fields of artificial intelligence technology and the Internet of Things technology, and in particular, to a device control method, apparatus, computer system, computer-readable storage medium, and computer program product. Background Art

[0002] With the rapid development of artificial intelligence and the Internet of Things (IoT), smart electronic devices, such as smart air conditioners and smart humidifiers, are playing a vital role in our daily lives and work. Users can control these devices by sending simple commands to their control devices, enabling them to turn them on and off, among other adjustments. This simplifies the user experience and streamlines the process of controlling these devices.

[0003] In the process of realizing the concept of the present disclosure, the inventors found that there are at least the following technical problems in the related technology: the control device cannot accurately identify the information content in the user control instructions, and the execution accuracy of the control instructions is low, resulting in poor user experience. Summary of the Invention

[0004] In view of this, the present disclosure provides a device control method, apparatus, computer system, computer-readable storage medium, and computer program product.

[0005] One aspect of the present disclosure provides a device control method, comprising:

[0006] In response to receiving control information from the user, extracting semantic features from the control information to obtain semantic feature information;

[0007] Using target convolution kernels corresponding to different label categories, respectively extract information associated with the label category in the semantic feature information to obtain N category-associated feature information corresponding to the label category, wherein the label category includes N, and N ≥ 2;

[0008] For each of the above tag categories, determine a target tag from a tag set corresponding to the above tag category based on the above semantic feature information and the category-associated feature information corresponding to the above tag category;

[0009] The target device is controlled to perform an operation according to the N target tags.

[0010] According to an embodiment of the present disclosure, for each of the above-mentioned tag categories, determining a target tag from a tag set corresponding to the above-mentioned tag category based on the above-mentioned semantic feature information and the category-associated feature information corresponding to the above-mentioned tag category includes:

[0011] For each of the above-mentioned label categories, the above-mentioned semantic feature information is fused with the category-associated feature information corresponding to the above-mentioned label category to obtain the target semantic feature information corresponding to the above-mentioned label category;

[0012] Input the target semantic feature information into the target recognition model, and output the target recognition result for the tag category, wherein the target recognition model corresponds to the tag category, and the target recognition result is the target recognition probability value of the tag in the tag set corresponding to the tag category;

[0013] According to the target recognition probability value, the target tag is determined from the tag set corresponding to the tag category.

[0014] According to an embodiment of the present disclosure, the semantic feature information includes a semantic feature vector, and the category association feature information includes a category association feature vector;

[0015] For each of the above-mentioned label categories, the above-mentioned semantic feature information is fused with the category-associated feature information corresponding to the above-mentioned label category to obtain target semantic feature information corresponding to the above-mentioned label category, including:

[0016] For each of the above label categories, a dot product is performed between the above semantic feature vector and the category-associated feature vector corresponding to the above label category to obtain target semantic feature information corresponding to the above label category.

[0017] According to an embodiment of the present disclosure, the above-mentioned tag categories include intent tag categories and device tag categories, and the above-mentioned target tags include target intent tags and target device tags;

[0018] Controlling the target device to perform operations according to the N target tags includes:

[0019] Determining the target device according to the target device label, wherein the target device label matches the name identifier of the target device;

[0020] According to the target intent tag, the target device is controlled to perform the operation corresponding to the target intent tag.

[0021] According to an embodiment of the present disclosure, in response to receiving control information from a user, semantic feature extraction is performed on the control information, and the obtained semantic feature information includes:

[0022] In response to receiving control information from the user, inputting the control information into the semantic feature extraction layer to obtain the semantic feature information;

[0023] The semantic feature extraction layer includes at least one of the following:

[0024] BERT feature extraction layer, recurrent neural network feature extraction layer, and long short-term memory feature extraction layer.

[0025] According to an embodiment of the present disclosure, the target device includes a smart home device;

[0026] The above control information includes at least one of the following: voice control information and text control information.

[0027] Another aspect of the present disclosure provides a device control apparatus, comprising:

[0028] a first extraction module, configured to extract semantic features from the control information received from the user to obtain semantic feature information;

[0029] A second extraction module is configured to extract information associated with the label category from the semantic feature information using target convolution kernels corresponding to different label categories, thereby obtaining N pieces of category-associated feature information corresponding to the label category, where the label category includes N, and N ≥ 2.

[0030] a determination module configured to determine, for each of the above-mentioned tag categories, a target tag from a tag set corresponding to the above-mentioned tag category based on the above-mentioned semantic feature information and the category-associated feature information corresponding to the above-mentioned tag category;

[0031] The control module is used to control the target device to perform operations according to the N target tags.

[0032] Another aspect of the present disclosure provides a computer system, including: one or more processors; and a memory for storing one or more programs, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the device control method described above in the embodiment of the present disclosure.

[0033] Another aspect of the present disclosure provides a computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, enables the processor to implement the device control method described above in the embodiment of the present disclosure.

[0034] Another aspect of the present disclosure provides a computer program product, including a computer program, which implements the device control method described above in the embodiment of the present disclosure when the computer program is executed by a processor.

[0035] According to an embodiment of the present disclosure, the user's control information may include multiple label categories. Because the target convolution kernels corresponding to different label categories are used to extract the information associated with the label category in the semantic feature information, the N category-related features obtained can reflect the category features of the corresponding label categories, at least partially overcoming the technical problem of low correlation between semantic feature information and label categories. For each label category, the target label is determined from the label set corresponding to the label category based on the semantic feature information and the category-related feature information corresponding to the label category. This can improve the recognition accuracy of the target label, and then control the target device based on the identified target label, thereby improving the accuracy of executing operation control on the target device, avoiding target device recognition errors, and avoiding target device operation execution errors, thereby improving the convenience of the user controlling the device and improving the user experience. BRIEF DESCRIPTION OF THE DRAWINGS

[0036] The above and other objects, features and advantages of the present disclosure will become more apparent through the following description of the embodiments of the present disclosure with reference to the accompanying drawings, in which:

[0037] Figure 1 Schematically illustrates an exemplary system architecture to which the device control method and apparatus of the present disclosure may be applied;

[0038] Figure 2 The following schematically shows a flow chart of a device control method according to an embodiment of the present disclosure;

[0039] Figure 3 Schematically illustrates a flow chart of determining a target tag from a tag set corresponding to a tag category based on semantic feature information and category-associated feature information corresponding to the tag category for each tag category according to an embodiment of the present disclosure;

[0040] Figure 4 A flowchart of controlling a target device to perform operations according to N target tags according to an embodiment of the present disclosure is schematically shown;

[0041] Figure 5 Schematically shows an application scenario diagram of the device control method according to an embodiment of the present disclosure;

[0042] Figure 6 A block diagram schematically illustrates a device control apparatus according to an embodiment of the present disclosure; and

[0043] Figure 7 A block diagram of an electronic device suitable for implementing a device control method according to an embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0044] Hereinafter, embodiments of the present disclosure will be described with reference to the accompanying drawings. However, it should be understood that these descriptions are merely exemplary and are not intended to limit the scope of the present disclosure. In the detailed description below, for ease of explanation, many specific details are set forth to provide a comprehensive understanding of the embodiments of the present disclosure. However, it is apparent that one or more embodiments may also be implemented without these specific details. In addition, in the following description, descriptions of well-known structures and technologies are omitted to avoid unnecessary confusion of the concepts of the present disclosure.

[0045] The terms used herein are only for describing specific embodiments and are not intended to limit the present disclosure. The terms "comprise," "include," etc. used herein indicate the presence of the features, steps, operations, and / or components, but do not exclude the presence or addition of one or more other features, steps, operations, or components.

[0046] All terms used herein (including technical and scientific terms) have the meanings commonly understood by those skilled in the art unless otherwise defined. It should be noted that the terms used herein should be interpreted as having a meaning consistent with the context of this specification and should not be interpreted in an idealized or overly rigid manner.

[0047] When expressions such as "at least one of A, B and C, etc." are used, they should generally be interpreted in accordance with the meaning of the expression commonly understood by those skilled in the art (for example, "a system having at least one of A, B and C" should include but is not limited to a system having A alone, B alone, C alone, A and B, A and C, B and C, and / or A, B, C, etc.).

[0048] With the development of science and technology, smart electronic devices have gradually become popular in people's lives and work. Users can send simple control information, such as voice control information, to the control devices of smart electronic devices to control smart electronic devices to turn on, off and other working conditions, thereby simplifying the user's operating process of controlling electronic devices and improving the user experience.

[0049] After receiving the user's control information, the control device of the smart electronic device can process the control information to identify the user's intention tag information and device tag information in the control information, thereby identifying the smart electronic device that the user needs to control, and identifying the user's intention to operate the smart electronic device.

[0050] For example, the user's control information may be a voice message "turn on the humidifier". The control device of the intelligent electronic device performs natural language processing on the voice control information, thereby identifying the intent label as "turn on" and the device label as "humidifier", and then can control the humidifier to perform the turn-on operation according to the identified intent label and device label.

[0051] Processing the control information to identify target labels corresponding to multiple label categories is called a multi-label classification task. In related technologies, multi-label classification tasks are processed by extracting semantic features from the control information. Multiple label classification tasks share the same semantic feature, but this semantic feature contains global information about the control information and does not reflect local information associated with the label category. This can result in the inability to identify valid information associated with the label category in the control information, leading to low label recognition accuracy, and can lead to operational errors such as smart device operation errors and / or smart device recognition errors, causing significant trouble for users.

[0052] Embodiments of the present disclosure provide a device control method, apparatus, computer system, computer-readable storage medium, and computer program product.

[0053] The device control method includes: in response to receiving control information from a user, performing semantic feature extraction on the control information to obtain semantic feature information; using target convolution kernels corresponding to different label categories, respectively extracting information associated with the label category in the semantic feature information to obtain N category-associated feature information corresponding to the label category, wherein the label category includes N, N ≥ 2; for each label category, determining a target label from a label set corresponding to the label category based on the semantic feature information and the category-associated feature information corresponding to the label category; and controlling a target device to perform an operation based on the N target labels.

[0054] According to an embodiment of the present disclosure, the user's control information may include multiple label categories. By using the target convolution kernels corresponding to different label categories, the information associated with the label category in the semantic feature information can be extracted respectively, so that N category-associated features can be obtained to reflect the category characteristics of the corresponding label categories. For each label category, the target label is determined from the label set corresponding to the label category based on the semantic feature information and the category-associated feature information corresponding to the label category, which can improve the recognition accuracy of the target label, and then control the target device according to the identified target label, which can improve the accuracy of executing operation control on the target device, avoid target device recognition errors, and avoid target device operation execution errors.

[0055] In the technical solution disclosed herein, the collection, storage, use, processing, transmission, provision, disclosure and application of user personal information involved comply with the provisions of relevant laws and regulations, take necessary confidentiality measures, and do not violate public order and good morals.

[0056] In the technical solution disclosed herein, the user's authorization or consent is obtained before obtaining or collecting the user's personal information.

[0057] Figure 1 The following schematically illustrates an exemplary system architecture 100 to which the device control method and apparatus of the present disclosure can be applied according to an embodiment of the present disclosure. Figure 1 The examples shown are merely examples of system architectures to which the embodiments of the present disclosure may be applied, to help those skilled in the art understand the technical content of the present disclosure, but do not mean that the embodiments of the present disclosure may not be used in other devices, systems, environments or scenarios.

[0058] like Figure 1 As shown, the system architecture 100 according to this embodiment may include terminal devices 101, 102, and 103, a network 104, a server 105, and an intelligent device 106. The network 104 is used as a medium for providing a communication link between the terminal devices 101, 102, and 103 and the server 105. The network 104 may include various connection types, such as wired and / or wireless communication links, etc.

[0059] User 120 can use terminal devices 101, 102, 103 to interact with server 105 via network 104 to receive or send control information, etc. Terminal devices 101, 102, 103 can be installed with various communication client applications, such as shopping applications, web browser applications, search applications, instant messaging tools, email clients, and / or social platform software (for example only).

[0060] The terminal devices 101 , 102 , and 103 may be various electronic devices having a display screen and supporting web browsing, including but not limited to smart phones, tablet computers, laptop computers, and desktop computers.

[0061] The server 105 may be a server that provides various services, such as a background management server (for example only) that supports websites browsed by users using the terminal devices 101, 102, and 103. The background management server may analyze and process received data such as user requests, and feed back processing results (such as web pages, information, or data obtained or generated according to user requests) to the terminal device.

[0062] The smart device 106 may include, for example, a smart air conditioner. For example, the terminal devices 101, 102, and 103 may control the smart air conditioner to turn on / off, cool / heat according to the target tag, but it is not limited to this. The smart device may also include a smart humidifier, a smart sweeping robot, etc.

[0063] The device control method provided in the embodiments of the present disclosure may generally be executed by the terminal device 101, 102, or 103, or may also be executed by another terminal device different from the terminal device 101, 102, or 103. Accordingly, the device control apparatus provided in the embodiments of the present disclosure may generally be provided in the terminal device 101, 102, or 103, or may be provided in another terminal device different from the terminal device 101, 102, or 103.

[0064] Accordingly, the device control method provided in the embodiment of the present disclosure may also be executed by the server 105. Accordingly, the device control apparatus provided in the embodiment of the present disclosure may also be provided in the server 105. The device control method provided in the embodiment of the present disclosure may also be executed by a server or server cluster that is different from the server 105 and that can communicate with the terminal devices 101, 102, 103 and / or the server 105. Accordingly, the device control apparatus provided in the embodiment of the present disclosure may also be provided in a server or server cluster that is different from the server 105 and that can communicate with the terminal devices 101, 102, 103 and / or the server 105.

[0065] For example, user 120 can send control information (e.g., voice control information) to any one of terminal devices 101, 102, or 103 (e.g., terminal device 101, but not limited thereto). Terminal device 101 can then locally execute the device control method provided in the embodiments of the present disclosure, or send the control information to other terminal devices, servers, or server clusters, and the other terminal devices, servers, or server clusters that receive the control information can execute the device control method provided in the embodiments of the present disclosure.

[0066] It should be understood that Figure 1 The number of terminal devices, networks, servers, and intelligent devices in the embodiment is merely illustrative. Any number of terminal devices, networks, servers, and intelligent devices may be provided as required.

[0067] Figure 2 The flowchart of the device control method according to the embodiment of the present disclosure is schematically shown.

[0068] like Figure 2 As shown, the device control method may include operations S210 to S240.

[0069] In operation S210 , in response to receiving control information from a user, semantic feature extraction is performed on the control information to obtain semantic feature information.

[0070] According to the embodiments of the present disclosure, the user's control information may include voice information, text information, etc., and the semantic features in the control information can be extracted using a model built based on a neural network, such as an LSTM model (long short-term memory network model), but is not limited to this. It can also include semantic feature extraction of the control information based on a syntactic analysis model. The embodiments of the present disclosure do not limit the specific method of extracting semantic features.

[0071] In operation S220, target convolution kernels corresponding to different label categories are used to extract information associated with the label category in the semantic feature information to obtain N category-associated feature information corresponding to the label category, where the label category includes N, and N≥2.

[0072] According to an embodiment of the present disclosure, the tag category may include the category to which the identified target tag belongs. For example, when the target tag is "humidifier", the tag category may be a device tag category, and the tag set corresponding to the device tag category may include multiple different device tags.

[0073] According to an embodiment of the present disclosure, semantic feature information may include information types such as matrices or vectors. The target convolution kernel corresponding to the label category can slide on the semantic feature information according to a preset step size, thereby extracting information associated with the label category in the semantic feature information and obtaining category-associated feature information corresponding to the label category.

[0074] It should be noted that the target convolution kernels corresponding to different label categories can be of the same type. For example, the target convolution kernels can all be 1×5 dimensional vectors, but the elements in each target convolution kernel are different. Furthermore, the target convolution kernels corresponding to different label categories can also be of different types. For example, the target convolution kernel corresponding to the intent label type can be a 1×5 dimensional vector, and the target convolution kernel corresponding to the device label type can be a 2×5 dimensional matrix. Those skilled in the art can set the specific information type of the target convolution kernel according to actual needs.

[0075] It should be understood that the sample control information can be assigned a sample label corresponding to the label category, and the sample control information and the sample label can be used to train an initial convolution kernel corresponding to the label category. The convolution kernel elements in the initial convolution kernel can then be reversely adjusted using a gradient descent algorithm to obtain a trained target convolution kernel. However, this is not limited to this, and the target convolution kernel can also be obtained based on other training methods in related technologies. The embodiments of this disclosure do not limit the specific training method.

[0076] In operation S230 , for each tag category, a target tag is determined from a tag set corresponding to the tag category based on the semantic feature information and the category-associated feature information corresponding to the tag category.

[0077] According to an embodiment of the present disclosure, category-associated feature information is used as the attention weight corresponding to the label category. The attention weight can reflect the local information associated with the label category. Based on the attention mechanism, the attention weight corresponding to the label category is assigned to the semantic feature information, so that the semantic feature information has local information associated with the label category. In the subsequent process of target label recognition based on the semantic feature information with local information, the accuracy of target label recognition in the label set corresponding to the label category can be improved to avoid recognition errors.

[0078] In one embodiment of the present disclosure, the target convolution kernel may be a 1×5 dimensional vector, and the semantic feature information may be a 1×512 dimensional vector. Before semantic feature extraction is performed on the semantic feature information, the semantic feature information may be padded, and the preset sliding step size of the target convolution kernel is 1, so that the obtained category association feature information can be a 1×512 dimensional vector.

[0079] In operation S240 , the target device is controlled to perform an operation according to the N target tags.

[0080] According to embodiments of the present disclosure, a target tag may include a key control signal for controlling a target device to perform an operation, or may include an identification signal for identifying a target device from among multiple devices, but is not limited thereto. The target tag may also include a prediction signal for predicting the device's geographical location. By controlling the target device to perform an operation using N target tags, the target device can be accurately identified and controlled to perform an operation consistent with the control information.

[0081] According to an embodiment of the present disclosure, the user's control information may include multiple label categories. Because the target convolution kernels corresponding to different label categories are used to extract information associated with the label category from the semantic feature information, the N category-associated features obtained can reflect the category characteristics of the corresponding label categories, at least partially overcoming the technical problem of low correlation between semantic feature information and label categories. For each label category, the target label is determined from the label set corresponding to the label category based on the semantic feature information and the category-associated feature information corresponding to the label category. This can improve the recognition accuracy of the target label, and then control the target device based on the identified target label, thereby improving the accuracy of executing operational control on the target device, avoiding target device recognition errors, and avoiding errors in target device operation execution.

[0082] According to an embodiment of the present disclosure, the target device may include a smart home device.

[0083] The control information includes at least one of the following: voice control information and text control information.

[0084] According to an embodiment of the present disclosure, smart home devices may include home appliances that can perform operations according to control signals extracted from target tags, such as smart air conditioners, smart humidifiers, smart sweeping robots, etc.

[0085] According to an embodiment of the present disclosure, voice control information may include voice information sent directly by a user to a control device of an intelligent electronic device, or may also include voice information sent indirectly to the control device of an intelligent electronic device via other devices such as a mobile phone. Correspondingly, text control information may include text information used by a user to control an intelligent electronic device, such as text messages sent by a user via a smartphone to the control device of an intelligent electronic device.

[0086] According to an embodiment of the present disclosure, operation S210, in response to receiving control information from a user, extracting semantic features from the control information to obtain semantic feature information may include the following operations.

[0087] In response to receiving control information from the user, the control information is input into the semantic feature extraction layer to obtain semantic feature information; wherein the semantic feature extraction layer includes at least one of the following: BERT feature extraction layer, recurrent neural network feature extraction layer, and long short-term memory feature extraction layer.

[0088] According to the embodiments of the present disclosure, the semantic feature extraction layer can fully learn the context information in the control information, so that the obtained semantic feature information can reflect all the semantic features in the control information.

[0089] It should be understood that before the control information is input into the semantic feature extraction layer, relevant technologies can be used to encode the control information and thus convert it into encoded information of the control information. The embodiments of the present disclosure do not limit the specific encoding method, and those skilled in the art can make a choice based on actual conditions.

[0090] Figure 3 The flowchart of an embodiment of the present disclosure schematically illustrates, for each tag category, determining a target tag from a tag set corresponding to the tag category based on semantic feature information and category-associated feature information corresponding to the tag category.

[0091] like Figure 3 As shown, operation S230, for each tag category, determining a target tag from a tag set corresponding to the tag category according to semantic feature information and category-associated feature information corresponding to the tag category, may include operations S310 to S330.

[0092] In operation S310 , for each tag category, feature fusion is performed on the semantic feature information and the category-associated feature information corresponding to the tag category to obtain target semantic feature information corresponding to the tag category.

[0093] In operation S320, the target semantic feature information is input into the target recognition model, and the target recognition result for the tag category is output, wherein the target recognition model corresponds to the tag category, and the target recognition result is the target recognition probability value of the tag in the tag set corresponding to the tag category.

[0094] In operation S330 , a target tag is determined from a tag set corresponding to the tag category according to the target recognition probability value.

[0095] According to an embodiment of the present disclosure, semantic feature information and category-associated feature information corresponding to a label category are input into a feature fusion model built based on a neural network to achieve feature fusion of the semantic feature information and the category-associated feature information, thereby obtaining target semantic feature information. The feature fusion model can, for example, be a model built based on an attention mechanism. Alternatively, the semantic feature information and the category-associated feature information can be correlated and calculated to obtain the target semantic feature information.

[0096] According to an embodiment of the present disclosure, the target recognition model may include a neural network model constructed based on a neural network, such as a multilayer perceptron (MLP), a convolutional neural network model (CNN model), a recurrent neural network model (RNN model), etc.

[0097] It should be understood that each tag category may correspond to a tag combination containing multiple tags. Therefore, the target recognition result for one tag category can be the target recognition probability value for each tag in the tag set corresponding to that tag category. By comparing the target recognition probability values ​​of each tag in the tag set, the tag with the largest target recognition probability value can be determined as the target tag for that tag category.

[0098] According to an embodiment of the present disclosure, the semantic feature information may include a semantic feature vector, and the category association feature information may include a category association feature vector.

[0099] According to an embodiment of the present disclosure, operation S310 , for each tag category, performing feature fusion on semantic feature information and category-associated feature information corresponding to the tag category to obtain target semantic feature information corresponding to the tag category may include the following operations.

[0100] For each label category, the semantic feature vector and the category-associated feature vector corresponding to the label category are multiplied to obtain the target semantic feature information corresponding to the label category.

[0101] According to an embodiment of the present disclosure, the semantic feature vector and the category-associated feature vector corresponding to the label category are vector information having the same dimensionality. For example, the semantic feature vector and the category-associated feature vector can both be 1×512-dimensional vector information. The semantic feature vector and the category-associated feature vector are dot-multiplied, i.e., the element in the semantic feature vector is multiplied with the element in the same position in the category-associated feature vector to obtain the target semantic feature information corresponding to the label category.

[0102] According to an embodiment of the present disclosure, when the semantic feature information and the associated feature information are 1×n dimensional vector information, the semantic feature vector and the associated feature vector are calculated using a dot multiplication algorithm to assign the weight for the label category represented by the associated feature vector to the semantic feature vector, so that the target semantic feature information has local features corresponding to the label category, thereby improving the accuracy of subsequent recognition of the target label.

[0103] Furthermore, by using the dot multiplication algorithm to calculate the semantic feature vector and the associated feature vector to obtain the target semantic feature information, the amount of calculation can be saved and the calculation efficiency can be improved, so that it is suitable for smart devices with low computing power such as smart phones and smart speakers to execute the device control method provided by the embodiments of the present disclosure. Similarly, it is also suitable for related devices in smart electronic devices to execute the device control method provided by the embodiments of the present disclosure, thereby expanding the scope of application of the device control method in the embodiments of the present disclosure.

[0104] According to an embodiment of the present disclosure, the tag category includes an intent tag category and a device tag category, and the target tag includes a target intent tag and a target device tag.

[0105] Figure 4 The flowchart of controlling a target device to perform an operation according to N target tags according to an embodiment of the present disclosure is schematically shown.

[0106] like Figure 4 As shown, operation S240, controlling the target device to perform an operation according to N target tags may include operations S410 to S420.

[0107] In operation S410 , a target device is determined according to a target device tag, wherein the target device tag matches a name identifier of the target device.

[0108] In operation S420, the target device is controlled to perform an operation corresponding to the target intent tag according to the target intent tag.

[0109] According to an embodiment of the present disclosure, a tag set corresponding to an intent tag category may include multiple different intent tags, such as "on," "off," "heat up," "cool down," "humidify," "dehumidify," etc. A tag set corresponding to a device tag category may include multiple different device tags, such as "smart air conditioner," "smart humidifier," "smart robot vacuum," etc.

[0110] According to the embodiments of the present disclosure, determining the target device using the target device tag avoids the problem of inaccurate device identification caused by the related art based on the overall semantic information. Furthermore, after determining the target device, the target device can be controlled to perform the operation that matches and corresponds to the target intent tag based on the intent information represented by the target intent tag.

[0111] Figure 5 The application scenario diagram of the device control method according to an embodiment of the present disclosure is schematically shown.

[0112] like Figure 5 As shown, the control information 510 sent by the user may be, for example, a voice control message “turn on humidification.” The control information 510 is input to the semantic feature extraction layer 520 , which extracts semantic features from the control information 510 to obtain semantic feature information 530 .

[0113] In this embodiment, the semantic feature extraction layer 520 may be a BERT neural network layer constructed based on the BERT model, and the semantic feature information 530 output by the BERT neural network layer may be a 1×512-dimensional semantic feature vector.

[0114] In the case where the label category includes the intent label category and the device label category, the convolutional attention model 540 corresponds to the intent label category, and the convolutional attention model 550 corresponds to the device label category.

[0115] Semantic feature information 530 is input into first convolutional layer 541. The target convolution kernel of first convolutional layer 541 extracts information associated with the intent label category from semantic feature information 530, obtaining category-associated feature information 542 corresponding to the intent label category. By performing a dot product between category-associated feature information 542 and semantic feature information 530, target semantic feature information 543 corresponding to the intent label category is obtained.

[0116] It should be understood that when the semantic feature information 530 is a 1×512-dimensional semantic feature vector, it is necessary to complete the semantic feature vector before inputting it into the first convolutional layer 541. The category-related feature information 542 and the target semantic feature information 543 output by the first convolutional layer 541 are both 1×512-dimensional vectors.

[0117] By inputting the target semantic feature information 543 into the first target recognition model 544, a target recognition result 545 for the intent label category can be obtained. According to the target recognition result 545, it can be determined that the target intent label corresponding to the intent label category in the control information 510 can be "on".

[0118] Accordingly, semantic feature information 530 is input into second convolutional layer 551, and the target convolution kernel of second convolutional layer 551 is used to extract information associated with the device tag category from semantic feature information 530, thereby obtaining category-associated feature information 552 corresponding to the device tag category. By performing a dot product between category-associated feature information 552 and semantic feature information 530, target semantic feature information 553 corresponding to the device tag category can be obtained.

[0119] It should be understood that when the semantic feature information 530 is a 1×512-dimensional semantic feature vector, it is necessary to complete the semantic feature vector before inputting it into the second convolutional layer 551. The category-related feature information 552 and the target semantic feature information 553 output by the second convolutional layer 551 are both 1×512-dimensional vectors.

[0120] By inputting the target semantic feature information 553 into the second target recognition model 554, a target recognition result 555 for the device tag category can be obtained. According to the target recognition result 555, it can be determined that the target device tag corresponding to the device tag category in the control information 510 may be "humidifier".

[0121] In this embodiment, the first target recognition model 544 and the second target recognition model 554 can be constructed based on the MLP model. The first target recognition model 544 and the second target recognition model 554 can respectively have one or more hidden layers. The embodiments of the present disclosure do not limit the number of hidden layers in the first target recognition model 544 and the second target recognition model 554.

[0122] According to an embodiment of the present disclosure, if the target intent tag is determined to be "on" and the target device tag is "humidifier," the target device tag can be used to determine that the smart device the user needs to operate is a humidifier, i.e., the target device is a humidifier. Simultaneously, based on the target intent tag "on," the humidifier can be controlled to perform the on-operation corresponding to the target intent tag "on," thereby turning on the humidifier and fulfilling the user's actual need to turn on the humidifier.

[0123] According to an embodiment of the present disclosure, since the target convolution kernel corresponding to the label category is used to extract the information associated with the label category in the semantic feature information, and the obtained category-associated feature information is assigned to the semantic feature information as the weight associated with the label category, after the obtained target semantic feature information is input into the target recognition model corresponding to the label category, the output target recognition result can accurately determine the target label.

[0124] Since devices that can provide humidification functions may include smart devices such as humidifiers and air conditioners, when using the relevant technology to perform overall recognition of the control information "turn on humidification" to implement multi-label classification tasks, it is not possible to accurately identify whether the user needs to operate a humidifier or an air conditioner, which may lead to incorrect recognition of the target tag corresponding to the device tag category, and then incorrectly execute the user's control information.

[0125] According to embodiments of the present disclosure, since users often issue voice control messages in a home environment, other people's voices are often mixed into the voice control messages. For example, when a user issues the voice control message "Turn on the humidifier," another family member may issue the voice message "Who is it?". This results in the voice control message being mixed with noise that is irrelevant to the semantics of the control message.

[0126] The device control method provided by the embodiments of the present disclosure utilizes a target convolution kernel corresponding to a label category to extract information associated with the label category in semantic feature information. The information associated with the label category can be assigned as a weight to the semantic feature information, thereby at least partially solving the problems of low accuracy and poor robustness in the recognition of voice control information in mixed noise in related technologies, thereby improving the accuracy of voice control information recognition and the overall robustness of the recognition model.

[0127] Figure 6 The block diagram schematically shows a device control apparatus according to an embodiment of the present disclosure.

[0128] like Figure 6 As shown, the device control apparatus 600 may include a first extraction module 601 , a second extraction module 602 , a determination module 603 and a control module 604 .

[0129] The first extraction module 601 is configured to extract semantic features of the control information in response to receiving the control information from the user, to obtain semantic feature information.

[0130] The second extraction module 602 is used to use the target convolution kernels corresponding to different label categories to respectively extract information associated with the label category in the semantic feature information, and obtain N category-associated feature information corresponding to the label category, wherein the label category includes N, and N≥2.

[0131] The determination module 603 is configured to determine, for each tag category, a target tag from a tag set corresponding to the tag category according to the semantic feature information and the category-associated feature information corresponding to the tag category.

[0132] The control module 604 is a control module configured to control the target device to perform operations according to N target tags.

[0133] According to an embodiment of the present disclosure, the determination module may include: a fusion unit, an identification unit, and a determination unit.

[0134] The fusion unit is used to perform feature fusion on the semantic feature information and the category-associated feature information corresponding to the label category for each label category, so as to obtain target semantic feature information corresponding to the label category.

[0135] The recognition unit is used to input the target semantic feature information into the target recognition model and output the target recognition result for the label category, wherein the target recognition model corresponds to the label category, and the target recognition result is the target recognition probability value of the label in the label set corresponding to the label category.

[0136] The determination unit is used to determine the target label from the label set corresponding to the label category according to the target recognition probability value.

[0137] According to an embodiment of the present disclosure, the semantic feature information includes a semantic feature vector, and the category association feature information includes a category association feature vector.

[0138] The fusion unit may include a computing subunit.

[0139] The calculation subunit is used to perform a dot product calculation on the semantic feature vector and the category-associated feature vector corresponding to the label category for each label category, so as to obtain target semantic feature information corresponding to the label category.

[0140] According to an embodiment of the present disclosure, the tag category includes an intent tag category and a device tag category, and the target tag includes a target intent tag and a target device tag.

[0141] The control module includes: a target device determination unit and a control unit.

[0142] The target device determining unit is configured to determine the target device according to the target device label, wherein the target device label matches the name identifier of the target device.

[0143] The control unit is used to control the target device to perform the operation corresponding to the target intention tag according to the target intention tag.

[0144] According to an embodiment of the present disclosure, the first extraction module includes a first extraction unit.

[0145] The first extraction unit is configured to, in response to receiving control information from the user, input the control information into the semantic feature extraction layer to obtain semantic feature information;

[0146] The semantic feature extraction layer includes at least one of the following:

[0147] BERT feature extraction layer, recurrent neural network feature extraction layer, and long short-term memory feature extraction layer.

[0148] According to an embodiment of the present disclosure, the target device may include a smart home device; the control information may include at least one of the following: voice control information, text control information.

[0149] It should be noted that the embodiments of the device part of the present disclosure are the same or similar to the embodiments of the method part of the present disclosure, and the present disclosure will not repeat them here.

[0150] According to the modules, units, and subunits of the embodiments of the present invention, any multiple or at least part of the functions of any multiple thereof can be implemented in one module. According to the modules, units, and subunits of the embodiments of the present invention, any one or more thereof can be split into multiple modules for implementation. According to the modules, units, and subunits of the embodiments of the present invention, any one or more thereof can be at least partially implemented as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be implemented by hardware or firmware of any other reasonable way of integrating or packaging the circuit, or implemented in any one of the three implementation modes of software, hardware, and firmware or in an appropriate combination of any several thereof. Alternatively, according to the modules, units, and subunits of the embodiments of the present invention, one or more thereof can be at least partially implemented as a computer program module, which can perform the corresponding function when the computer program module is run.

[0151] For example, any multiple of the first extraction module 601, the second extraction module 602, the determination module 603 and the control module 604 can be combined in one module / unit / subunit to realize, or any one of the modules / units / subunits can be split into multiple modules / units / subunits. Or, at least part of the functions of one or more modules / units / subunits in these modules / units / subunits can be combined with at least part of the functions of other modules / units / subunits and realized in one module / unit / subunit. According to an embodiment of the present disclosure, at least one of the first extraction module 601, the second extraction module 602, the determination module 603 and the control module 604 can be at least partially realized as a hardware circuit, such as a field programmable gate array (FPGA), a programmable logic array (PLA), a system on a chip, a system on a substrate, a system on a package, an application specific integrated circuit (ASIC), or can be realized by hardware or firmware such as any other reasonable way of integrating or packaging the circuit, or by any one of the three implementation modes of software, hardware and firmware or by a suitable combination of any of them. Alternatively, at least one of the first extraction module 601 , the second extraction module 602 , the determination module 603 and the control module 604 may be at least partially implemented as a computer program module, which may perform corresponding functions when executed.

[0152] Figure 7 A block diagram of an electronic device suitable for implementing a device control method according to an embodiment of the present disclosure is schematically shown. Figure 7 The electronic device shown is only an example and should not limit the functions and scope of use of the embodiments of the present disclosure.

[0153] like Figure 7 As shown, the electronic device 700 according to an embodiment of the present disclosure includes a processor 701, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 702 or a program loaded from a storage part 708 into a random access memory (RAM) 703. The processor 701 may include, for example, a general-purpose microprocessor (e.g., a CPU), an instruction set processor and / or a related chipset and / or a special-purpose microprocessor (e.g., an application-specific integrated circuit (ASIC)), etc. The processor 701 may also include an onboard memory for caching purposes. The processor 701 may include a single processing unit or multiple processing units for performing different actions of the method flow according to an embodiment of the present disclosure.

[0154] Various programs and data required for the operation of the electronic device 700 are stored in the RAM 703. The processor 701, ROM 702, and RAM 703 are connected to each other via a bus 704. The processor 701 executes the various operations of the method flow according to the embodiment of the present disclosure by executing the programs in the ROM 702 and / or RAM 703. It should be noted that the programs may also be stored in one or more memories other than the ROM 702 and RAM 703. The processor 701 may also execute the various operations of the method flow according to the embodiment of the present disclosure by executing the programs stored in the one or more memories.

[0155] According to an embodiment of the present disclosure, electronic device 700 may further include an input / output (I / O) interface 705, which is also connected to bus 704. System 700 may also include one or more of the following components connected to I / O interface 705: an input section 706 including a keyboard, a mouse, etc.; an output section 707 including devices such as a cathode ray tube (CRT), a liquid crystal display (LCD), and speakers; a storage section 708 including a hard disk; and a communication section 709 including a network interface card such as a LAN card or a modem. Communication section 709 performs communication processing via a network such as the Internet. Drive 710 is also connected to I / O interface 705 as needed. Removable media 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed in drive 710 as needed, so that computer programs read therefrom can be installed into storage section 708 as needed.

[0156] According to an embodiment of the present disclosure, the method flow according to an embodiment of the present disclosure can be implemented as a computer software program. For example, an embodiment of the present disclosure includes a computer program product, which includes a computer program carried on a computer-readable storage medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part 709, and / or installed from the removable medium 711. When the computer program is executed by the processor 701, the above-mentioned functions defined in the system of the embodiment of the present disclosure are executed. According to an embodiment of the present disclosure, the system, equipment, device, module, unit, etc. described above can be implemented by a computer program module.

[0157] The present disclosure also provides a computer-readable storage medium, which may be included in the device / apparatus / system described in the above embodiments, or may exist independently and not be incorporated into the device / apparatus / system. The computer-readable storage medium carries one or more programs, and when executed, implements the method according to the embodiments of the present disclosure.

[0158] According to an embodiment of the present disclosure, a computer-readable storage medium may be a non-volatile computer-readable storage medium. For example, it may include, but is not limited to: a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device.

[0159] For example, according to an embodiment of the present disclosure, the computer-readable storage medium may include the ROM 702 and / or the RAM 703 described above and / or one or more memories other than the ROM 702 and the RAM 703 .

[0160] The flowcharts and block diagrams in the accompanying drawings illustrate the possible implementation architecture, functions and operations of the systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each box in the flowchart or block diagram can represent a module, program segment, or a part of code, and the above-mentioned module, program segment, or a part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the box can also occur in an order different from that marked in the accompanying drawings. For example, two boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram or flowchart, and the combination of boxes in the block diagram or flowchart, can be implemented with a dedicated hardware-based system that performs the specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0161] Those skilled in the art will appreciate that various combinations and / or combinations of features described in the various embodiments and / or claims of this disclosure may be made, even if such combinations or combinations are not explicitly described in this disclosure. In particular, various combinations and / or combinations of features described in the various embodiments and / or claims of this disclosure may be made, without departing from the spirit and teachings of this disclosure. All such combinations and / or combinations fall within the scope of this disclosure.

[0162] The embodiments of the present disclosure are described above. However, these embodiments are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Although each embodiment has been described separately above, this does not mean that the measures in each embodiment cannot be used in combination to advantage. The scope of the present disclosure is defined by the appended claims and their equivalents. Without departing from the scope of the present disclosure, those skilled in the art may make various substitutions and modifications, which should all fall within the scope of the present disclosure.

Claims

1. A device control method, comprising: In response to receiving control information from a user, extracting semantic features from the control information to obtain semantic feature information; Using target convolution kernels corresponding to different label categories, respectively extract information associated with the label category in the semantic feature information to obtain N category-associated feature information corresponding to the label category, wherein the label category includes N, and N ≥ 2; For each of the tag categories, fusing the semantic feature information and the category-associated feature information corresponding to the tag category according to the attention mechanism to obtain semantic feature information fused with the category-associated feature information; Performing target label recognition on a label set corresponding to the label category based on the semantic feature information fused with the category-associated feature information to obtain a target label; The target device is controlled to perform an operation according to the N target tags.

2. The method according to claim 1, wherein The target label is determined based on the following operations: For each of the tag categories, performing feature fusion on the semantic feature information and the category-associated feature information corresponding to the tag category to obtain target semantic feature information corresponding to the tag category; Inputting the target semantic feature information into a target recognition model and outputting a target recognition result for the tag category, wherein the target recognition model corresponds to the tag category, and the target recognition result is a target recognition probability value of the tag in the tag set corresponding to the tag category; The target tag is determined from a tag set corresponding to the tag category according to the target recognition probability value.

3. The method according to claim 2, wherein: The semantic feature information includes a semantic feature vector, and the category association feature information includes a category association feature vector; For each of the tag categories, the semantic feature information is fused with the category-associated feature information corresponding to the tag category to obtain target semantic feature information corresponding to the tag category, including: For each of the tag categories, a dot product is performed on the semantic feature vector and the category-associated feature vector corresponding to the tag category to obtain target semantic feature information corresponding to the tag category.

4. The method according to claim 1, wherein The tag categories include intent tag categories and device tag categories, and the target tags include target intent tags and target device tags; Controlling the target device to perform an operation according to the N target tags includes: Determining the target device according to the target device tag, wherein the target device tag matches the name identifier of the target device; The target device is controlled to perform an operation corresponding to the target intention tag according to the target intention tag.

5. The method according to claim 1, wherein In response to receiving control information from a user, performing semantic feature extraction on the control information to obtain semantic feature information includes: In response to receiving control information from the user, inputting the control information into the semantic feature extraction layer to obtain the semantic feature information; The semantic feature extraction layer includes at least one of the following: BERT feature extraction layer, recurrent neural network feature extraction layer, and long short-term memory feature extraction layer.

6. The method according to any one of claims 1 to 5, wherein The target device includes a smart home device; The control information includes at least one of the following: voice control information and text control information.

7. A device control apparatus comprising: a first extraction module, configured to extract semantic features from control information received from a user to obtain semantic feature information; A second extraction module is configured to use target convolution kernels corresponding to different label categories to respectively extract information associated with the label category in the semantic feature information, and obtain N category-associated feature information corresponding to the label category, wherein the label category includes N, and N ≥ 2; a determination module, configured to determine, for each of the tag categories, a target tag from a tag set corresponding to the tag category based on the semantic feature information and the category-associated feature information corresponding to the tag category; A control module, configured to control a target device to perform an operation according to the N target tags; The determination module is configured as follows: For each of the tag categories, fusing the semantic feature information and the category-associated feature information corresponding to the tag category according to the attention mechanism to obtain semantic feature information fused with the category-associated feature information; Based on the semantic feature information fused with the category-associated feature information, target label recognition is performed on the label set corresponding to the label category to obtain a target label.

8. A computer system comprising: one or more processors; a memory for storing one or more programs, When the one or more programs are executed by the one or more processors, the one or more processors are enabled to implement the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having executable instructions stored thereon, which, when executed by a processor, causes the processor to implement the method according to any one of claims 1 to 6.

10. A computer program product comprising a computer program, wherein when the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images