Equipment control device based on pre-training large model, terminal equipment and storage medium

Abstract

The invention provides an equipment control device based on a pre-training large model, terminal equipment and a computer readable storage medium, and the device comprises an obtaining module which is used for obtaining a voice control instruction; the analysis module is used for analyzing the voice control instruction to obtain a target device and a target task corresponding to the voice control instruction; and the control module is used for controlling the target equipment to execute the target task. According to the equipment control device provided by the invention, the voice control instruction of the user is received through the acquisition module, and the specific task demand of the user and the execution of the target equipment in the voice control instruction are determined through the analysis module. And then the control module controls the corresponding target equipment to execute the specific target task. According to the equipment control device, each piece of equipment can be controlled through one general model, different intelligent control models do not need to be arranged in each piece of equipment, the equipment control universality is improved, and the storage space for storing the intelligent models in the equipment is saved.

Description

Technical Field

[0001] The present application relates to the field of control technology, and in particular to a device control apparatus, a terminal device and a computer-readable storage medium based on a pre-trained large model. Background Art

[0002] At present, with the development of the field of artificial intelligence, artificial intelligence has reached a new level. General models such as ChatGPT that can interact through natural language can understand the user's intention very well and complete high-quality human-like conversations. Based on this, from the perspective of natural language, it can be concluded that large general language models have great development potential in various fields. Recently, everyone is exploring the capabilities of large models, hoping to use large models to complete product transformation or create more attractive products. At the same time, if the large general language model can be improved, it can better improve the interaction between the product and the user, and improve the efficiency of product use.

[0003] However, in the prior art, the field of device control is either based on the Internet of Things for control, or different artificial intelligence control models are set in separate devices, so that different artificial intelligence models are based on different user needs to control the device to work. It can be seen that each of the multiple devices needs to be set with at least a different artificial intelligence model to meet the needs of users, so the versatility of device control is poor, and each device needs to provide a certain storage space to store different artificial intelligence models. Summary of the invention

[0004] The present application provides a device control device based on a pre-trained large model, which can control the specified device to perform the corresponding task by parsing the user's voice control instructions, without the need to set up an artificial intelligence model for each device.

[0005] In a first aspect, the present application provides a device control apparatus, the apparatus comprising:

[0006] An acquisition module, used to acquire voice control instructions;

[0007] An analysis module, used to analyze the voice control instruction to obtain a target device and a corresponding target task corresponding to the voice control instruction;

[0008] A control module is used to control the target device to execute the target task.

[0009] In a second aspect, the present application also provides a device control method based on a pre-trained large model, the method comprising:

[0010] Get voice control instructions;

[0011] Analyze the voice control command to obtain a target device and a target task corresponding to the voice control command;

[0012] Control the target device to execute the target task.

[0013] In a third aspect, the present application also provides a terminal device, comprising a processor, a memory, and a computer program stored in the memory and executable on the processor, the terminal device being equipped with any one of the device control devices based on the pre-trained large model.

[0014] In a fourth aspect, the present application also provides a computer-readable storage medium, on which a computer program is stored, and the computer program is executed by a processor to control any one of the equipment control devices based on the pre-trained large model.

[0015] The device control device provided by the present application receives the user's voice control instructions through an acquisition module, and then determines the user's specific task requirements and target device execution in the voice control instructions through an analysis module. After that, the corresponding target device is controlled by the control module to perform the specific target task. The device control device of the present application can control each device through a common model, without the need to set up different intelligent control models inside each device, thereby improving the versatility of device control and saving storage space for storing intelligent models inside the device. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

[0017] Figure 1 It is a scene schematic diagram of a device control system based on a pre-trained large model provided in an embodiment of the present application;

[0018] Figure 2 It is a functional module schematic diagram of a device control device based on a pre-trained large model in an embodiment of the present application;

[0019] Figure 3 This is a flow chart of an embodiment of a device control method based on a pre-trained large model in an embodiment of the present application;

[0020] Figure 4 It is a schematic diagram of the structure of the terminal device in the embodiment of the present application. DETAILED DESCRIPTION

[0021] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of this application.

[0022] In the description of the present application, it should be understood that the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the feature. In the description of the present application, "plurality" means two or more, unless otherwise clearly and specifically defined.

[0023] In this application, the word "exemplary" is used to mean "used as an example, illustration or description". Any embodiment described as "exemplary" in this application is not necessarily to be construed as being more preferred or advantageous than other embodiments. At the same time, it is to be understood that in the specific implementation of this application, when user information, user data and other related data are involved, when the above embodiments of this application are applied to specific products or technologies, user permission or consent is required, and the collection, use and processing of relevant data shall comply with relevant laws, regulations and standards of relevant countries and regions.

[0024] In order to enable any person skilled in the art to implement and use the present application, the following description is provided. In the following description, details are listed for the purpose of explanation. It should be understood that those of ordinary skill in the art will recognize that the present application can be implemented without using these specific details. In other examples, known structures and processes will not be elaborated in detail to avoid unnecessary details that make the description of the present application obscure. Therefore, the present application is not intended to be limited to the embodiments shown, but is consistent with the widest range of principles and features disclosed in the present application.

[0025] The present application provides a device control apparatus, a terminal device and a computer-readable storage medium based on a pre-trained large model, which are described in detail below.

[0026] See also Figure 1 , Figure 1 This is a scene diagram of a device control system provided in an embodiment of the present application. The device control system may include a terminal device 100 and a device 200. The terminal device 100 may be equipped with any device control device in the embodiment of the present application and is used to receive voice control instructions issued by a user to control each device.

[0027] In the embodiment of the present application, the terminal device 100 includes but is not limited to a desktop computer, a portable computer, a network server, a PDA (Personal Digital Assistant), a tablet computer, a wireless terminal device, an embedded device, a mobile phone, etc.

[0028] In the embodiments of the present application, communication between the terminal device 100 and the device 200 can be achieved through any communication method, including but not limited to mobile communications based on the 3rd Generation Partnership Project (3GPP), Long Term Evolution (LTE), and Worldwide Interoperability for Microwave Access (WiMAX), or computer network communications based on the TCP / IP Protocol Suite (TCP / IP), User Datagram Protocol (UDP), etc.

[0029] It should be noted that Figure 1 The scenario diagram of the device control system shown is only an example. The device control system and scenario described in the embodiment of the present application are intended to more clearly illustrate the technical solution of the embodiment of the present application, and do not constitute a limitation on the technical solution provided in the embodiment of the present application. A person of ordinary skill in the art can know that with the evolution of the device control system and the emergence of new business scenarios, the technical solution provided in the embodiment of the present application is also applicable to similar technical problems.

[0030] like Figure 2 As shown, Figure 2 A device control device based on a pre-trained large model is provided in an embodiment of the present application, such as Figure 2 As shown, the device 200 includes:

[0031] The acquisition module 201 is used to acquire the voice control instruction.

[0032] The analysis module 202 is used to analyze the voice control instruction and obtain the target device and target task corresponding to the voice control instruction.

[0033] The control module 203 is used to control the target device to execute the target task.

[0034] In the embodiment of the present application, the acquisition module 201 is a voice sensor, so the user's voice control instructions can be obtained through the voice sensor. Among them, the voice sensor can be set in a terminal device, or it can also be installed in each room such as the living room, dining room, bedroom, etc. according to the actual application scenario. For example, in a home environment, voice sensors can be installed in each room such as the living room, dining room, bedroom, etc., so as to be able to obtain the voice control instructions issued by the user in an all-round way in a regional space. At this time, the voice sensor can be connected to a computer, mobile phone and other devices through the home network, and finally control various devices in the home according to the voice control instructions through a computer or mobile phone and other devices. Among them, the devices in the home can also maintain communication connection with computers, mobile phones and other control devices through the home network.

[0035] It should be noted that the home environment is only one of the scenarios listed in the embodiments of the present application, and in actual situations, it can be any target area space, such as an office environment, etc. Or it can also be a larger area space, such as the interior of a building, etc., which is not limited by the specific embodiments of the present application.

[0036] After that, the analysis module 202 can analyze what kind of control information the current voice control instruction specifically includes. Among them, in the embodiment of the present application, it is still possible to extract the sound features of the voice control instruction, and determine each voice entity in the voice control instruction based on the sound features, and finally obtain the corresponding target task and target device through each voice entity. It should be noted that in the embodiment of the present application, determining whether the voice control instruction is a valid voice control instruction and determining the target task and target device corresponding to the voice control instruction can be determined by two artificial intelligence models, or by one artificial intelligence model, and the specific embodiment of the present application is not limited.

[0037] Further, after the specific target task and target device are determined by the analysis module 202, the device control device can locate the target device in the network through the control module 203, and send a control instruction to the target device, so that the target device performs the target task according to the corresponding instruction. Alternatively, the control module 203 can call the control interface of each device in the network, so that the device control device can directly control the specific device.

[0038] The device control device provided by the present application receives the user's voice control instructions through an acquisition module, and then determines the user's specific task requirements and target device execution in the voice control instructions through an analysis module. After that, the corresponding target device is controlled by the control module to perform the specific target task. The device control device of the present application can control each device through a common model, without the need to set up different intelligent control models inside each device, thereby improving the versatility of device control and saving storage space for storing intelligent models inside the device.

[0039] In order to better implement the embodiment of the present application, in one embodiment of the present application, the analysis module is specifically used for:

[0040] The task type of the voice control instruction is analyzed to obtain the target task type; based on the target task type, keywords are extracted from the voice control instruction to obtain the target task.

[0041] The above embodiment provides a solution for determining the target task and the target device through the voice entity. In order to better determine the control requirements of the user in the voice control instruction, the embodiment of the present application provides a solution for classifying the voice control instruction and then determining the specific target task based on different classification results.

[0042] Specifically, in an embodiment of the present application, the method of analyzing the task classification information of the voice control instruction can be classified by the prompt method. Among them, the prompt method is also called the prompting technology. Prompt is an input method that can limit the input of the artificial intelligence model to a specific range, so as to better control the output of the model. Compared with general classification technology, the prompting technology can be better used to fine-tune and optimize the model, so that it is more suitable for certain specific tasks. For example: Fine-tune is a traditional classification training method, which requires a large amount of data to train the model so that it can learn to complete specific tasks. This method usually takes a lot of time and computing resources, and requires more manual participation. However, Fine-tune requires more time and computing resources, but it can only make the model better adapt to specific tasks, but not better adapt to different task scenarios. Prompt is more flexible and easy to control, suitable for different task scenarios.

[0043] Therefore, in an embodiment of the present application, an open source general large model can be selected, for example: an open source natural language large model GLM (General Language Model). Furthermore, after determining the large model, different classification training corpora can be set based on the user's usage scenarios, so that the language large model GLM can classify the voice control instructions issued by the user. After the large model completes the classification of the voice control instructions, the analysis strategy corresponding to the classification can be determined in advance. For example: if the classification of the voice control instruction is determined to be a call classification, the analysis strategy corresponding to the call classification can be called, including an analysis strategy for call classification that can detect the name in the voice control instruction, and other information can be analyzed without analysis. After determining the specific name, the phone number corresponding to the name in the call device can be queried first, and then the call device can be controlled to make the call.

[0044] In order to better implement the embodiment of the present application, in one embodiment of the present application, the analysis module analyzes the voice control instruction to obtain the target device, including:

[0045] Determine the device type of each device; determine the target device type corresponding to the target task type; determine that the device corresponding to the target device type is the target device obtained by the voice control instruction.

[0046] In the embodiment of the present application, since the application scenario can be a home environment. Therefore, in the home environment, there are various types of devices, including air conditioners, televisions, etc. The air conditioner is used for ambient temperature control, and the television is used for video playback. Therefore, the devices can be divided into types according to different functions. It should be noted that in the embodiment of the present application, the device type can be pre-set for each device in the environmental area. After the voice control command is obtained, the device type of each device can be determined according to the pre-set device type.

[0047] In addition, according to the above embodiment, the target task type of the target task can be determined. Based on this, the task type can be matched with the device type to determine the specific execution device. It should be noted that in the embodiment of the present application, the task type can also be set in advance. Specifically, the setting method of the task type can be the same as the setting method of the device type, for example, the type is divided according to the function that the task needs to achieve. For example: the voice command is to lower the temperature by two degrees. At this time, it can be determined according to the voice that the task type is the ambient temperature control type, which is the same as the ambient temperature control type of the air conditioner device, and then the task type can be matched with the device type. Therefore, it can be determined that the air conditioner is the target device for lowering the temperature by two degrees.

[0048] In order to better implement the embodiment of the present application, in one embodiment of the present application, the analysis module is used to determine that the device corresponding to the target device type is the target device obtained by the voice control instruction, including:

[0049] If the target device type corresponds to multiple initial target devices, obtain the command receiving position for receiving the voice control command and determine the device position of each initial target device; determine the initial target device with the shortest distance between the device position of each initial target device and the command receiving position as the target device.

[0050] The above embodiment provides an implementation method for matching the task type with the device type. However, in actual application scenarios, there may be multiple devices of the same type. Therefore, when there are multiple devices of the same type, it is also necessary to determine which device specifically performs the target task. For example: In a home environment, there may be multiple air conditioners. When an instruction to reduce the temperature by two degrees is obtained, there is still no way to determine the target air conditioner that specifically performs the cooling. Therefore, in the embodiment of the present application, multiple voice acquisition devices can be added to the regional environment, such as one in the living room, one in bedroom 1, one in bedroom 2, one in the study, and so on. At this time, according to the voice control instruction obtained by the voice acquisition device, it can be determined that the user is in the living room, bedroom 1, bedroom 2 or study, and then it can be determined which air conditioner in the living room, bedroom 1, bedroom 2 or study is the shortest distance between the user and the air conditioner, so as to determine the air conditioner in the specific room as the target air conditioner. It should be noted that the above embodiment only lists air conditioners for illustration. In actual situations, the air conditioner can be any terminal device that can communicate with the network, such as a TV, a computer, a tablet computer, a mobile phone, etc., and the embodiment of the present application is not limited.

[0051] In order to better implement the embodiments of the present application, in one embodiment of the present application, the analysis module is used to extract keywords from the voice control instructions based on the target task type to obtain the target task, including:

[0052] Based on the target task type, keywords are extracted from the voice control instruction to obtain at least one target keyword; based on each target keyword, a target task is obtained.

[0053] The above embodiment provides a scheme for determining the task type. Since different task types perform different tasks. Therefore, in order to improve the accuracy of identifying tasks, keyword extraction can be performed based on different task types and according to different strategies. After the target task type is determined according to the scheme of the above embodiment, for example, assuming that the input voice control instruction can be "help me set the air conditioner to cooling mode", when the current task type is determined to be the target task type of the device control type according to the voice information, the extracted keywords can be the device name and function. At this time, for the input voice of "help me set the air conditioner to cooling mode", the extracted device name keyword is air conditioner, and the extracted function keyword is cooling mode. Specifically, the method of keyword extraction can determine the semantics of the input voice through a BERT language model, a BILSTM model, etc., so as to extract the corresponding keywords. Since, in the case of the task type of the device control type, it is only necessary to determine the specific task performed by the device, and there is no need to extract the subject, such as the keyword "I", as in other scenarios, so the number of keyword extraction is reduced, the amount of data for subsequent keyword analysis is reduced, and it will not be disturbed by invalid keywords, thereby improving the accuracy of the analysis.

[0054] In order to better implement the embodiments of the present application, in one embodiment of the present application, the analysis module is used to extract keywords from the voice control instruction based on the target task type to obtain at least one target keyword, including:

[0055] Based on the target task type, at least one keyword extraction type corresponding to the target task type is determined; based on each keyword extraction type, the voice control instruction is analyzed to obtain the target keyword matching each keyword extraction type; if there are some target keyword types that do not match the corresponding target keyword, it is determined that the target keyword corresponding to the target keyword type is empty.

[0056] According to the above embodiments, the extraction of unnecessary keywords can be reduced according to the target task type. However, in actual situations, for example, when the user is performing a task of the type of device control, other key information will also be included. For example: "Help me set the air conditioner to 26 degrees in cooling mode." At this time, according to the input voice, it can be obtained that the user has further requirements for the temperature. Therefore, in order to reduce the number of keyword extractions, it is also necessary to ensure that the extraction of some keywords is not omitted. The embodiments of the present application can also set corresponding keyword extraction types for different task types. For example: action (action type), target (target type), condition (condition type) are parsed, where condition (condition type) includes relate (related item), time (time), number (number of times), type (type), device (device) and other different types of keyword extraction types. At this time, according to the input voice "Help me set the air conditioner to 26 degrees in cooling mode" and the corresponding keyword extraction type, the extracted keywords are: {action: setting, target: cooling mode, condition: {relate: 26 degrees, time: none, number: none, type: none, device: air conditioner}}, thus obtaining the target keywords corresponding to the four keyword extraction types of "setting", "cooling mode", "26 degrees", and "air conditioner", and the unextracted keywords corresponding to other keyword extraction types are none. Afterwards, the corresponding tasks can be determined based on the extracted keywords.

[0057] In order to better implement the embodiment of the present application, in one embodiment of the present application, the analysis module is used to obtain the target task based on each of the target keywords, including:

[0058] Based on each of the target keywords, a task execution parameter is determined; and according to the task execution parameter, the target task is obtained.

[0059] According to the above embodiment, in a specific application scenario, when controlling a device, specific execution parameters may be included. Therefore, when the extracted keywords also include some parameter-related keywords, the relevant target tasks are determined, and it is also necessary to determine which actual parameters the device needs to operate according to. Therefore, in an embodiment of the present application, after obtaining the target keywords, parameter detection can be performed on each keyword to determine whether there are specific execution parameters, such as 26 degrees in the above embodiment. Afterwards, when the air conditioner switches to cooling mode, the corresponding operating power of the compressor can be determined based on the control parameter 26 degrees.

[0060] In order to better implement the embodiment of the present application, in one embodiment of the present application, the analysis module is further used to:

[0061] If the voice control instruction is analyzed to obtain task classification information representing the control type, the analysis strategy corresponding to the control type is determined; according to the analysis strategy corresponding to the control type, the keywords of the control action and the keywords of the execution object are extracted according to the voice control instruction to obtain the target control task.

[0062] The above embodiment provides a scenario of making a phone call, but in actual situations, especially in the environment of household appliances, the scenario of controlling each household appliance to perform corresponding tasks is more frequent, and this scenario can be regarded as controlling household appliances, and therefore is a control type. For example: the voice control instruction can be "help me set the air conditioner to cooling mode", at this time the analysis module 202 can extract sound features based on the voice control instruction, and determine that the voice control instruction is a control type based on the sound features. At this time, the analysis strategy corresponding to the control type can be to analyze the corresponding control device function. For example: for the control type, a unified feedback format can be designed, and key information can be filled in the unified format. Instruction: "Execute the semantic analysis task, input the voice control instruction, and parse it according to action, target, and condition, where condition includes relate, time, number, type, device and other information. If no corresponding information is found, fill in none. The contents in these corresponding formats are the task execution parameters. Extract the valid information in the voice control instruction, correspond to the above information types, and feedback in json format. Since in the embodiment of the present application, the voice control instruction is: "Help me set the air conditioner to cooling mode", output: "{action: setting, target: cooling mode, condition: {relate: none, t ime:none, number:none, type:none, device:air conditioner}}"}. "Settings" and "Cooling mode" are keywords extracted based on "Help me set the air conditioner to cooling mode". Therefore, the target control task is to control the air conditioner to switch to cooling mode. It should be noted that if there are multiple air conditioners in the area where the user is located, the embodiment of the present application can also install multiple infrared sensors in the space to detect the current distance between each air conditioner and the user, so as to determine a target air conditioner with the shortest distance to the user, and then control the mode of the target air conditioner to switch to cooling mode. In other words, in the embodiment of the present application, the type of device corresponding to the execution of the target control task can also be determined based on the voice control instruction; then, based on the type of device, the distance between each device of the device type in the current target area and the user is determined; then, based on each distance, the specific target device is determined.

[0063] In order to better implement the embodiment of the present application, in one embodiment of the present application, the analysis module is further used to:

[0064] If task classification information representing the playback type is obtained by analyzing the voice control instruction, an analysis strategy corresponding to the playback type is determined; based on the analysis strategy corresponding to the playback type, keywords of the target content to be played are extracted according to the voice control instruction to obtain a target playback task.

[0065] In the embodiment of the present application, a scenario of controlling home devices to play videos is also provided. For example, the voice control instruction may be "I want to watch Zhang San's AAA third episode in 2023 in Cantonese". At this time, the analysis module 202 may extract sound features based on the voice control instruction, and determine that the voice control instruction is a play type based on the sound features. At this time, the analysis strategy corresponding to the play type may be to analyze the relevant information of the target content to be played. For example, for the play type, a unified feedback format may be designed, and key information may be filled in the unified format. Including: parsing the input voice control instruction according to action, target, condition, where condition contains relate, time, number, type, device information, and if no corresponding information is found, fill in none. Extract the valid information in the voice control instruction and feedback it in json format. For example, "I want to watch Zhang San's AAA third episode in 2023 in Cantonese", output: "{action: watch, target: AAA, condition: {relate: Zhang San, time: 2023, number: third episode, type: Cantonese, device: none}}"}. As shown in the embodiment of the present application, if the device does not extract the corresponding keyword, then the device capable of video playback in the target area network can be determined based on the task classification information of the playback type, such as a TV, a mobile phone, etc. At this time, the distance between the user and each device capable of video playback can still be determined by the set infrared sensor, so as to determine the target playback device with the shortest distance, and finally the target playback device can be used to play the target content.

[0066] In order to better implement the embodiment of the present application, in one embodiment of the present application, the analysis module is further used to:

[0067] If task classification information characterizing the calculation type is obtained by analyzing the voice control instruction, the analysis strategy corresponding to the calculation type is determined; according to the analysis strategy corresponding to the calculation type, keywords of the content to be calculated are extracted according to the voice control instruction to obtain the target calculation task.

[0068] In an embodiment of the present application, a scenario of controlling household appliances to perform calculations is also provided. For example, the voice control instruction may be "calculate what is the 10th power of 2", in which case the analysis module 202 may extract sound features based on the voice control instruction, and determine that the voice control instruction is a calculation type based on the sound features. At this time, the analysis strategy corresponding to the calculation type may be to analyze the keywords corresponding to the content to be calculated. Including: parsing the input voice control instruction according to action, target, condition, where condition contains relate, time, number, type, and device information, and if no corresponding information is found, fill in none. Extract the valid information in the voice control instruction and feedback it in json format. For example, the voice control instruction may be "calculate what is the 10th power of 2", output: "{action: calculation, target: power, condition: {relate: none, time: none, number: (2, 10), type: none, device: none}}"}. As shown in the embodiment of the present application, if the device still fails to extract the corresponding keywords, the devices in the network in the target area that can perform the computing task, such as computers, mobile phones and other devices, can be determined based on the task classification information of the computing type. At this time, the distance between the user and each device that can perform the computing task can still be determined by the set infrared sensor, so as to determine the target computing device with the shortest distance, and finally perform the relevant calculations through the target computing device. It should be noted that when completing a computing type task, if the target device performing the calculation also includes a speaker, the speaker of the target device can be controlled to play the calculation results.

[0069] In order to better implement the embodiment of the present application, in one embodiment of the present application, the analysis module is further used to:

[0070] If task classification information representing the translation type is obtained by analyzing the voice control instruction, an analysis strategy corresponding to the translation type is determined; based on the analysis strategy corresponding to the translation type, keywords of the content to be translated and the language to be translated are extracted according to the voice control instruction to obtain a target translation task.

[0071] In the embodiment of the present application, a scenario of controlling home appliances for translation is also provided. For example, the voice control instruction may be "translate apple into English", in which case the analysis module 202 may extract sound features based on the voice control instruction, and determine that the voice control instruction is a translation type based on the sound features. At this time, the analysis strategy corresponding to the translation type may be to analyze the keywords corresponding to the content to be translated. Including: parsing the input voice control instruction according to action, target, condition, where condition contains relate, time, number, type, and device information, and if no corresponding information is found, fill in none. Extracting valid information from the voice control instruction and feeding back the json format. For example, the voice control instruction may be "translate apple into English", output: "{action: translation, target: apple, condition: {relate: none, time: none, number: none, type: English, device: none}}"}. As shown in the embodiment of the present application, if the device still does not extract the corresponding keywords, the device that can perform the translation task in the network of the target area, such as a computer, a mobile phone, etc., may be determined based on the task classification information of the translation type. At this time, the infrared sensor can still be used to determine the distance between the user and each device capable of translation, thereby determining a target translation device with the shortest distance, and finally performing the relevant translation through the target translation device. It should be noted that when completing a translation task, if the target device performing the translation also includes a speaker, the speaker of the target device can be controlled to play the translation result.

[0072] In order to better implement the embodiment of the present application, in one embodiment of the present application, the analysis module is further used to:

[0073] If task classification information characterizing the query type is obtained by analyzing the voice control instruction, an analysis strategy corresponding to the query type is determined; based on the analysis strategy corresponding to the query type, keywords of the content to be queried are extracted according to the voice control instruction to obtain a target query task.

[0074] In the embodiment of the present application, a scenario of controlling home devices to perform inquiries is also provided. For example, the voice control instruction may be "translate apple into English", and the analysis module 202 may extract sound features based on the voice control instruction, and determine that the voice control instruction is a query type based on the sound features. At this time, the analysis strategy corresponding to the query type may be to analyze the keywords corresponding to the content to be queried. Including: parsing the input voice control instruction according to action, target, and condition, wherein the condition contains relate, time, number, type, and device information, and if no corresponding information is found, fill in none. Extract the valid information in the voice control instruction and feedback it in json format. For example, the voice control instruction may be "What month and day is the Dragon Boat Festival", output: "{action: query time, target: Dragon Boat Festival, condition: {relate: none, time: none, number: none, type: none, device: none}}". As shown in the embodiment of the present application, if the device still does not extract the corresponding keyword, the device that can perform the query task in the network of the target area can be determined based on the task classification information of the query type, such as computers, mobile phones and other devices. At this time, the infrared sensor can still be used to determine the distance between the user and each query device, thereby determining the target query device with the shortest distance, and finally performing related queries through the target query device. It should be noted that when completing a query-type task, if the target device for querying also includes a speaker, the speaker of the target device can be controlled to play the query results.

[0075] In order to better implement the embodiment of the present application, in one embodiment of the present application, the analysis module is further used to:

[0076] If task classification information representing the reminder type is obtained by analyzing the voice control instruction, the analysis strategy corresponding to the reminder type is determined; according to the analysis strategy corresponding to the reminder type, the keywords of the content to be reminded and the target reminder time are extracted according to the voice control instruction to obtain the target reminder task.

[0077] In the embodiment of the present application, a scenario of controlling household devices to remind is also provided. For example, the voice control instruction may be "translate apple into English", and the analysis module 202 may extract sound features based on the voice control instruction, and determine that the voice control instruction is a reminder type based on the sound features. At this time, the analysis strategy corresponding to the reminder type may be to analyze the keywords corresponding to the content to be reminded. Including: parsing the input voice control instruction according to action, target, condition, where condition contains relate, time, number, type, device information, and if no corresponding information is found, fill in none. Extract the valid information in the voice control instruction and feedback it in json format. For example, the voice control instruction may be "remind me to buy groceries at 3 pm", output: "{action: reminder, target: buy groceries, condition: {relate: none, time: 3 pm, number: none, type: none, device: none}}"}. As shown in the embodiment of the present application, if the device still does not extract the corresponding keyword, the device that can perform the reminder task in the network of the target area can be determined based on the task classification information of the reminder type, such as computers, mobile phones and other devices. At this time, the distance between the user and each device capable of making a reminder can still be determined by the set infrared sensor, so as to determine the target reminder device with the shortest distance, and finally the relevant reminder can be made through the target reminder device. It should be noted that when completing a reminder-type task, if the target device that executes the reminder also includes a speaker, the speaker of the target device can be controlled to play the reminder result. At the same time, since the above embodiment provides a method for identifying whether the voice control instruction issued by the user includes the authority, the user's identity information can be determined by the voice features in the voice control instruction. Therefore, the relevant reminder information can also be sent to the mobile phone of the user whose identity information matches through the network, text messages, etc.

[0078] In order to better implement the embodiment of the present application, in one embodiment of the present application, the analysis module is used to analyze the voice control instruction to obtain the target device and target task corresponding to the voice control instruction, including:

[0079] Determine the sound characteristics of the voice control instruction; determine the control authority of the voice characteristics based on the sound characteristics; if the sound characteristics include the control authority, analyze the voice control instruction to obtain the target device and target task corresponding to the voice control instruction.

[0080] Further, after acquiring the voice control instruction issued by the user, the terminal device may first analyze whether the voice control instruction is a valid voice control instruction when analyzing through the analysis module 202. For example, analyzing whether the voice control instruction issued by the current user has the corresponding control authority.

[0081] Therefore, the analysis module 202 can extract the sound features in the voice control instruction and analyze whether the extracted sound features have corresponding control authority, for example, by comparing with the sound features with control authority stored in the terminal device. If the similarity exceeds 90%, it can be determined that the current voice control instruction is a valid voice control instruction. Among them, the method of determining the similarity of two sounds can be determined by calculating the Euclidean distance, cosine similarity, etc. between the two sound feature vectors, and the specific embodiments of the present application are not limited.

[0082] In order to better implement the embodiment of the present application, in one embodiment of the present application, the analysis module determines the control authority of the sound feature according to the sound feature, including:

[0083] Determine whether there is a noise feature in the sound feature; if there is a noise feature in the sound feature, remove the noise feature to obtain a processed sound feature; and determine the control authority of the sound feature based on the processed sound feature.

[0084] Alternatively, the analysis module 202 may also first analyze whether there is noise in the currently received voice control instruction, for example, the current voice control instruction may also be subjected to sound feature extraction, and whether there is environmental noise in the sound feature, etc., to determine whether there is noise in the voice control instruction. If there is more noise in the voice control instruction, that is, other non-user sounds account for more than 30% of the user's voice, or other non-user sound decibels exceed 20 decibels, then the voice control instruction can be determined to be an invalid voice control instruction, otherwise it can be a valid voice control instruction. Afterwards, the relevant noise features are removed, and after the processed sound features are obtained, the process of determining the authority in the above embodiment is executed. Alternatively, the analysis module 202 can determine whether the voice control instruction issued by the current user has control authority, and can also determine whether there is more noise in the current voice control instruction at the same time. When both pass the analysis, it is a valid instruction, otherwise, when only one of the two passes the analysis, or both do not pass the analysis, it is an invalid voice control instruction.

[0085] In order to better implement the device control method in the embodiment of the present application, in addition to the device control device based on the pre-trained large model, the embodiment of the present application also provides a device control method based on the pre-trained large model, such as Figure 3 As shown, including:

[0086] 301. Obtain voice control instructions;

[0087] 302. Analyze the voice control instruction to obtain a target device and a target task corresponding to the voice control instruction;

[0088] 303. Control the target device to execute the target task.

[0089] The device control method provided by the present application receives the user's voice control command, and then determines the user's specific task requirements and target device execution in the voice control command. After that, the corresponding target device is controlled to perform the specific target task. The device control method of the present application can control each device through a common model, without the need to set up different intelligent control models inside each device, thereby improving the versatility of device control and saving storage space for storing intelligent models inside the device.

[0090] In order to better implement the embodiments of the present application, in one embodiment of the present application, analyzing the voice control instruction to obtain the target task includes:

[0091] Analyze the task type of the voice control instruction and obtain the target task type;

[0092] Based on the target task type, keywords are extracted from the voice control instructions to obtain the target task.

[0093] In order to better implement the embodiments of the present application, in one embodiment of the present application, analyzing the voice control instruction to obtain the target device includes:

[0094] Determine the device type of the device;

[0095] Determine a target device type that matches the target task type from the device types;

[0096] Determine that the device corresponding to the target device type is the target device obtained by the voice control instruction.

[0097] In order to better implement the embodiment of the present application, in one embodiment of the present application, determining that the device corresponding to the target device type is a target device for a voice control instruction includes:

[0098] If the target device type corresponds to multiple initial target devices, obtaining a command receiving position for receiving the voice control command and determining a device position of each initial target device;

[0099] The initial target device with the shortest distance between the device position of each initial target device and the command receiving position is determined as the target device.

[0100] In order to better implement the embodiment of the present application, in one embodiment of the present application, based on the target task type, keyword extraction is performed on the voice control instruction to obtain the target task, including:

[0101] Based on the target task type, extract keywords from the voice control instruction to obtain at least one target keyword;

[0102] Based on each target keyword, a target task is obtained.

[0103] In order to better implement the embodiments of the present application, in one embodiment of the present application, based on the target task type, the voice control instruction is subjected to keyword extraction to obtain at least one target keyword, including:

[0104] Based on the target task type, determining at least one keyword extraction type corresponding to the target task type;

[0105] Based on each keyword extraction type, the voice control instruction is analyzed to obtain a target keyword that matches each keyword extraction type;

[0106] If there are some target keyword types that do not match the corresponding target keywords, it is determined that the target keyword corresponding to the target keyword type is empty.

[0107] In order to better implement the embodiment of the present application, in one embodiment of the present application, based on each target keyword, a target task is obtained, including:

[0108] Determine task execution parameters based on each target keyword;

[0109] According to the task execution parameters, the target task is obtained.

[0110] In order to better implement the embodiment of the present application, in one embodiment of the present application, the voice control instruction is analyzed to obtain the target task corresponding to the voice control instruction, including:

[0111] Analyze the voice control command to obtain task classification information of the voice control command;

[0112] Determine the corresponding analysis strategy based on the task classification information;

[0113] According to the corresponding analysis strategy, keywords are extracted from the voice control instructions to obtain the target task.

[0114] In order to better implement the embodiment of the present application, in one embodiment of the present application, the voice control instruction is analyzed to obtain the target task corresponding to the voice control instruction, including:

[0115] If task classification information representing the control type is obtained by analyzing the voice control command, an analysis strategy corresponding to the control type is determined;

[0116] According to the analysis strategy corresponding to the control type, the keywords of the control action and the keywords of the execution object are extracted according to the voice control instruction to obtain the target control task.

[0117] In order to better implement the embodiment of the present application, in one embodiment of the present application, the voice control instruction is analyzed to obtain the target task corresponding to the voice control instruction, including:

[0118] If task classification information representing the playback type is obtained by analyzing the voice control instruction, an analysis strategy corresponding to the playback type is determined;

[0119] According to the analysis strategy corresponding to the playback type, keywords of the target content to be played are extracted according to the voice control instruction to obtain the target playback task.

[0120] In order to better implement the embodiment of the present application, in one embodiment of the present application, the voice control instruction is analyzed to obtain the target task corresponding to the voice control instruction, including:

[0121] If task classification information representing the computing type is obtained by analyzing the voice control instruction, an analysis strategy corresponding to the computing type is determined;

[0122] According to the analysis strategy corresponding to the calculation type, the keywords of the content to be calculated are extracted according to the voice control instructions to obtain the target calculation task.

[0123] In order to better implement the embodiment of the present application, in one embodiment of the present application, the voice control instruction is analyzed to obtain the target task corresponding to the voice control instruction, including:

[0124] If task classification information representing the translation type is obtained by analyzing the voice control instruction, an analysis strategy corresponding to the translation type is determined;

[0125] According to the analysis strategy corresponding to the translation type, the content to be translated and the keywords of the language to be translated are extracted according to the voice control instructions to obtain the target translation task.

[0126] In order to better implement the embodiment of the present application, in one embodiment of the present application, the voice control instruction is analyzed to obtain the target task corresponding to the voice control instruction, including:

[0127] If task classification information representing the query type is obtained by analyzing the voice control instruction, an analysis strategy corresponding to the query type is determined;

[0128] According to the analysis strategy corresponding to the query type, the keywords of the content to be queried are extracted according to the voice control instructions to obtain the target query task.

[0129] In order to better implement the embodiment of the present application, in one embodiment of the present application, the voice control instruction is analyzed to obtain the target task corresponding to the voice control instruction, including:

[0130] If task classification information representing the reminder type is obtained by analyzing the voice control instruction, an analysis strategy corresponding to the reminder type is determined;

[0131] According to the analysis strategy corresponding to the reminder type, the keywords of the content to be reminded and the target reminder time are extracted according to the voice control instruction to obtain the target reminder task.

[0132] The present application also provides a terminal device, which includes a processor, a memory, and a computer program stored in the memory and executable on the processor. The processor executes the computer program to implement the steps of any device control method in the present application and any device control device equipped with the device control method. The terminal device integrates any device control method provided in the present application and any device control device equipped with the device control method. Figure 4 As shown, it shows a schematic diagram of the structure of the terminal device involved in the embodiment of the present application, specifically:

[0133] The terminal device may include one or more processing core processors 401, one or more computer-readable storage media memories 402, a power supply 403, an input unit 404 and other components. Those skilled in the art will appreciate that Figure 4 The terminal device structure shown in the figure does not constitute a limitation on the terminal device, and may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. Among them:

[0134] The processor 401 is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device. By running or executing software programs and / or modules stored in the memory 402, and calling data stored in the memory 402, the processor 401 executes various functions of the terminal device and processes data, thereby monitoring the terminal device as a whole. Optionally, the processor 401 may include one or more processing cores; the processor 401 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. Preferably, the processor 401 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, the user interface and the application program, etc., and the modem processor mainly processes wireless communication. It is understandable that the above-mentioned modem processor may not be integrated into the processor 401.

[0135] The memory 402 can be used to store software programs and modules. The processor 401 executes various functional applications and data processing by running the software programs and modules stored in the memory 402. The memory 402 may mainly include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, or other volatile solid-state storage devices. Accordingly, the memory 402 may also include a memory controller to provide the processor 401 with access to the memory 402.

[0136] The terminal device also includes a power supply 403 for supplying power to each component. Preferably, the power supply 403 can be logically connected to the processor 401 through a power management system, so as to manage charging, discharging, power consumption management and other functions through the power management system. The power supply 403 can also include any components such as one or more DC or AC power supplies, recharging systems, power failure detection circuits, power converters or inverters, and power status indicators.

[0137] The terminal device may further include an input unit 404, which may be used to receive input digital or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

[0138] Although not shown, the terminal device may further include a display unit, etc., which will not be described in detail herein. Specifically in this embodiment, the processor 401 in the terminal device will load the executable files corresponding to the processes of one or more application programs into the memory 402 according to the following instructions, and the processor 401 will run the application programs stored in the memory 402, thereby realizing various functions, such as:

[0139] Get voice control instructions;

[0140] Analyze the voice control command to obtain the target device and target task corresponding to the voice control command;

[0141] Control the target device to perform the target task.

[0142] Those skilled in the art will appreciate that all or part of the steps in the various schemes of the above embodiments may be completed by instructions, or by controlling related hardware through instructions. The instructions may be stored in a computer-readable storage medium and loaded and executed by a processor.

[0143] To this end, the embodiment of the present application provides a computer-readable storage medium, which may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc. A computer program is stored thereon, and the computer program is loaded by a processor to execute the steps in any device control method provided in the embodiment of the present application. For example, the computer program loaded by the processor can execute the following steps:

[0144] Get voice control instructions;

[0145] Analyze the voice control command to obtain the target device and target task corresponding to the voice control command;

[0146] Control the target device to perform the target task.

[0147] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, please refer to the detailed description of other embodiments above, and will not be repeated here.

[0148] In specific implementation, the above units or structures can be implemented as independent entities, or can be arbitrarily combined to be implemented as the same or several entities. The specific implementation of the above units or structures can refer to the previous method and device embodiments, which will not be repeated here.

[0149] The specific implementation of the above operations can be found in the previous embodiments, which will not be described in detail here.

[0150] The above is a detailed introduction to a device control method and apparatus provided in an embodiment of the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A device control device based on a pre-trained large model, characterized in that: The device comprises: An acquisition module, used to acquire voice control instructions; An analysis module, used to analyze the voice control instruction to obtain a target device and a target task corresponding to the voice control instruction; A control module is used to control the target device to execute the target task.

2. The device control device based on the pre-trained large model according to claim 1, characterized in that: The analyzing module analyzes the voice control instruction to obtain a target task, including: Analyze the task type of the voice control instruction to obtain a target task type; Based on the target task type, keywords are extracted from the voice control instruction to obtain the target task.

3. The device control device based on the pre-trained large model according to claim 2, characterized in that: The analyzing module analyzes the voice control instruction to obtain the target device, including: Determine the device type of the device; Determining a target device type matching the target task type from the device types; The device corresponding to the target device type is determined to be the target device obtained by the voice control instruction.

4. The device control device based on the pre-trained large model according to claim 3 is characterized in that: The analyzing module is used to determine that the device corresponding to the target device type is the target device obtained by the voice control instruction, including: If the target device type corresponds to multiple initial target devices, obtaining a command receiving position for receiving the voice control command and determining a device position of each of the initial target devices; The initial target device having the shortest distance between the device position of each of the initial target devices and the instruction receiving position is determined as the target device.

5. The device control device based on the pre-trained large model according to claim 2, characterized in that: The analysis module is used to extract keywords from the voice control instruction based on the target task type to obtain the target task, including: Based on the target task type, extracting keywords from the voice control instruction to obtain at least one target keyword; Based on each of the target keywords, the target task is obtained.

6. The device control device based on the pre-trained large model according to claim 5, characterized in that: The analysis module is used to extract keywords from the voice control instruction based on the target task type to obtain at least one target keyword, including: Based on the target task type, determining at least one keyword extraction type corresponding to the target task type; Based on each of the keyword extraction types, analyzing the voice control instruction to obtain a target keyword that matches each of the keyword extraction types; If there are some target keyword types that do not match the corresponding target keywords, it is determined that the target keyword corresponding to the target keyword type is empty.

7. The device control device based on the pre-trained large model according to claim 5, characterized in that: The analysis module is used to obtain the target task based on each of the target keywords, including: Determining task execution parameters based on each of the target keywords; The target task is obtained according to the task execution parameters.

8. A device control method based on a pre-trained large model, characterized in that: The method comprises: Get voice control instructions; Analyze the voice control command to obtain a target device and a target task corresponding to the voice control command; Control the target device to execute the target task.

9. A terminal device, characterized in that: The terminal device includes a processor, a memory, and a computer program stored in the memory and executable on the processor. The terminal device is equipped with a device control device based on a pre-trained large model as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, which is executed by a processor to control the device control apparatus based on the pre-trained large model as described in any one of claims 1 to 7.

Images