Equipment control method and device, equipment, storage medium and program product

Abstract

The embodiment of the invention provides an equipment control method and device, equipment, a storage medium and a program product. The method comprises the following steps: receiving an environment image and voice information sent by wearable equipment; the environment image is acquired in real time when the wearable equipment receives voice information used for indicating equipment control; determining a target device corresponding to the voice information based on the environment image and a plurality of pre-stored scene images; and based on the target device and the voice information, generating a device control instruction and sending the device control instruction to the control server, thereby realizing control of the target device through the control server. The method is used for breaking through the pickup distance limitation of an existing voice receiver, preventing a user from memorizing and distinguishing a large number of intelligent devices, and improving the simplicity and flexibility of intelligent control.

Description

Technical Field

[0001] This application relates to the field of artificial intelligence technology, and particularly to a device control method, device, equipment, storage medium and program product. Background Art

[0002] With the development of Internet of Things and artificial intelligence technologies, the smart home control system has gradually become an important part of modern home life. This system can connect various household appliances and devices through a network, and users can remotely manage and control the lights, temperature control, security, etc. in the home through a mobile phone client, voice control, etc., thereby improving the convenience and comfort of life.

[0003] Although smart home control provides certain convenience, the existing smart home control methods still have some significant defects: on the one hand, users need to manually name multiple smart devices to distinguish multiple smart devices in a scene; on the other hand, if users control smart devices by voice control, they need to ensure that the distance from the voice receiver is less than the sound pickup range of the voice receiver. When the user is far from the voice receiver, the user's voice may not be accurately received, which seriously affects the immediacy and reliability of control.

[0004] These problems not only increase the usage complexity for users, but also reduce the usage experience of smart homes to a certain extent. Summary of the Invention

[0005] Embodiments of this application provide a device control method, device, equipment, storage medium and program product, so as to achieve the effect that users do not need to remember the names of various controllable devices, and increase the control flexibility and timeliness of controllable devices.

[0006] In a first aspect, embodiments of this application provide a device control method, which is applied to an AI cloud platform server that is respectively communicatively connected to a wearable device and a control server. The method includes:

[0007] Receiving the environmental image and voice information sent by the wearable device; the environmental image is obtained by the wearable device in real time when it receives the voice information for instructing device control;

[0008] Based on the environmental image and multiple pre-stored scene images, determining the target device corresponding to the voice information;

[0009] Generating a device control instruction based on the target device and the voice information, and sending it to the control server to implement control of the target device through the control server.

[0010] In a possible implementation, receiving the environmental image and voice information sent by the wearable device further includes:

[0011] Receiving the target text corresponding to the environmental image and the voice information sent by the wearable device; the target text is obtained by the wearable device through text recognition of the voice information;

[0012] Determining the target device corresponding to the voice information based on the environmental image and multiple pre-stored scene images includes:

[0013] Based on the environmental image and the scene images, determining the target device corresponding to the target text;

[0014] Generating a device control instruction based on the target device and the voice information includes:

[0015] Based on the target device and the target text, generating the device control instruction.

[0016] In a possible implementation, before determining the target device corresponding to the voice information based on the environmental image and multiple pre-stored scene images, it further includes:

[0017] Receiving a marking instruction for the scene image, and determining the controllable devices included in the scene image based on the marking instruction;

[0018] Obtaining the device features corresponding to each of the controllable devices, and storing the device features in correspondence with the corresponding controllable devices.

[0019] In a possible implementation, before determining the target device corresponding to the voice information based on the environmental image and multiple pre-stored scene images, it further includes:

[0020] Determining the target device type corresponding to the voice information;

[0021] Determining the target device corresponding to the voice information based on the environmental image and multiple pre-stored scene images includes:

[0022] Performing scene recognition on the environmental image to determine at least one controllable device included in the environmental image;

[0023] When at least one controllable device included in the environmental image includes at least one first device to be screened that conforms to the target device type, determine the target device from the at least one first device to be screened; when the controllable devices corresponding to the environmental image do not include the first device to be screened that conforms to the target device type, determine the target device from at least one second device to be screened that conforms to the target device type included in the scene image.

[0024] In a possible implementation manner, the performing scene recognition on the environmental image to determine at least one controllable device included in the environmental image includes:

[0025] Match at least one scene image that is in the same scene as the environmental image from multiple scene images;

[0026] Use the controllable devices included in the scene image corresponding to the environmental image as at least one controllable device included in the environmental image.

[0027] In a possible implementation manner, the determining the target device from at least one second device to be screened that conforms to the target device type included in the scene image includes:

[0028] Generate a prompt message based on at least one second device to be screened that conforms to the target device type included in the scene image;

[0029] Receive a selection instruction corresponding to the prompt message, and obtain the target device feature corresponding to the selection instruction;

[0030] Compare and match the target device feature and the device feature corresponding to the second device to be screened, so as to determine the target device from the second devices to be screened.

[0031] In a second aspect, an embodiment of the present application provides a device control device, including:

[0032] A receiving module, configured to receive the environmental image and voice information sent by the wearable device; the environmental image is obtained by the wearable device in real time when receiving the voice information for instructing device control;

[0033] A determining module, configured to determine the target device corresponding to the voice information based on the environmental image and multiple pre-stored scene images;

[0034] A generating module, configured to generate a device control instruction based on the target device and the voice information and send it to the control server, so as to implement control of the target device through the control server.

[0035] In a third aspect, an embodiment of the present application provides an electronic device, including: a memory and a processor;

[0036] The memory stores computer-executable instructions;

[0037] The processor executes the computer-executable instructions stored in the memory, so that the processor executes the above first aspect and / or various possible implementation manners of the first aspect.

[0038] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the computer-executable instructions are executed by a processor, they are used to implement the above first aspect and / or various possible implementation manners of the first aspect.

[0039] In a fifth aspect, an embodiment of the present application provides a computer program product, including a computer program, and when the computer program is executed by a processor, it implements the above first aspect and / or various possible implementation manners of the first aspect.

[0040] The device control method, device, device, storage medium and program product provided by the embodiments of the present application enable the user of the wearable device to collect images of the current scene through the wearable device in a voice-activated manner without being affected by daily activities, and use the AI cloud platform server to identify the controllable devices in the current scene, and control the identified controllable devices according to the content of the voice, so as to break through the pickup distance limitation of the existing voice receiver while avoiding the user's memory and distinction of a large number of intelligent devices, and improving the simplicity and flexibility of intelligent control. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The drawings here are incorporated into the specification and form a part of the specification, showing embodiments consistent with the present application, and are used together with the specification to explain the principles of the present application.

[0042] Figure 1 It is a schematic diagram of the scenario of the device control method provided by the present application;

[0043] Figure 2 It is a schematic flowchart of the device control method applied to the wearable device provided by the present application;

[0044] Figure 3 It is a schematic structural diagram of the device control device provided by the present application;

[0045] Figure 4 It is a schematic structural diagram of the electronic device provided by the present application.

[0046] Through the above-mentioned accompanying drawings, specific embodiments of the present application have been shown, and will be described in more detail hereinafter. These drawings and the written description are not intended to limit the scope of the concept of the present application in any way, but to illustrate the concept of the present application to those skilled in the art by referring to specific embodiments. Detailed Embodiments

[0047] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0048] The device control method provided by the embodiments of the present application can be applied to an application environment as Figure 1 shown. Among them, the wearable device 102, the control server 104, and the AI cloud platform server 106 are respectively connected to each other, and the wearable device 102 and the control server 104 can communicate with a plurality of controllable devices.

[0049] The AI cloud platform server 106 refers to a server that provides artificial intelligence-related computing, storage, and services. The AI cloud platform server 106 is equipped with high-performance hardware resources to support AI model training, inference, data processing, and various intelligent applications.

[0050] For example, when the device control method is applied to the AI cloud platform server 106, the wearable device 102 can continuously collect voice information; and when the voice information indicates device control, collect the environmental image corresponding to the current environment; subsequently, the wearable device 102 can send the environmental image and the voice information to the AI cloud platform server 106 together. The AI cloud platform server 106 can receive the environmental image and the voice information sent by the wearable device 102; and obtain multiple pre-stored scene images from the data storage system of the control server 104, and then determine the target device corresponding to the voice information based on the environmental image and the multiple scene images; finally, generate a device control instruction based on the target device and the voice information and send it to the control server 104 to implement the control of the target device through the control server 104. The control server 104 can be implemented by an independent server or a server cluster composed of multiple servers. The wearable device 102, the control server 104, and the AI cloud platform server 106 can be directly or indirectly connected through wired or wireless communication means, such as through a network connection. It can be understood that the data storage system of the control server 104 can be an independent storage device, or the data storage system is located on the control server 104, or the data storage system is located on another terminal.

[0051] As an example, the AI cloud platform server 106 can also obtain multiple pre-stored scene images from its corresponding data storage system.

[0052] In one embodiment, a device control method is provided. This embodiment is exemplified by the method being applied to an AI cloud platform server. As Figure 2 shown, the device control method includes:

[0053] Step 202, receiving an environmental image and voice information sent by a wearable device; the environmental image is obtained by the wearable device in real time when receiving the voice information used to indicate device control.

[0054] As an example, the wearable device can use built-in microphones or voice sensors and other recording devices to record or obtain the user's voice input for a long time and continuously to achieve specific functions (such as voice control, real-time translation, environmental analysis, etc.).

[0055] Wearable devices are usually equipped with high-sensitivity microphone arrays for directional sound pickup, noise reduction, or far-field voice capture. Some wearable devices support low-power always-on microphones, which only collect complete voice information after detecting a specific wake word. The wake word can be preset by the user through a terminal communicatively connected to the wearable device. The human-computer interaction interface of the terminal can specifically display a specified platform interface related to the wearable device. The user inputs text through a custom component on the specified platform interface to set the wake word of the wearable device. Among them, the terminal can be, but is not limited to, various personal computers, laptop computers, smartphones, tablet computers, and Internet of Things devices. The Internet of Things devices can be smart speakers, smart TVs, smart air conditioners, smart in-vehicle devices, etc.

[0056] Alternatively, the user can also trigger the voice collection of the wearable device by clicking on a virtual power-on button representing power supply to the wearable device on the specified platform interface.

[0057] In this embodiment, a collection device is provided on the wearable device. The collection device can include cameras, cameras, etc. for image and video collection, or can also include recording devices for voice collection, etc.

[0058] Among them, the voice information used to indicate device control refers to the voice information applied to manage the state of the device. The voice information used to indicate device control can, for example, contain two key contents, namely the control object and the control content. For example, if the voice information is "Set the light to warm yellow, 50% brightness", it can be known that the control object is the "light" and the control content is "Set to warm yellow, 50% brightness".

[0059] The environmental image refers to the visual representation of the current environment, which can be a static photo or a dynamic video stream captured by the image acquisition device built into the wearable device from the current perspective. If the current environment is an indoor environment, the environmental image can, for example, include the indoor spatial layout, the structure and levels of the room (such as the positions of walls, doors, and windows), and the furniture arrangement (sofas, tables, chairs, bookshelves, etc.).

[0060] As an example, the wearable device can preprocess the captured environmental image, such as eliminating the noise in the environmental image to improve the accuracy of subsequent processing; adjusting the contrast, brightness, and saturation to make the environmental features included in the environmental image more prominent.

[0061] Step 204: Based on the environmental image and multiple pre-stored scene images, determine the target device corresponding to the voice information.

[0062] In one embodiment, the AI cloud platform server can, for example, perform semantic recognition on the voice information to determine the type of the target device corresponding to the voice information, and then perform scene recognition on the environmental image to determine all the controllable devices included in the current environmental image, and then screen out the controllable devices that meet the target device type from these controllable devices as the target device.

[0063] Alternatively, the AI cloud platform server can directly use all the controllable devices included in the current environmental image as the target device.

[0064] Step 206: Based on the target device and the voice information, generate a device control instruction and send it to the control server to control the target device through the control server.

[0065] As an example, the AI cloud platform server can generate a device control instruction based on the device information corresponding to the target device and the control content included in the voice information.

[0066] This device control instruction is used to instruct the control server to perform specific control on a specific target device.

[0067] After receiving the device control instruction, the control server can perform corresponding device control operations according to the device control instruction and the actual state of the target device.

[0068] In one embodiment, since both the wearable device and the control server can be communicatively connected to the controllable device, the AI cloud platform server can also generate a device control instruction based on the target device and the voice information and send it to the wearable device to control the target device through the wearable device.

[0069] The device control method provided by the embodiments of the present application enables the user of the wearable device to collect images of the current scene through voice activation by the wearable device without being affected by daily activities, and uses the AI cloud platform server to identify the controllable devices in the current scene, and control the identified controllable devices according to the content of the voice, thereby breaking through the pickup distance limitation of the existing voice receiver while avoiding the need for users to memorize and distinguish a large number of smart devices, improving the simplicity and flexibility of intelligent control.

[0070] In some alternative embodiments, step 202 includes:

[0071] Receiving the environmental image and the target text corresponding to the voice information sent by the wearable device; the target text is obtained by the wearable device through text recognition of the voice information;

[0072] Step 204 includes:

[0073] Based on the environmental image and the scene image, determining the target device corresponding to the target text;

[0074] Step 206 includes:

[0075] Generating a device control instruction based on the target device and the target text.

[0076] In this embodiment, the wearable device can perform text recognition on the received voice information and obtain the target text corresponding to the voice information.

[0077] As an example, the wearable device can convert the voice signal into the corresponding target text through a voice recognition engine (such as Google Speech-to-Text, Microsoft Azure Speech Service). This step usually involves a deep learning model to improve the recognition accuracy.

[0078] In one embodiment, after obtaining the target text, the wearable device can also perform semantic analysis on it to identify the intention of the target text and determine that the voice information is used for device control. The process of intention recognition can include, for example: using natural language processing (NLP) technology to identify nouns and text other than nouns in the target text; then comparing the text other than nouns with the texts of multiple pre-set device control instructions for similarity to determine that the target text is used for device control. The above process of similarity comparison can adopt methods such as cosine similarity or edit distance calculation. When the similarity between the text other than nouns and the text of a pre-set device control instruction reaches a preset threshold, it is determined that the target text is used for device control.

[0079] In the above device control method, the user of the wearable device can use colloquial expressions without having to follow a specific instruction format, which reduces the user's usage threshold. The user's input is closer to natural language, making the interaction between the user and the wearable device more intuitive and convenient. The wearable device can accurately determine whether the user's speech is used for device control through similarity comparison. After determining that it is used for device control, it collects the environmental image of the current environment, thereby enhancing the fault tolerance ability of the wearable system for user expressions.

[0080] In some optional embodiments, before step 204, it further includes:

[0081] Receiving a marking instruction for the scene image, and determining the controllable device included in the scene image based on the marking instruction;

[0082] Obtaining the device features corresponding to each controllable device, and storing the device features in correspondence with the corresponding controllable devices.

[0083] As an example, the user can pre-take multiple scene images through the wearable device. The wearable device can further display the scene images through the human-computer interaction interface of the terminal, and at the same time send these scene images to the control server for storage. The user can send a marking instruction by touching, clicking, selecting the position on the scene image that represents the controllable device, etc. The terminal can send the marking instruction to the control server or the AI cloud platform server. The control server or the AI cloud platform server can, for example, perform target detection on the position on the scene image corresponding to the marking instruction based on the marking instruction, thereby determining the controllable device at the corresponding position of the marking instruction, and can number each controllable device on the scene image to determine the unique representation corresponding to each controllable device for distinguishing and identification.

[0084] Subsequently, the control server or the AI cloud platform server can extract features from the corresponding positions of each controllable device included in the scene image to obtain the device features corresponding to the controllable devices. The device features can include appearance features, positional relationship features, etc., which are used to help identify different controllable devices and avoid confusing information. For example, the appearance features can involve the device type, color, shape, label information of the controllable device, etc. The positional relationship features can involve the relative position of the controllable device in the current scene, the proximity relationship between the controllable device and other objects in the current scene, etc.

[0085] Furthermore, the control server or the AI cloud platform server can associate and store the unique number corresponding to each controllable device with the device features corresponding to the controllable device.

[0086] In the above device control method, by receiving the marking instructions issued by the user for the scene image to determine the controllable devices included in different scene images, it can ensure that the identified controllable devices are consistent with the controllable devices existing in the actual scene, thereby improving the recognition accuracy, avoiding the omission of the recognition of controllable devices, and the user does not need to input device information through cumbersome operations. Just simply click on the position representing the controllable device on the scene image to achieve the automatic recognition of the controllable device, greatly simplifying the operation process; moreover, by identifying the device characteristics corresponding to each controllable device, the unique number of each controllable device can be bound to its device characteristics. Subsequently, when the user uses the wearable device, the controllable device can be quickly determined by giving various device characteristics without the need to remember the specific name of the controllable device, improving the user experience.

[0087] In some alternative embodiments, before step 204, it further includes:

[0088] Determine the target device type corresponding to the voice information;

[0089] Step 204 includes:

[0090] Perform scene recognition on the environmental image to determine at least one controllable device included in the environmental image;

[0091] When at least one of the at least one controllable device included in the environmental image includes at least one first device to be screened that conforms to the target device type, determine the target device from the first devices to be screened; when none of the at least one controllable device included in the environmental image includes a first device to be screened that conforms to the target device type, determine the target device from at least one second device to be screened that conforms to the target device type included in the scene image.

[0092] As an example, the AI cloud platform server can determine the target device type based on the voice information. For example, the AI cloud platform server can use natural language processing technology to perform semantic recognition on the voice information and determine the target device type corresponding to the voice information based on the obtained nouns. As an example, if the noun included in the voice information is "lamp", the AI cloud platform server can determine the target device type as a lighting fixture according to the noun "lamp".

[0093] Specifically, performing scene recognition on the environmental image to determine at least one controllable device included in the environmental image includes:

[0094] Match at least one scene image that is in the same scene as the environmental image from multiple scene images;

[0095] Use the controllable devices included in the scene image corresponding to the environmental image as at least one controllable device included in the environmental image.

[0096] As an example, in the process of matching at least one scene image that is in the same scene as the environmental image, a method of comparing the feature vector of the environmental image with the feature vectors corresponding to each scene image can be adopted. The AI cloud platform server can use a scene feature extraction algorithm (such as a scene feature encoding model of deep learning) to extract the feature vectors of the environmental image and each scene image, and further compare the feature vector of the environmental image with the feature vectors of each scene image to calculate the similarity (such as cosine similarity, Euclidean distance). When the similarity reaches a preset similarity threshold, it can be determined that the corresponding scene image and the environmental image are in the same scene.

[0097] Furthermore, taking the controllable devices corresponding to the scene images that are in the same scene as the environmental image as at least one controllable device included in the environmental image, the AI cloud platform server can extract the information of the device type corresponding to each controllable device from the device features corresponding to at least one controllable device included in the environmental image, and match it with the target device type corresponding to the voice information, so as to screen out the controllable devices that meet the target device type as the first devices to be screened.

[0098] When the controllable devices corresponding to the environmental image do not include the first devices to be screened that meet the target device type, the AI cloud platform server can determine the target device from at least one second device to be screened that meets the target device type included in the scene image. This step can specifically include:

[0099] Generating a prompt message based on at least one second device to be screened that meets the target device type included in the scene image;

[0100] Receiving a selection instruction corresponding to the prompt message and obtaining the target device features corresponding to the selection instruction;

[0101] Comparing and matching the target device features with the device features corresponding to the second devices to be screened to determine the target device from the second devices to be screened.

[0102] The prompt message can be displayed, for example, through a wearable device or a terminal connected to the wearable device. Further, the user can issue a selection instruction based on the prompt message. The selection instruction can be issued by the user through the human-computer interaction interface of the terminal, or can also be the voice information issued by the user again. The wearable device or the AI cloud platform server can perform semantic recognition on the voice information issued by the user again to determine the corresponding target device features based on the result of the semantic recognition.

[0103] As an example, the voice message sent again by the user can be, for example, "the lamp above the sink" or "the spotlight on the bedside". The wearable device or the AI cloud platform server can perform semantic recognition on the voice message sent again by the user to extract the target device features included in the voice message sent again by the user. The AI cloud platform server can compare and match these target device features with the device features corresponding to the second device to be screened to determine the target device from the second device to be screened.

[0104] In the above device control method, it is possible to first determine whether the controllable devices included in the real-time captured environmental image conform to the target device type corresponding to the voice message. If not, the controllable devices that conform to the target device type are screened out from all the controllable devices included in the pre-stored scene image, so as to avoid miscontrolling irrelevant or non-conforming controllable devices to the user's intention and improve the accuracy of the operation. If there are no controllable devices that conform to the target device type in the current environment, it is possible to automatically switch to the pre-stored scene image to screen the controllable devices that conform to the target device type, so as to ensure that the operation will not be interrupted or unresponsive, thereby providing a more complete and intelligent device recognition service and making the device control more reliable and convenient.

[0105] It should be understood that although the steps in the flowcharts involved in the above embodiments are shown in sequence according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless there is a clear indication in this article, the execution of these steps has no strict order limit, and these steps can be executed in other orders. Moreover, at least a part of the steps in the flowcharts involved in the above embodiments may include multiple steps or multiple stages. These steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be executed alternately or alternately with at least a part of other steps or steps or stages in other steps.

[0106] Based on the same inventive concept, an embodiment of the present application also provides a device control device for implementing the above-mentioned device control method. The solution provided by the device control device to solve the problem is similar to the solution described in the above device control method. Therefore, the specific limitations in one or more of the following device embodiments can be referred to the limitations on the device control method in the above text, and will not be repeated here.

[0107] In one embodiment, as Figure 3 shown, a device control device 300 is provided, including:

[0108] A receiving module 302, configured to receive the environmental image and voice information sent by the wearable device; the environmental image is obtained by the wearable device in real time when receiving the voice information for indicating device control.

[0109] A determining module 304, configured to determine the target device corresponding to the voice information based on the environmental image and multiple pre-stored scene images.

[0110] A generating module 306, configured to generate a device control instruction based on the target device and the voice information and send it to the control server to implement the control of the target device through the control server.

[0111] In some optional embodiments, the receiving module 302 is further configured to:

[0112] Receive the target text corresponding to the environmental image and voice information sent by the wearable device; the target text is obtained by the wearable device through text recognition of the voice information.

[0113] The determining module 304 is further configured to:

[0114] Determine the target device corresponding to the target text based on the environmental image and the scene image.

[0115] The generating module 306 is further configured to:

[0116] Generate a device control instruction based on the target device and the target text.

[0117] In some optional embodiments, the determining module 304 is further configured to:

[0118] Receive a marking instruction for the scene image, and determine the controllable devices included in the scene image based on the marking instruction.

[0119] Obtain the device features corresponding to each controllable device, and store the device features in correspondence with the corresponding controllable devices.

[0120] In some optional embodiments, the determining module 304 is further configured to:

[0121] Determine the target device type corresponding to the voice information.

[0122] Perform scene recognition on the environmental image to determine at least one controllable device included in the environmental image.

[0123] When among at least one controllable device included in the environmental image, there is at least one first device to be screened that conforms to the target device type, determine the target device from the first devices to be screened; when the controllable devices corresponding to the environmental image do not include a first device to be screened that conforms to the target device type, determine the target device from at least one second device to be screened that conforms to the target device type included in the scene image.

[0124] In some alternative embodiments, the determination module 304 is further configured to:

[0125] Match at least one scene image that is in the same scene as the environmental image from multiple scene images;

[0126] Use the controllable devices included in the scene image corresponding to the environmental image as at least one controllable device included in the environmental image.

[0127] In some alternative embodiments, the determination module 304 is further configured to:

[0128] Generate a prompt message based on at least one second device to be screened that conforms to the target device type included in the scene image;

[0129] Receive a selection instruction corresponding to the prompt message, and obtain the target device feature corresponding to the selection instruction;

[0130] Compare and match the target device feature with the device features corresponding to the second devices to be screened, so as to determine the target device from the second devices to be screened.

[0131] Each module in the above device can be implemented in whole or in part by software, hardware, and their combination. The above modules can be embedded in the processor in the electronic device in hardware form or be independent of it, or be stored in the memory in the electronic device in software form, so that the processor can call and execute the operations corresponding to the above modules.

[0132] The device control device provided in this embodiment can execute the device control method provided in the above method embodiment, and its implementation principle and technical effects are similar, and will not be elaborated here in this embodiment.

[0133] Figure 4 This is a schematic structural diagram of the electronic device provided in this application. As Figure 4 shown, the electronic device 400 provided in this embodiment includes: at least one processor 401 and a memory 402. Optionally, the device 40 further includes a communication component 403. Among them, the processor 401, the memory 402, and the communication component 403 are connected through a bus 404.

[0134] In a specific implementation process, at least one processor 401 executes computer-executable instructions stored in a memory 402, so that at least one processor 401 executes the above-mentioned method.

[0135] For the specific implementation process of the processor 401, reference can be made to the above method embodiment, and its implementation principle and technical effect are similar, so they will not be elaborated here in this embodiment.

[0136] In the above embodiment, it should be understood that the processor may be a central processing unit (Central Processing Unit, CPU for short), or other general-purpose processors, digital signal processors (Digital Signal Processor, DSP for short), application-specific integrated circuits (Application Specific Integrated Circuit, ASIC for short), etc. The general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc. The steps of the method disclosed in combination with the invention can be directly embodied as being executed by a hardware processor, or executed by a combination of hardware and software modules in the processor.

[0137] The memory may include a high-speed memory (Random Access Memory, RAM), and may also include a non-volatile memory (Non-volatile Memory, NVM), such as at least one disk memory.

[0138] The bus may be an Industry Standard Architecture (ISA) bus, a Peripheral Component Interconnect (PCI) bus, an Extended Industry Standard Architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of convenience in representation, the buses in the drawings of this application are not limited to only one bus or one type of bus.

[0139] This application also provides a computer program product, including a computer program, and when the computer program is executed by a processor, the above-mentioned method is implemented.

[0140] This application also provides a computer-readable storage medium, in which computer-executable instructions are stored, and when the processor executes the computer-executable instructions, the above-mentioned method is implemented.

[0141] The above-readable storage medium can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk. The readable storage medium can be any available medium accessible by a general-purpose or special-purpose computer.

[0142] An exemplary readable storage medium is coupled to the processor, enabling the processor to read information from and write information to the readable storage medium. Of course, the readable storage medium can also be a component of the processor. The processor and the readable storage medium can be located in an application specific integrated circuit (ASIC). Of course, the processor and the readable storage medium can also exist as discrete components in a device.

[0143] The division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the couplings or direct couplings or communication connections shown or discussed among each other can be indirect couplings or communication connections through some interfaces, devices or units, and can be in electrical, mechanical or other forms.

[0144] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0145] In addition, the functional units in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.

[0146] If a function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or a part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing an electronic device (which may be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods in various embodiments of the present invention. The aforementioned storage medium includes: USB flash drives, mobile hard disks, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical discs, etc., all kinds of media that can store program codes.

[0147] Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by hardware related to program instructions. The aforementioned program can be stored in a computer-readable storage medium. When this program is executed, it executes the steps including the above method embodiments; and the aforementioned storage medium includes: ROMs, RAMs, magnetic disks, or optical discs, etc., all kinds of media that can store program codes.

[0148] Finally, it should be noted that: after considering the specification and practicing the invention disclosed herein, those skilled in the art will readily think of other implementation manners of the present invention. The present invention is intended to cover any variations, uses, or adaptive changes of the present invention. These variations, uses, or adaptive changes follow the general principles of the present invention and include the common general knowledge or conventional technical means in the technical field of the present invention that is not disclosed in the present invention. It is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present invention is only limited by the appended claims.

Claims

1. A device control method, characterized in that, An AI cloud platform server applied to communicate with a wearable device and a control server respectively. The method includes: Receiving an environmental image and voice information sent by the wearable device; the environmental image is obtained by the wearable device in real time when receiving voice information for instructing device control; Based on the environmental image and multiple pre-stored scene images, determining a target device corresponding to the voice information; Based on the target device and the voice information, generating a device control instruction and sending it to the control server to implement control of the target device through the control server.

2. The method according to claim 1, wherein The receiving the environmental image and voice information sent by the wearable device further includes: Receiving the environmental image sent by the wearable device and a target text corresponding to the voice information; the target text is obtained by the wearable device through text recognition of the voice information; The determining the target device corresponding to the voice information based on the environmental image and multiple pre-stored scene images includes: Based on the environmental image and the scene images, determining the target device corresponding to the target text; The generating a device control instruction based on the target device and the voice information includes: Based on the target device and the target text, generating the device control instruction.

3. The method according to claim 1 or 2, characterized in that, Before the determining the target device corresponding to the voice information based on the environmental image and multiple pre-stored scene images, it further includes: Receiving a marking instruction for the scene image, and determining controllable devices included in the scene image based on the marking instruction; Obtaining device features corresponding to each of the controllable devices, and storing the device features in correspondence with the corresponding controllable devices.

4. The method according to claim 3, characterized in that Before the determining the target device corresponding to the voice information based on the environmental image and multiple pre-stored scene images, it further includes: Determining a target device type corresponding to the voice information; The determining the target device corresponding to the voice information based on the environmental image and multiple pre-stored scene images includes: Performing scene recognition on the environmental image to determine at least one controllable device included in the environmental image; When at least one of the at least one controllable device included in the environmental image includes at least one first device to be screened that meets the target device type, determining the target device from the at least one first device to be screened; when none of the at least one controllable device included in the environmental image includes a first device to be screened that meets the target device type, determining the target device from at least one second device to be screened that meets the target device type included in the scene images.

5. The method according to claim 4, wherein The performing scene recognition on the environmental image to determine at least one controllable device included in the environmental image includes: Matching at least one scene image in the same scene as the environmental image from multiple scene images; Regarding the controllable devices included in the scene image corresponding to the environmental image as at least one controllable device included in the environmental image.

6. The method according to claim 4, wherein Determining the target device from at least one second device to be screened that conforms to the target device type included in the scene image includes: Generating a prompt message based on at least one second device to be screened that conforms to the target device type included in the scene image; Receiving a selection instruction corresponding to the prompt message and obtaining target device features corresponding to the selection instruction; Comparing and matching the target device features with the device features corresponding to the second device to be screened to determine the target device from the second device to be screened.

7. A device control apparatus, characterized in that, Including: A receiving module, configured to receive an environmental image and voice information sent by the wearable device; The environmental image is obtained by the wearable device in real time when receiving voice information for instructing device control; A determining module, configured to determine a target device corresponding to the voice information based on the environmental image and multiple pre-stored scene images; A generating module, configured to generate a device control instruction based on the target device and the voice information and send it to the control server to implement control of the target device through the control server.

8. An electronic device, characterized in that, Including: A memory and a processor; The memory stores computer execution instructions; The processor executes the computer execution instructions stored in the memory, so that the processor executes the method according to any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, Computer execution instructions are stored in the computer-readable storage medium, and when the computer execution instructions are executed by the processor, they are used to implement the method according to any one of claims 1-6.

10. A computer program product, characterized in that, Including a computer program, which when executed by the processor implements the method according to any one of claims 1-6.

Images