Method for controlling household equipment and electronic equipment

Abstract

The invention provides a method for controlling household equipment and electronic equipment, and the method comprises the steps: responding to a first operation of a user, and displaying a graph recognition interface; determining that the image recognition interface comprises a first home device, displaying N controls on the image recognition interface, the N controls being used for controlling the first home device, the first home device being a home device bound with the electronic device, or the first home device being a home device bound with a remote controller and the electronic device. In the embodiment of the invention, the electronic equipment can directly display the control for controlling the household equipment on the image recognition interface when recognizing that the household equipment is included in the image recognition interface, so that a user can control the household equipment on the image recognition interface, the operation steps of the user are reduced, and the user experience can be effectively improved.

Description

Technical Field

[0001] This application relates to the field of electronic devices, and more particularly, to a method for controlling home appliances and an electronic device. Background Art

[0002] With the progress of technology, there are more and more home appliances in users' homes. The electronic device binds and registers the home appliances to the server through a home application. After the electronic device binds the home appliance, the control corresponding to the home appliance appears on the interface of the home application. The electronic device can send an instruction to the server in response to the user's operation on the control, and the server can control the home appliance after receiving the instruction. However, the above method has the following deficiencies: (1) The operation steps are cumbersome, and the user needs to perform multiple steps to make the electronic device send an instruction to the server; (2) When there are many home appliances in the user's home, the user needs to spend a long time looking for the home appliance to be controlled in the home application. In summary, the above method has low control efficiency and cumbersome operation, which affects the user experience. Therefore, how to control home appliances has become a technical problem to be solved urgently. Summary of the Invention

[0003] This application provides a method for controlling home appliances and an electronic device. When it is recognized that the image recognition interface includes a home appliance, controls for controlling the home appliance can be directly displayed on the image recognition interface, enabling the user to control the home appliance on the image recognition interface, reducing the user's operation steps, and effectively improving the user experience.

[0004] In a first aspect, a method for controlling a home appliance is provided. The method is applied to an electronic device and includes: in response to a first operation of a user, displaying an image recognition interface; determining that the image recognition interface includes a first home appliance, and displaying N controls on the image recognition interface, where N≥1 and is an integer, and the N controls are used to control the first home appliance, where the first home appliance is a home appliance bound to the electronic device, or the first home appliance is a home appliance bound to the electronic device by a remote control.

[0005] In an embodiment of this application, when the electronic device recognizes that the image recognition interface includes a home appliance, controls for controlling the home appliance can be directly displayed on the image recognition interface, enabling the user to control the home appliance on the image recognition interface, reducing the user's operation steps, and effectively improving the user experience.

[0006] In combination with the first aspect, in certain implementations of the first aspect, the displaying of the image recognition interface in response to the user's first operation includes: in response to an operation to trigger the camera, starting the camera of the electronic device; obtaining an image of the home device through the camera, and displaying the image recognition interface including the image of the home device, so as to determine whether the image of the home device matches the stored image of the home device. In other implementations, an image of the home device may also be obtained through other types of sensors or components other than the camera; or the camera and other types of sensors may jointly obtain an image of the home device.

[0007] In combination with the first aspect, in certain implementations of the first aspect, determining that the image recognition interface includes a first home device includes: obtaining the image of the home device included in the image recognition interface; when the image of the home device in the image recognition interface matches the image of the first home device in the image library, determining that the image recognition interface includes the first home device, where the image library includes images of M home devices, and the M home devices are home devices bound to the electronic device and / or home devices whose remote controls are bound to the electronic device, the M home devices include the first home device, and M≥1 and is an integer.

[0008] In the embodiments of the present application, when the electronic device recognizes a home device, it can capture the features of the home device in the image recognition interface and compare them with the previously stored images of the home devices, which can effectively avoid misrecognition and help improve the user experience.

[0009] In combination with the first aspect, in certain implementations of the first aspect, the M home devices include a second home device, and the second home device is a home device of the same model as the first home device. Determining that the image recognition interface includes the first home device includes: displaying a first option and a second option in the image recognition interface, where the first option corresponds to the first home device and the second option corresponds to the second home device; in response to the user's operation of selecting the first option, determining that the image recognition interface includes the first home device.

[0010] In the embodiments of the present application, when multiple home devices of the same model are bound, the electronic device can display options for determining the home device in the image recognition interface, which can effectively avoid misrecognition and help improve the user experience.

[0011] In combination with the first aspect, in some implementations of the first aspect, the M home devices include a second home device, which is the same model of home device as the first home device. The scene where the user is located is the first sub-scene of the first scene. The first scene further includes a second sub-scene. The first home device is located in the first sub-scene, and the second home device is located in the second sub-scene. Determining that the image recognition interface includes the first home device includes: detecting the user's location; when it is detected that the user's location is in the first sub-scene, determining that the image recognition interface includes the first home device.

[0012] In the embodiments of the present application, when multiple home devices of the same model are bound, the electronic device can detect the user's location and display the control corresponding to the home device at the user's location in the image recognition interface, which can effectively avoid misrecognition and help improve the user experience.

[0013] In combination with the first aspect, in some implementations of the first aspect, the image library is stored on the server.

[0014] In combination with the first aspect, in some implementations of the first aspect, the type of the first home device is the first type, and the number of home devices of the first type bound to the electronic device is 1.

[0015] In combination with the first aspect, in some implementations of the first aspect, the electronic device is also bound with a second home device, which is the same type of home device as the first home device. Determining that the image recognition interface includes the first home device includes: displaying a first option and a second option in the image recognition interface, where the first option corresponds to the first home device and the second option corresponds to the second home device; in response to the user's operation of selecting the first option, determining that the image recognition interface includes the first home device.

[0016] In the embodiments of the present application, when multiple home devices of the same type are bound, the electronic device can also avoid image comparison and instead display options for determining the home device in the image recognition interface, which can effectively avoid misrecognition and help improve the user experience.

[0017] In combination with the first aspect, in some implementations of the first aspect, the electronic device is also bound with a second home device, which is the same type of home device as the first home device. The scene where the user is located is the first sub-scene of the first scene. The first scene further includes a second sub-scene. The first home device is located in the first sub-scene, and the second home device is located in the second sub-scene. Determining that the image recognition interface includes the first home device includes: detecting the user's location; when it is detected that the user's location is in the first sub-scene, determining that the image recognition interface includes the first home device.

[0018] In the embodiments of the present application, when multiple home devices of the same type are bound, the electronic device may not perform image comparison, but display options for determining the home device on the image recognition interface, which can effectively avoid misrecognition and help improve the user experience.

[0019] In combination with the first aspect, in some implementation manners of the first aspect, the N controls include a first control, and the first home device has been registered with the server. The method further includes: in response to an operation of the user on the first control, sending a first operation instruction to the server, so that the server controls the first home device according to the first operation instruction.

[0020] In combination with the first aspect, in some implementation manners of the first aspect, the N controls include a first control, and the first home device has been bound to the smart gateway. The method further includes: in response to an operation of the user on the first control, sending a second operation instruction to the smart gateway, so that the smart gateway controls the first home device according to the second operation instruction, where the first home device and the smart gateway are in the same local area network.

[0021] In combination with the first aspect, in some implementation manners of the first aspect, the N controls include a first control. The method further includes: in response to an operation of the user on the first control, sending a third operation instruction to the first home device, so that the first home device performs an operation associated with the third operation instruction according to the third operation instruction.

[0022] In a second aspect, an electronic device is provided. The electronic device includes one or more processors; one or more memories; the one or more memories store one or more computer programs, and the one or more computer programs include instructions. When the instructions are executed by the one or more processors, the above aspects or any possible implementation manner of the above aspects is executed.

[0023] In a third aspect, a computer-readable storage medium is provided. The computer-readable storage medium includes a computer program or instructions. When the computer program or instructions are run on a computer, the first aspect and any possible implementation method of the first aspect are executed.

[0024] In a fourth aspect, a computer program product is provided. The computer program product includes a computer program or instructions. When the computer program or instructions are run on a computer, the first aspect and any possible implementation method of the first aspect are executed.

[0025] In a fifth aspect, a computer program is provided. When it runs on a computer, the method in the first aspect and any of its possible implementation manners is executed.

[0026] Sixth aspect, an electronic device according to an embodiment of the present application, the electronic device includes modules / units that execute the methods of the above aspects or any possible design of the above aspects; these modules / units can be implemented by hardware or by hardware executing corresponding software.

[0027] For the beneficial effects of the second aspect to the sixth aspect, please refer to the beneficial effects of the first aspect and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0028] Figure 1 is a schematic structural diagram of an electronic device provided by an embodiment of the present application.

[0029] Figure 2 is a software structure block diagram of an electronic device provided by an embodiment of the present application.

[0030] Figure 3 is a set of GUIs provided by an embodiment of the present application.

[0031] Figure 4 is a set of GUIs provided by an embodiment of the present application.

[0032] Figure 5 is a set of GUIs provided by an embodiment of the present application.

[0033] Figure 6 is a set of GUIs provided by an embodiment of the present application.

[0034] Figure 7 is a set of GUIs provided by an embodiment of the present application.

[0035] Figure 8 is a set of GUIs provided by an embodiment of the present application.

[0036] Figure 9 is a schematic flowchart of a method for controlling a home appliance provided by an embodiment of the present application.

[0037] Figure 10 is a schematic flowchart of a method for controlling a home appliance provided by an embodiment of the present application.

[0038] Figure 11 is a schematic flowchart of a method for controlling a home appliance provided by an embodiment of the present application.

[0039] Figure 12 is a schematic diagram of the composition of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0040] Next, the technical solutions in the embodiments of the present application will be described with reference to the drawings.

[0041] The terms used in the following embodiments are for the purpose of describing specific embodiments only and are not intended to limit this application. As used in the specification and appended claims of this application, the singular forms "a", "an", "the", "above-mentioned", "said", and "this" are also intended to include forms such as "one or more", unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, "at least one" and "one or more" mean one, two, or more than two. The term "and / or" is used to describe the association relationship of associated objects and indicates that three relationships can exist; for example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0042] References in this specification to "one embodiment" or "some embodiments" or the like mean that a particular feature, structure, or characteristic described in connection with the embodiment is included in one or more embodiments of the application. Thus, the phrases "in one embodiment", "in some embodiments", "in other some embodiments", "in still other embodiments", etc., which appear in different places in this specification, are not necessarily all referring to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in another way. The terms "comprise", "include", "have" and their variants all mean "including but not limited to", unless otherwise specifically emphasized in another way.

[0043] Embodiments of an electronic device, a user interface for such an electronic device, and for using such an electronic device are described below. In some embodiments, the electronic device may be a portable electronic device that also includes other functions such as a personal digital assistant and / or music player functions, such as a mobile phone, a tablet computer, a wearable electronic device with wireless communication functions (such as a smart watch), etc. Exemplary embodiments of the portable electronic device include, but are not limited to, portable electronic devices equipped with or other operating systems. The above-mentioned portable electronic device may also be other portable electronic devices, such as a laptop computer, etc. It should also be understood that in other some embodiments, the above-mentioned electronic device may not be a portable electronic device but a desktop computer.

[0044] Exemplarily, Figure 1The structural schematic diagram of the electronic device 100 is shown. The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0045] It can be understood that the structure schematically shown in the embodiments of the present application does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than those shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0046] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processing unit (GPU), an image signal processor (ISP), a controller, a memory, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Among them, different processing units may be independent devices or integrated in one or more processors.

[0047] Among them, the controller may be the nerve center and command center of the electronic device 100. The controller may generate operation control signals according to the instruction operation code and timing signals to complete the control of fetching instructions and executing instructions.

[0048] A memory can also be provided in the processor 110 for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store the instructions or data that the processor 110 has just used or recycled. If the processor 110 needs to use the instruction or data again, it can directly call it from the said memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0049] In some embodiments, the processor 110 may include one or more interfaces. The interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0050] The wireless communication function of the electronic device 100 can be implemented by the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modulation and demodulation processor, and the baseband processor, etc.

[0051] The antenna 1 and the antenna 2 are used for transmitting and receiving electromagnetic wave signals. Each antenna in the electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be multiplexed to improve the utilization rate of the antennas. For example, the antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0052] The mobile communication module 150 may provide solutions for wireless communications such as 2G / 3G / 4G / 5G applied to the electronic device 100. The mobile communication module 150 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 150 may receive electromagnetic waves through the antenna 1, filter and amplify the received electromagnetic waves, and then transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 may also amplify the signal modulated by the modulation and demodulation processor and convert it into electromagnetic waves through the antenna 1 for radiation. In some embodiments, at least some functional modules of the mobile communication module 150 may be provided in the processor 110. In some embodiments, at least some functional modules of the mobile communication module 150 and at least some modules of the processor 110 may be provided in the same device.

[0053] The modulation and demodulation processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be transmitted into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. Subsequently, the demodulator transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After being processed by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs a sound signal through an audio device (not limited to the speaker 170A, receiver 170B, etc.), or displays an image or video through the display screen 194. In some embodiments, the modulation and demodulation processor may be an independent device. In other embodiments, the modulation and demodulation processor may be independent of the processor 110 and be provided in the same device as the mobile communication module 150 or other functional modules.

[0054] The wireless communication module 160 may provide solutions for wireless communications applied to the electronic device 100, including wireless local area networks (WLANs) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite systems (GNSSs), frequency modulation (FM), near field communication (NFC), infrared (IR), etc. The wireless communication module 160 may be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, performs frequency modulation and filtering processing on the electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 160 may also receive signals to be sent from the processor 110, perform frequency modulation and amplification on them, and convert them into electromagnetic waves through the antenna 2 for radiation.

[0055] In some embodiments, antenna 1 of electronic device 100 is coupled to mobile communication module 150, and antenna 2 is coupled to wireless communication module 160, such that electronic device 100 can communicate with a network and other devices via wireless communication technologies. The wireless communication technologies may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-CDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include global positioning system (GPS), global navigation satellite system (GLONASS), beidou navigation satellite system (BDS), quasi-zenith satellite system (QZSS), and / or satellite based augmentation systems (SBAS).

[0056] Electronic device 100 implements a display function through a GPU, display screen 194, and an application processor, etc. The GPU is a microprocessor for image processing, and is connected to display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or change display information.

[0057] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a MiniLED, a MicroLED, a Micro-OLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include one or N display screens 194, where N is a positive integer greater than 1.

[0058] The electronic device 100 can implement the shooting function through an ISP, a camera 193, a video codec, a GPU, a display screen 194, an application processor, etc.

[0059] The ISP is used to process the data fed back by the camera 193. For example, when taking a photo, the shutter is opened, and light passes through the lens and is transmitted to the camera sensor. The light signal is converted into an electrical signal, and the camera sensor transmits the electrical signal to the ISP for processing and converts it into an image visible to the naked eye. The ISP can also optimize the noise, brightness, and skin color of the image through algorithms. The ISP can also optimize parameters such as the exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 193.

[0060] The camera 193 is used to capture static images or videos. An object generates an optical image through the lens and projects it onto the sensor. The sensor can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The sensor converts the light signal into an electrical signal, and then transmits the electrical signal to the ISP to convert it into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in standard RGB, YUV, etc. formats. In some embodiments, the electronic device 100 may include one or N cameras 193, where N is a positive integer greater than 1.

[0061] The digital signal processor is used to process digital signals. In addition to being able to process digital image signals, it can also process other digital signals. For example, when the electronic device 100 selects a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy, etc.

[0062] The video codec is used to compress or decompress digital videos. The electronic device 100 can support one or more video codecs. In this way, the electronic device 100 can play or record videos in multiple encoding formats, such as: Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0063] The NPU is a neural-network (NN) computing processor. By drawing on the structure of the biological neural network, such as the transmission mode between human brain neurons, it can quickly process the input information and can also continuously self-learn. Through the NPU, applications such as intelligent cognition of the electronic device 100 can be realized, such as: image recognition, face recognition, speech recognition, text understanding, etc.

[0064] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to achieve the data storage function. For example, files such as music and videos are saved in the external memory card.

[0065] The internal memory 121 can be used to store computer-executable program code, and the executable program code includes instructions. The processor 110 executes various functional applications and data processing of the electronic device 100 by running the instructions stored in the internal memory 121. The internal memory 121 can include a program storage area and a data storage area. Among them, the program storage area can store the operating system, application programs required for at least one function (such as the sound playback function, the image playback function, etc.). The data storage area can store the data created during the use of the electronic device 100 (such as audio data, phone book, etc.). In addition, the internal memory 121 can include high-speed random access memory and can also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0066] The electronic device 100 can implement audio functions through the audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and the application processor, etc. For example, music playback, recording, etc.

[0067] The audio module 170 is used to convert digital audio information into an analog audio signal for output, and is also used to convert an analog audio input into a digital audio signal. The audio module 170 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 170 can be disposed in the processor 110, or some functional modules of the audio module 170 can be disposed in the processor 110.

[0068] The speaker 170A, also referred to as a "loudspeaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or a hands-free call through the speaker 170A.

[0069] The receiver 170B, also referred to as an "earpiece", is used to convert an audio electrical signal into a sound signal. When the electronic device 100 answers a call or a voice message, the voice can be listened to by placing the receiver 170B close to the human ear.

[0070] The microphone 170C, also referred to as a "microphone" or "transmitter", is used to convert a sound signal into an electrical signal. When making a call or sending a voice message, the user can speak by bringing the mouth close to the microphone 170C to input the sound signal into the microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In some other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also implement a noise reduction function. In some other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, and implement a directional recording function, etc.

[0071] The pressure sensor 180A is used to sense a pressure signal and can convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194. There are many types of pressure sensors 180A, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. The capacitive pressure sensor can include at least two parallel plates with conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure according to the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the intensity of the touch operation according to the pressure sensor 180A. The electronic device 100 can also calculate the position of the touch according to the detection signal of the pressure sensor 180A. In some embodiments, touch operations with the same touch position but different touch operation intensities can correspond to different operation instructions. For example, when a touch operation with a touch operation intensity greater than or equal to a first pressure threshold acts on the alarm application icon, the instruction to create a new alarm is executed.

[0072] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access to application locks, fingerprint photography, fingerprint answering of incoming calls, etc. For example, when the mobile phone detects a user's touch operation on the locked screen interface, the mobile phone can collect the user's fingerprint information through the fingerprint sensor 180H and match the collected fingerprint information with the fingerprint information preset in the mobile phone. If the match is successful, the mobile phone can enter the non-locked screen interface from the locked screen interface.

[0073] The touch sensor 180K, also known as the "touch panel". The touch sensor 180K can be disposed on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also known as a "touch screen". The touch sensor 180K is used to detect touch operations acting thereon or nearby. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In some other embodiments, the touch sensor 180K can also be disposed on the surface of the electronic device 100, at a different position from the display screen 194.

[0074] Figure 2 It is a software structure block diagram of the electronic device 100 according to an embodiment of the present application. The layered architecture divides the software into several layers, and each layer has a clear role and division of labor. Communication between layers is through software interfaces. In some embodiments, the Android system is divided into four layers, from top to bottom are the application layer, the application framework layer, the Android runtime, and the system library, and the kernel layer. The application layer can include a series of application packages.

[0075] As Figure 2 shown, the application layer can include a camera, settings, third-party applications, etc. Among them, the third-party applications can include a gallery, a calendar, a call, a map, a navigation, a WLAN, a Bluetooth, music, a video, a short message, etc.

[0076] The application framework layer provides application programming interfaces (APIs) and programming frameworks for the applications in the application layer. The application framework layer can include some predefined functions.

[0077] As Figure 2 shown, the application framework layer can include a window manager, a content provider, a view system, a telephone manager, a resource manager, a notification manager, etc.

[0078] The window manager is used to manage window programs. The window manager can obtain the display screen size, determine whether there is a status bar, lock the screen, capture the screen, etc. The content provider is used to store and obtain data, and enable these data to be accessible by application programs. The data may include videos, images, audio, incoming and outgoing calls, browsing history and bookmarks, phone books, etc.

[0079] The view system includes visual controls, such as controls for displaying text, controls for displaying pictures, etc., such as the indication information for prompting the virtual shutter key in the embodiments of the present application. The view system can be used to build application programs. The display interface can be composed of one or more views. For example, a display interface including a short message notification icon may include a view for displaying text and a view for displaying pictures.

[0080] The phone manager is used to provide the communication function of the electronic device 100. For example, the management of call states (including answering, hanging up, etc.).

[0081] The resource manager provides various resources for application programs, such as localized strings, icons, pictures, layout files, video files, and so on.

[0082] The notification manager enables application programs to display notification information in the status bar. It can be used to convey message types that need to be informed, and can automatically disappear after a short stay without user interaction. For example, the notification manager is used to inform that the download is completed, message reminders, etc. The notification manager can also be a notification that appears in the system top status bar in the form of a chart or a scroll bar text, such as the notification of a background running application program, and can also be a notification that appears on the screen in the form of a dialogue window. For example, prompting text information in the status bar, emitting a prompt tone, vibrating the electronic device, flashing the indicator light, etc.

[0083] Android runtime includes a core library and a virtual machine. Android runtime is responsible for the scheduling and management of the Android system.

[0084] The core library contains two parts: one part is the functional functions that need to be called by the Java language, and the other part is the core library of Android.

[0085] The application layer and the application framework layer run in the virtual machine. The virtual machine executes the Java files of the application layer and the application framework layer as binary files. The virtual machine is used to perform functions such as object life cycle management, stack management, thread management, security and exception management, and garbage collection.

[0086] The system library can include multiple functional modules. For example: the surface manager, media libraries, 3D graphics processing libraries (such as OpenGL ES), 2D graphics engines (such as SGL), etc.

[0087] The surface manager is used to manage the display subsystem and provides the fusion of 2D and 3D layers for multiple applications.

[0088] The media library supports the playback and recording of various common audio and video formats, as well as static image files, etc. The media library can support multiple audio and video coding formats, such as: MPEG4, H.264, MP3, AAC, AMR, JPG, PNG, etc.

[0089] The 3D graphics processing library is used to implement 3D graphics drawing, image rendering, synthesis, and layer processing, etc.

[0090] The 2D graphics engine is a drawing engine for 2D drawing.

[0091] In addition, the system library can also include a status monitoring service module, etc., such as a physical state recognition module for analyzing and recognizing user gestures; a sensor service module for monitoring sensor data uploaded by various sensors in the hardware layer to determine the physical state of the electronic device 100.

[0092] The kernel layer is the layer between the hardware and the software. The kernel layer at least includes a display driver, a camera driver, an audio driver, and a sensor driver.

[0093] The hardware layer can include various sensors, such as Figure 1 the various sensors introduced in

[0094] With the progress of technology, there are more and more home appliances in users' homes. The electronic device can bind home appliances in the home application and register the home appliances with the server. After the electronic device binds the home appliances, controls corresponding to the home appliances will appear on the interface of the home application. The electronic device can send an instruction to the server in response to the user's operation on the control, and the server can control the home appliances after receiving the instruction.

[0095] Figure 3 Shows a set of graphical user interfaces (GUI) for controlling home appliances.

[0096] Such as Figure 3As shown in (a) therein, the display interface 301 of the electronic device, and this interface 301 is the interface of the home application. The electronic device can display multiple controls on the interface 301, and these multiple controls include information of home devices. For example, the electronic device displays on the control 302 that the lights are all off. When the electronic device detects the operation that the user clicks the control 302, in response to this operation, the electronic device can display a GUI as shown in Figure 3 in (b) therein.

[0097] As Figure 3 shown in (b) therein, in response to the operation that the user clicks the control 302, the electronic device displays the interface 303. The electronic device can display controls corresponding to multiple lights on the interface 303. For example, the control 304 corresponds to the light #1. The electronic device can also display on the control 304 the control 305 for controlling the switch of the light #1. When the electronic device detects the operation that the user clicks the control 304, in response to this operation, it can display a GUI as shown in Figure 3 in (c) therein.

[0098] As Figure 3 shown in (c) therein, in response to the operation that the user clicks the control 304, the electronic device can display the interface 306, and this interface 306 is the interface corresponding to the light #1. The electronic device can display multiple controls for controlling the light #1 on the interface 306. For example, the control 307 for controlling the switch of the light #1. Another example is the control 308 for controlling the brightness of the light #1.

[0099] When the electronic device detects the operation that the user clicks the control 307, the electronic device can send the instruction #1 to the server, and this instruction #1 is used to instruct to turn on the light #1. After receiving this instruction #1, the server can turn on the light #1. There are the following two possible implementation methods for the server to turn on the light #1:

[0100] One possible implementation method: After receiving the instruction #1, the server can send the instruction #2 to the smart gateway, and this smart gateway is the smart gateway for controlling the light #1. After receiving this instruction #2, the smart gateway can send the instruction #3 to the light #1. After receiving this instruction #3, the light #1 can adjust its state to the on state. In this possible implementation method, the smart gateway and the light #1 can be in the same local area network.

[0101] One possible implementation method: After receiving the instruction #1, the server can directly send the instruction #2 to the light #1. After receiving this instruction #3, the light #1 can adjust its state to the on state. In this possible implementation method, the electronic device and the light #1 can correspond to the same account.

[0102] It should be noted that the home appliances displayed on the interface 302 of the electronic device can be the home appliances that have been bound to the electronic device. In the embodiments of the present application, the method for the electronic device to bind home appliances is not limited.

[0103] For example, the electronic device and the home appliance correspond to the same account.

[0104] For another example, the electronic device and the home appliance are in the same local area network.

[0105] For another example, the electronic device stores information of the home appliance (for example, the type of the home appliance, the model of the home appliance, the virtual remote control corresponding to the home appliance).

[0106] From the above description, it is not difficult to conclude that the above methods have the following deficiencies: (1) The operation steps are cumbersome, and the user needs to perform multiple steps to make the electronic device send instructions to the server to control the home appliance; (2) When there are many home appliances in the user's home, the user needs to spend a long time looking for the home appliance to be controlled in the home application. In summary, the above methods have low control efficiency and cumbersome operations, which affect the user experience. Therefore, how to control home appliances has become a technical problem to be solved urgently.

[0107] Based on this, the embodiments of the present application provide a method for controlling a home appliance, which can quickly identify the home appliance and display a control for controlling the home appliance, with simple operation and improved user experience. First, the following is introduced in combination with Figures 4 - 8 the GUI shown.

[0108] Figure 4 FIG. shows a set of GUIs provided by the embodiments of the present application.

[0109] As Figure 4 shown in (a) of, the electronic device displays an interface 401, and this interface 401 is the interface of the home application. The electronic device can display a control corresponding to the bound home appliance on the interface 401, such as a control 402 corresponding to an air conditioner. The electronic device can also display an add control 403 on the interface 401, and this add control is used to add a new home appliance. When the electronic device detects that the user clicks the add control 403, in response to this operation, it can display the GUI shown in Figure 4 (b) of.

[0110] As Figure 4As shown in (b) of [description], the electronic device displays an option box 404 in response to the user's operation of clicking the add control 403. The electronic device can display multiple controls in the option box 404, such as an add device control 405, a create scene control, a share device control, and a scan control. When the electronic device detects that the user clicks the add device control 405, in response to this operation, it can display a GUI as shown in (c)-(e) of Figure 4 [description].

[0111] As Figure 4 shown in (c)-(e) of [description], the electronic device can display an interface 406 in response to the user's operation of clicking the add device control 405. The electronic device can display the progress of connecting to the table lamp on the interface 406. After the electronic device successfully connects to the table lamp, it can jump from the interface 406 to the interface 401 and display the control 407 corresponding to the newly added table lamp on the interface 401.

[0112] In some embodiments, the electronic device can be directly bound to the table lamp.

[0113] In some embodiments, the electronic device can be bound to the table lamp through a smart gateway. In other words, the smart gateway forms a network with the table lamp, and the smart gateway sends the information of the table lamp to the electronic device, and then the electronic device can complete the binding with the table lamp.

[0114] In some embodiments, after the electronic device is bound to the table lamp through the smart gateway and the electronic device and the smart gateway are in the same local area network, the electronic device can control the table lamp through the smart gateway, that is, the electronic device sends an operation instruction to the smart gateway, and after the smart gateway receives the operation instruction, it can control the table lamp.

[0115] In some embodiments, the electronic device can also register the table lamp with the server. After the electronic device registers the table lamp with the server, the electronic device can control the table lamp through the server.

[0116] In some embodiments, the electronic device can also upload the image of the table lamp to the server. The electronic device can obtain the image of the table lamp through the following several possible implementation methods.

[0117] One possible implementation method is that the table lamp sends its image to the electronic device. The table lamp stores the image of the table lamp. During the connection process between the electronic device and the table lamp, the table lamp can send the image of the table lamp to the electronic device, etc. In this possible implementation method, the table lamp can directly send the image to the electronic device or forward the image through the smart gateway.

[0118] A possible implementation is that the electronic device stores an image of a table lamp. The application data of the home application can include images of various home devices. After the electronic device is bound to the table lamp, the model of the table lamp can be determined, and thus the image of the table lamp can be obtained according to the model of the table lamp and the application data of the home application.

[0119] A possible implementation is that the electronic device obtains an image of the table lamp through a camera. For example, as shown in (f) of Figure 4 , after the electronic device is bound to the table lamp, it may not first jump from interface 406 to interface 401, but instead display an option box 408 on interface 409. This option box 408 is used to prompt the user whether to select to take an image of the table lamp. When the electronic device detects that the user clicks the OK control 409, in response to this operation, the shooting function can be started, and after the user finishes shooting, the image of the table lamp can be obtained from the taken image and uploaded to the server.

[0120] In some embodiments, the electronic device may not upload the image of the table lamp to the server, but instead the server stores the image of the table lamp. In these embodiments, when the electronic device registers the table lamp to the server, it can send the model of the table lamp to the server, and then the server can determine the image of the table lamp according to the model of the table lamp.

[0121] In some embodiments, the electronic device can also upload the image of the table lamp to the smart gateway.

[0122] In some embodiments, the electronic device may not upload the image of the table lamp to the smart gateway, but instead the smart gateway stores the image of the table lamp. In these embodiments, when the smart gateway forms a network with the table lamp, it can obtain the model of the table lamp, and then the smart gateway can determine the image of the table lamp according to the model of the table lamp.

[0123] In some embodiments, the electronic device may not upload the image of the table lamp to the server and the smart gateway, but instead store it locally.

[0124] It should be noted that Figure 4 only the table lamp is taken as an example for introduction, but the embodiments of the present application are not limited thereto. In the embodiments of the present application, more home devices can also be bound, such as air conditioners, cameras, speakers, etc.

[0125] Figure 4 The binding of home devices by the electronic device has been introduced. In some other embodiments of the present application, the electronic device can also bind virtual remote controls of each home device, and then the electronic device can act as a remote control of the home device to control the home device. It can be understood that in these embodiments, the electronic device can control the home device without passing through the smart gateway or the server.

[0126] Figure 5Shows a set of GUIs provided by an embodiment of the present application.

[0127] As Figure 5 shown in (a) of [], the electronic device displays the interface 501, and this interface 501 is the interface of a home application. The electronic device can display an addition control 502 on the interface 501. In addition to being used to add new home devices, the addition control 502 can also add a virtual remote control. When the electronic device detects that the user clicks the addition control 502, in response to this operation, it can display a GUI as shown in Figure 5 (b) of [].

[0128] As Figure 5 shown in (b) of [], the electronic device, in response to the user's operation of clicking the addition control 502, displays an option box 503. The electronic device can display multiple controls in the option box 503, such as an add device control, a create scene control, a share device control, a scan control, and an add remote control control 504. When the electronic device detects that the user clicks the add remote control control 504, in response to this operation, it can display a GUI as shown in Figure 5 (c) of [].

[0129] As Figure 5 shown in (c) of [], the electronic device, in response to the user's operation of clicking the add remote control control 504, displays the interface 505. The electronic device can display multiple home device controls in the interface 505, such as a TV control, an air conditioner control 506, a projector control, etc. When the electronic device detects that the user clicks the air conditioner control 506, in response to this operation, it can display a GUI as shown in Figure 5 (d) of [].

[0130] As Figure 5 shown in (d) of [], the electronic device, in response to the user's operation of clicking the air conditioner control 506, can display the interface 507. The electronic device can display multiple controls in the interface 507, and these multiple controls correspond to different air conditioner brands. For example, the control 508 corresponds to the brand: AAA,. When the electronic device detects that the user clicks the control 508, in response to this operation, it can display a GUI as shown in Figure 5 (e) of [].

[0131] As Figure 5 shown in (e) of [], the electronic device, in response to the user's operation of clicking the control 508, can display the interface 509. The electronic device can display a prompt message in the interface 509, and this prompt message is used to prompt the user to click the switch control 510 to confirm whether the air conditioner can be turned on. When the electronic device detects that the user clicks the switch control 510, in response to this operation, it can display a GUI as shown in Figure 5 (f) of [].

[0132] AsFigure 5 As shown in (f) in Figure 5 the GUI shown in (g).

[0133] It can be understood that when the air conditioner is not turned on, the user can switch the model of the remote control.

[0134] As Figure 5 shown in (g), when the electronic device detects the operation of the user clicking on control 512, in response to this operation, a prompt message can be displayed on interface 509. This prompt message is used to prompt the user to click on switch control 510 to confirm whether the air conditioner is turned off. When the electronic device detects that the user clicks on switch control 510, in response to this operation, the GUI shown in (h) in Figure 5 can be displayed.

[0135] As Figure 5 shown in (h), when the electronic device detects the operation of the user clicking on switch control 510, option box 511 can be displayed again on interface 509. This option box 511 is used to prompt the user to detect whether the air conditioner is turned off. When the electronic device detects the operation of the user clicking on control 512, in response to this operation, the GUI shown in (i) in Figure 5 can be displayed.

[0136] It can be understood that when the air conditioner is not turned off, the user can switch the model of the remote control.

[0137] As Figure 5 shown in (i), when the electronic device detects the operation of the user clicking on control 512, interface 513 can be displayed. The electronic device can determine information such as the name of the remote control and the location of the air conditioner according to the user's operation on interface 513. When the electronic device detects that the user clicks on complete control 514, in response to this operation, the electronic device completes the binding of the remote control and displays the GUI shown in (j) in Figure 5 can be displayed.

[0138] As Figure 5 shown in (j), when the electronic device detects the operation of the user clicking on complete control 514, interface 515 can be displayed. This interface 515 is a control interface, and this interface 515 includes multiple controls that can be used to control the air conditioner.

[0139] In some embodiments, the electronic device acquires an image of the air conditioner corresponding to the remote control. The electronic device can acquire the image of the air conditioner through several possible implementation manners as follows.

[0140] A possible implementation is that the electronic device obtains an image of the air conditioner according to the model of the remote control.

[0141] For example, remote control model #1 corresponds to air conditioner model #1, and remote control model #2 corresponds to air conditioner model #2. When the electronic device determines that the remote control model is model #2, it can determine that the air conditioner model is model #2, and then can obtain an image of the air conditioner (for example, it can be obtained from the application data of the home application, or from the picture library on the Internet, or obtained in the shopping application, or obtained on the official website of the manufacturer that produces the air conditioner, etc.).

[0142] A possible implementation is that the electronic device obtains an image of the air conditioner through the camera. For example, as Figure 5 shown in (k), after the electronic device is bound to the remote control of the air conditioner, it may not jump from interface 514 to interface 515 first, but display an option box 516 on interface 514, and this option box 516 is used to prompt the user whether to select to take an image of the air conditioner. When the electronic device detects that the user clicks the OK control 517, in response to this operation, it can start the shooting function, and obtain an image of the air conditioner from the taken image after the user completes the shooting.

[0143] It should be noted that Figure 5 only the example of binding the remote control of the air conditioner is introduced, but the embodiments of the present application are not limited thereto.

[0144] It should also be noted that Figure 5 only the example of the electronic device binding the remote control of the home device through the home application is introduced, but the embodiments of the present application are not limited thereto. In some other embodiments of the present application, the electronic device can also bind the remote control of the home device through other applications (for example, the universal remote control application).

[0145] In Figure 4 and Figure 5 shown in the example, the electronic device can be bound to the home device or to the remote control of the home device. In the embodiments of the present application, when the electronic device is bound to the home device or to the remote control of the home device, it can identify the home device through image recognition and display the corresponding controls. The following will be introduced in combination with Figure 6 for introduction.

[0146] Figure 6 Shows a set of GUIs provided by the embodiments of the present application.

[0147] As Figure 6 shown in (a), the electronic device displays interface 601, and this interface 601 is the desktop. The electronic device can display icons of multiple applications on the desktop, such as the picture 602 of the camera application. When the electronic device detects the operation of the user clicking the icon 602, it can display asFigure 6 The GUI shown in (b) in

[0148] As Figure 6 shown in (b) in

[0149] As Figure 6 shown in (c) in

[0150] In some embodiments, the electronic device is bound to the remote control of the table lamp. When the electronic device detects an operation by the user on the control, the electronic device can directly control the table lamp by means such as infrared remote control or Bluetooth remote control.

[0151] In some embodiments, the electronic device is directly bound to the table lamp. When the electronic device and the table lamp are in the same local area network and the electronic device detects an operation by the user on the control, the electronic device can directly send a control instruction to the table lamp.

[0152] In some embodiments, the electronic device and the table lamp are bound through a smart gateway. When the electronic device, the smart gateway, and the table lamp are in the same local area network and the electronic device detects an operation by the user on the control, the electronic device can send a control instruction to the smart gateway, and then the smart gateway sends a control instruction to the table lamp.

[0153] In some embodiments, the electronic device is directly bound to the table lamp and the electronic device registers the table lamp with the server. When the electronic device detects an operation by the user on the control, the electronic device can send a control instruction to the server, and then the server sends a control instruction to the table lamp.

[0154] In some embodiments, the electronic device is bound to the table lamp through the smart gateway and the electronic device registers the table lamp with the server. When the electronic device detects a user's operation on the control, the electronic device can send a control instruction to the server, and the server then sends a control instruction to the smart gateway. Finally, the smart gateway sends a control instruction to the table lamp.

[0155] In Figure 6 In the GUI shown in (c) of Figure 6 , the control displayed by the electronic device obscures part of the table lamp. In some other embodiments of the present application, the electronic device may also not obscure the table lamp when displaying the control. For example, as

[0156] shown in (d) of

[0157] When the electronic device recognizes that the table lamp is included in the image recognition interface, it can display the control 606, and the control 606 is used to control the brightness of the table lamp. In the embodiments of the present application, when the electronic device recognizes that a home device is included in the image recognition interface, it can directly display the control for controlling the home device on the image recognition interface, so that the user can control the home device on the image recognition interface, reducing the user's operation steps and effectively improving the user experience.

[0158] In some embodiments, the electronic device stores the image corresponding to the table lamp. The electronic device can capture the feature data of the table lamp in the image recognition interface and compare it with the feature data of the stored image of the table lamp. When the matching degree reaches the threshold, it is determined that the table lamp in the image recognition interface 603 is the previously bound table lamp, and then the multiple controls are displayed.

[0159] In some embodiments, the smart gateway stores the image corresponding to the table lamp. The electronic device can capture the feature data of the table lamp in the image recognition interface and obtain the feature data of the image of the table lamp stored in the smart gateway for comparison. When the matching degree reaches the threshold, it is determined that the table lamp in the image recognition interface 603 is the previously bound table lamp, and the multiple controls are displayed.

[0160] In some embodiments, the server stores the image corresponding to the table lamp. The electronic device can upload the feature data of the table lamp in the image recognition interface to the server, and the server compares the feature data of the table lamp in the image recognition interface uploaded by the electronic device with the feature data of the stored image of the table lamp. When the matching degree reaches the threshold, it is determined that the table lamp in the image recognition interface 603 is the previously bound table lamp, and the electronic device is notified to display the multiple controls.

[0161] In some embodiments, the smart gateway stores an image corresponding to the table lamp. The electronic device may upload the feature data of the table lamp in the image recognition interface to the smart gateway. The smart gateway compares the feature data of the table lamp in the image recognition interface uploaded by the electronic device with the feature data of the stored image of the table lamp. When the matching degree reaches the threshold, it is determined that the table lamp in the image recognition interface 603 is the previously bound table lamp, and the electronic device is notified to display the multiple controls.

[0162] In the embodiments of the present application, when the electronic device recognizes a home device in the image, it can capture the features of the home device in the image recognition interface and compare them with the stored image of the home device, which can effectively avoid misrecognition and help improve the user experience.

[0163] It should be noted that Figure 6 only takes the example that the electronic device enters the recognition state through the camera application, but the embodiments of the present application are not limited thereto. For example, as shown in (b) of Figure 5 , the electronic device can also enter the recognition mode in response to the user's operation of clicking the scan button; or, in other embodiments, it can also enter the recognition mode through the smart home application, or can enter the image recognition mode through other independent application software entrances, etc.

[0164] Figure 6 takes the example that the electronic device displays the controls of the table lamp, but the embodiments of the present application are not limited thereto. In other embodiments of the present application, the electronic device can also recognize other home devices in the shooting interface and display the corresponding controls.

[0165] Figure 7 shows a set of GUIs provided by the embodiments of the present application.

[0166] As Figure 7 shown in (a) of

[0167] As Figure 7 shown in (b) of

[0168] The electronic device can also display an image corresponding to the song on the interface 701. For example, the song being played by the electronic device is Song #1, the previous song of Song #1 is Song #2, and the next song of Song #1 is Song #3. Image 705 is the image of Song #2, Image 706 is the image of Song #1, and Image 707 is the image of Song #3.

[0169] The electronic device can also switch the played song in response to the user's operation of clicking on the image. For example, when the electronic device detects the operation of the user clicking on Image 705, in response to this operation, it can switch Song #1 to Song #2, and Image 705 switches from the left position to the middle position.

[0170] In addition to responding to the user's operation of clicking on the image, the electronic device can also respond to the user's operation of swiping to select the image. For example, when the electronic device detects the operation of the user swiping to the right, in response to this operation, the electronic device switches Image 705 from the left position to the middle position, switches Image 706 from the middle position to the right position, and switches Song #1 to Song #2.

[0171] In the embodiments of the present application, when the electronic device recognizes a home device, it can capture the features of the home device in the recognition interface and compare them with the previously stored images of the home device, which can effectively avoid misrecognition and help improve the user experience.

[0172] It should be understood that Figure 7 In the illustrated example, the electronic device can also capture the feature data of the speaker in the recognition interface and compare it with the previously stored images of the speaker. When the matching degree is satisfied, the controls of the speaker can be displayed in the recognition interface. For specific descriptions, reference can be made to the above text. For the sake of brevity, it will not be elaborated here.

[0173] In some scenarios, the electronic device may be bound to multiple home devices of the same type or the same model. The electronic device can also determine the user's location, and thus more accurately determine the device that the user wants to control.

[0174] Figure 8 A group of GUIs provided by the embodiments of the present application is shown.

[0175] As Figure 8 shown in (a) of [], the electronic device displays the interface 801, which is a shooting interface, and the electronic device enters the recognition state, that is, the interface 801 can also be called the recognition interface.

[0176] Assume that Table Lamp #1 is installed in the living room of the user's home, Table Lamp #2 is installed in the master bedroom, and Table Lamp #3 is installed in the secondary bedroom, and the above 3 table lamps have been bound to the electronic device. As Figure 8As shown in (b) and (c) thereof, after the electronic device enters the image recognition state, it can obtain the image of the image recognition interface 801 and, after recognizing the table lamp included according to the image of the image recognition interface 801, can display an option box 802, which is used to prompt the user to select the location where they are. Among them, the control 803 corresponds to the living room, the control 804 corresponds to the master bedroom, and the control 805 corresponds to the secondary bedroom. When the electronic device detects the operation of the user clicking the control 804, in response to this operation, it can display a plurality of controls, and the plurality of controls are used to control the table lamp #1.

[0177] In some other embodiments, the above three table lamps are table lamps of the same model, or at least two of the above three table lamps are of different models, but the matching degree of the characteristic data of the images of the above three table lamps is greater than or equal to the threshold. When the electronic device enters the image recognition state, it can obtain the image of the image recognition interface 801 and, after recognizing the table lamp included according to the image of the image recognition interface 801, can capture the characteristic data of the table lamp in the image recognition interface 801 and compare it with the characteristic data of the images of the three stored table lamps. Since the three stored table lamps are table lamps of the same model, or although at least two of them are of different models, the matching degree of the characteristic data of the images of the above three table lamps is greater than or equal to the threshold, the electronic device cannot determine which specific table lamp it is through image comparison. Then the electronic device can display an option box 802, which is used to prompt the user to select the location where they are. Among them, the control 803 corresponds to the living room, the control 804 corresponds to the master bedroom, and the control 805 corresponds to the secondary bedroom. When the electronic device detects the operation of the user clicking the control 804, in response to this operation, it can display a plurality of controls, and the plurality of controls are used to control the table lamp #1.

[0178] It can be understood that when the electronic device determines that the table lamp in the image recognition interface is the table lamp in the master bedroom and detects the operation of the user on the control, when sending an operation instruction to the server or the smart gateway, it can indicate that the operation instruction is for the table lamp in the master bedroom. Then the server or the smart gateway can control the table lamp in the master bedroom.

[0179] In some embodiments, when there are home devices such as cameras and infrared sensors for monitoring deployed in the scene where the user is located, the electronic device can also obtain the data generated by the cameras, infrared sensors, etc., and locate the specific location of the user in the scene according to the above data.

[0180] It can be understood that when there are multiple users in the scene where the user is located, the electronic device can analyze the action behaviors of each user according to the above data, so as to determine which specific user is using the electronic device to aim at the home device.

[0181] For example, User #1 is in the master bedroom, User #2 is in the second bedroom, and User #3 is in the living room. Table lamps are installed at all of the above three locations. Among them, User #1 uses an electronic device to aim at the table lamp, User #2 is on the phone, and User #3 is watching TV. The electronic device of User #1 can obtain the action behaviors of the above three users through the camera, and then determine that it is User #1 who needs to control the table lamp and is in the master bedroom. Then, a control widget for the table lamp in the master bedroom can be displayed on the image recognition interface.

[0182] The method for controlling home appliances provided in the embodiments of the present application has been introduced above in combination with the GUI. The method for controlling home appliances provided in the embodiments of the present application will be introduced below in combination with the schematic diagram of the method.

[0183] Figure 9 The schematic flowchart of the method for controlling home appliances provided in the embodiments of the present application is shown. As Figure 9 shown, the method includes:

[0184] S901, the electronic device binds to home appliance #1 in response to Operation #1 of the user.

[0185] For example, as Figure 4 shown, the electronic device binds to the table lamp in response to the user's operation of adding a home appliance.

[0186] In some embodiments, the electronic device can directly bind to home appliance #1.

[0187] In some embodiments, the electronic device can bind to home appliance #1 through a smart gateway. It can be understood that when the electronic device binds to home appliance #1 through the smart gateway, the smart gateway can store the information of home appliance #1.

[0188] S902, the electronic device registers home appliance #1 with the server.

[0189] The electronic device can send the information of home appliance #1 to the server to register home appliance #1 with the server. The information of home appliance #1 includes but is not limited to: the model of home appliance #1, the location of home appliance #1, etc. The electronic device and home appliance #1 can correspond to the same account. In other words, when the electronic device logs in to Account #1, the electronic device can register home appliance #1 under the home appliances of Account #1. When the electronic device binds to home appliance #1 through the smart gateway, the smart gateway and the electronic device can correspond to the same account, that is, the electronic device, home appliance #1, and the smart gateway correspond to the same account.

[0190] In some embodiments, the electronic device can store an image of home appliance #1.

[0191] In some embodiments, the electronic device may also upload an image of Home Device #1 to the server.

[0192] In some embodiments, the electronic device may also upload an image of Home Device #1 to the smart gateway.

[0193] In some embodiments, the electronic device may not upload the image of Home Device #1 to the server, but the server stores the image of Home Device #1. In these embodiments, when the electronic device registers Home Device #1 with the server, it may send the model of Home Device #1 to the server, and then the server can determine the image of Home Device #1 according to the model of Home Device #1.

[0194] In some embodiments, the electronic device may not upload the image of Home Device #1 to the smart gateway, but the smart gateway stores the image of Home Device #1. In these embodiments, when the smart gateway forms a network with the table lamp, it can obtain the model of Home Device #1, and then the smart gateway can determine the image of the table lamp according to the model of Home Device #1.

[0195] S903. The electronic device displays an image recognition interface in response to Operation #2 of the user.

[0196] For example, as shown in (a) and (b) of Figure 6 , the electronic device displays Interface 603 in response to the user's operation of clicking on the icon 602 of the camera application, and then enters the recognition state in response to the user's operation of clicking on the image recognition control 604. At this time, Interface 603 can be referred to as an image recognition interface.

[0197] For another example, as shown in (a) and (b) of Figure 4 , the electronic device displays Option Box 404 in response to the user's operation of clicking on the Add Control 403, and then can display an image recognition interface in response to the user's operation of clicking on the Scan Control.

[0198] For another example, the electronic device may display an image recognition interface in response to a preset gesture of the user. The preset gesture can be set by the user or can also be set when the electronic device leaves the factory. The preset gesture can be a gesture that touches the display screen or an air gesture. The embodiments of the present application do not limit this.

[0199] S904. The electronic device determines that the image recognition interface includes Home Device #1 and displays N controls in the image recognition interface.

[0200] After the electronic device displays the image recognition interface, it can obtain the image of the image recognition interface, and then can identify the type of home device #1 in the image recognition interface through image recognition technology. When the type of the identified home device #1 is the type of a home device that has been bound to the electronic device, N controls for controlling the home device #1 can be displayed on the image recognition interface, where N≥1 and is an integer.

[0201] For example, if the electronic device identifies that the type of home device #1 is a table lamp and the electronic device is only bound to one table lamp, the electronic device can display N controls corresponding to the bound table lamp.

[0202] As Figure 6 shown in (c) and (d) of , the electronic device can display the controls for controlling the table lamp on the interface 603.

[0203] In some embodiments, after the electronic device identifies the type of the home device in the image recognition interface and the electronic device is bound to multiple home devices of the same type, the electronic device can first determine the user's location, and then determine the N controls to be displayed according to the user's location. For specific descriptions, reference can be made to the description for Figure 8 this.

[0204] In the embodiments of the present application, when the electronic device recognizes that the image recognition interface includes a home device, it can directly display the controls for controlling the home device on the image recognition interface, enabling the user to control the home device on the image recognition interface, reducing the user's operation steps, and effectively improving the user experience.

[0205] In some embodiments, the electronic device can capture the feature data of the home device #1 in the image recognition interface and compare it with the feature data of the images in the stored image library of home devices. When a match is determined, it indicates that the home device #1 in the image recognition interface is a home device bound to the electronic device, and then the N controls can be displayed. The image library of home devices can include M images of home devices, where M≥1 and is an integer. The image library of home devices can be stored in the electronic device, smart gateway or server, and the home devices corresponding to the image library of home devices are home devices that have been bound to the electronic device. In these embodiments, the following two possible implementation methods can be included.

[0206] One possible implementation method is that the electronic device is bound to a home device of the same type as the home device #1.

[0207] For example, the electronic device identifies that the type of home device #1 is a table lamp. The server stores an image of a table lamp (i.e., the image of the home device #1). The electronic device can communicate with the server to compare the feature data of the table lamp in the image recognition interface with the feature data of the table lamp stored in the server. When the matching degree is greater than or equal to the threshold, the electronic device can display the N controls.

[0208] A possible implementation is that the electronic device is bound to multiple home devices of the same type as Home Device #1.

[0209] For example, the electronic device identifies that the type of Home Device #1 is a table lamp, and the model of the table lamp in the recognition interface is Model #1. The server stores images of 2 table lamps, with models Model #1 and Model #2 respectively, and the matching degree of the feature data of the images of the table lamps of Model #1 and Model #2 is less than the threshold. The electronic device can communicate with the server to compare the feature data of the table lamp in the recognition interface with the feature data of the table lamps stored in the server, and then can determine that the model of the table lamp in the recognition interface is Model #1. The electronic device can display N controls corresponding to Model #1.

[0210] For another example, the electronic device identifies that the type of Home Device #1 is a table lamp, and the model of the table lamp in the recognition interface is Model #1. The server stores images of 2 table lamps, both with the model of Model #1. In other words, the electronic device is bound to two table lamps of the same model. The electronic device can communicate with the server to compare the feature data of the table lamp in the recognition interface with the feature data of the table lamps stored in the server, and then can determine that the model of the table lamp in the recognition interface is Model #1. In this case, the electronic device can also determine the user's location and display the corresponding N controls according to the user's location. For specific description, please refer to the description for Figure 8 of.

[0211] For another example, the electronic device identifies that the type of Home Device #1 is a table lamp, and the model of the table lamp in the recognition interface is Model #1. The server stores images of 2 table lamps, with models Model #1 and Model #2 respectively, and the matching degree of the feature data of the images of the table lamps of Model #1 and Model #2 is greater than or equal to the threshold. In this case, since the feature data of the images of the two models of table lamps are similar, the electronic device cannot determine which specific table lamp through image comparison. Then the electronic device can also determine the user's location and display the corresponding N controls according to the user's location. For specific description, please refer to the description for Figure 8 of.

[0212] In the embodiments of the present application, when the electronic device recognizes a home device, it can capture the features of the home device in the recognition interface and compare them with the images of the home devices stored previously, which can effectively avoid misrecognition and help improve the user experience.

[0213] In some embodiments, the method further includes:

[0214] S905, the electronic device sends Operation Instruction #1 to the server.

[0215] Correspondingly, the server receives the Operation Instruction #1 sent by the electronic device.

[0216] In response to a user's operation on control #1, the electronic device may send operation instruction #1 to the server, where control #1 is one of the N controls.

[0217] For example, as Figure 6 shown in (c) of [], in response to a user's operation of clicking on control 605, the electronic device may send an operation instruction to the server, and the operation instruction is used to indicate turning on the table lamp.

[0218] S906. The server sends operation instruction #2 to home device #1.

[0219] Correspondingly, home device #1 receives operation instruction #2 sent by the server.

[0220] In some embodiments, operation instruction #2 may be generated by the server according to operation instruction #1.

[0221] In some embodiments, operation instruction #2 is the same as operation instruction #1. In other words, after receiving operation instruction #1, the server forwards operation instruction #1 to home device #1. In this case, the operation instruction #1 forwarded by the server may also be referred to as operation instruction #2.

[0222] In some embodiments, the server may directly send operation instruction #2 to home device #1.

[0223] In some embodiments, the server may indirectly send operation instruction #2 to home device #1, that is, the server first sends operation instruction #2 to the smart gateway associated with home device #1, and the smart gateway then sends operation instruction #2 to home device #1.

[0224] S907. Home device #1 executes operation instruction #2.

[0225] After receiving operation instruction #2, the home device may execute the operation corresponding to operation instruction #2.

[0226] For example, as Figure 6 shown in (c) of [], in response to a user's operation of clicking on control 605, the electronic device may send an operation instruction to the server, and the server then sends an operation instruction to the table lamp, and the table lamp adjusts its state to the on state according to the operation instruction.

[0227] In the embodiments of the present application, the user can control the home device by operating the control on the recognition interface, without having to find the home device to be controlled in the home application, saving the user's operation steps and improving the user experience.

[0228] Figure 10 shows a schematic flowchart of the method for controlling a home device provided by the embodiments of the present application, asFigure 10 As shown, the method includes:

[0229] S1001, the electronic device responds to user operation #1 and binds to home device #1.

[0230] In some embodiments, the electronic device can directly bind to home device #1.

[0231] In some embodiments, the electronic device can bind to home device #1 through a smart gateway. It can be understood that when the electronic device binds to home device #1 through the smart gateway, the smart gateway can store the information of home device #1. In other words, when the electronic device binds to home device #1 through the smart gateway, home device #1 is registered to the smart gateway.

[0232] In some embodiments, the electronic device can store an image of home device #1.

[0233] In some embodiments, the electronic device can also upload an image of home device #1 to the smart gateway.

[0234] In some embodiments, the electronic device may not upload the image of home device #1 to the smart gateway, but the smart gateway stores the image of home device #1. In these embodiments, when the smart gateway forms a network with the table lamp, it can obtain the model of home device #1, and then the smart gateway can determine the image of the table lamp according to the model of home device #1.

[0235] S1002, the electronic device responds to user operation #2 and displays an image recognition interface.

[0236] S1003, the electronic device determines that the image recognition interface includes home device #1 and displays N controls on the image recognition interface.

[0237] It should be understood that the descriptions of S1002 and S1003 can be referred to the above, and for the sake of brevity, they will not be elaborated here.

[0238] In the embodiments of the present application, when the electronic device recognizes that the image recognition interface includes a home device, it can directly display controls for controlling the home device on the image recognition interface, so that the user can control the home device on the image recognition interface, reducing the user's operation steps and effectively improving the user experience.

[0239] In some embodiments, the electronic device may capture the feature data of Home Device #1 in the image recognition interface and compare it with the feature data of the images in the stored image library of home devices. When a match is determined, it indicates that Home Device #1 in the image recognition interface is the home device bound to the electronic device, and then the N controls are displayed. The image library of home devices may include images of M home devices, where M ≥ 1 and is an integer. The image library of home devices may be stored in the electronic device, the smart gateway, or the server, and the home devices corresponding to the image library of home devices are the home devices that have been bound to the electronic device.

[0240] In the embodiments of the present application, when the electronic device recognizes a home device, it can capture the features of the home device in the image recognition interface and compare them with the previously stored images of home devices, which can effectively avoid misrecognition and help improve the user experience.

[0241] In some embodiments, the method further includes:

[0242] S1004, the electronic device sends Operation Instruction #1 to Home Device #1.

[0243] Correspondingly, Home Device #1 receives the Operation Instruction #1 sent by the electronic device.

[0244] The electronic device can be directly bound to Home Device #1. In response to the user's operation on Control #1, the electronic device can send Operation Instruction #1 to Home Device #1, and this Control #1 is one of the N controls.

[0245] For example, as shown in (c) of Figure 6 , in response to the user's operation of clicking Control 605, the electronic device can send an operation instruction to Home Device #1, and this operation instruction is used to indicate turning on the table lamp.

[0246] S1005, Home Device #1 executes Operation Instruction #1.

[0247] In the embodiments of the present application, the user can control the home device by operating the controls in the image recognition interface, without having to find the home device to be controlled in the home application, saving the user's operation steps and improving the user's usage experience.

[0248] In some other embodiments, the method further includes:

[0249] S1006, the electronic device sends Operation Instruction #1 to the smart gateway.

[0250] Correspondingly, the smart gateway receives the Operation Instruction #1 sent by the electronic device.

[0251] The electronic device is bound to the home device #1 through the smart gateway. In response to the user's operation on the control #1, the electronic device can send an operation instruction #1 to the smart gateway, and the control #1 is one of the N controls.

[0252] S1007, the smart gateway sends the operation instruction #2 to the home device #1.

[0253] Correspondingly, the home device #1 receives the operation instruction #2 sent by the smart gateway.

[0254] In some embodiments, the operation instruction #2 can be generated by the smart gateway according to the operation instruction #1.

[0255] In some embodiments, the operation instruction #2 is the same as the operation instruction #1. In other words, after receiving the operation instruction #1, the smart gateway forwards the operation instruction #1 to the home device #1. In this case, the operation instruction #1 forwarded by the smart gateway can also be referred to as the operation instruction #2.

[0256] S1008, the home device #1 executes the operation instruction #2.

[0257] In the embodiments of the present application, the user can control the home device by operating the control on the recognition interface, without searching for the home device to be controlled in the home application, saving the user's operation steps and improving the user experience.

[0258] Figure 11 The schematic flowchart of the method for controlling a home device provided by the embodiments of the present application is shown, as Figure 11 shown, the method includes:

[0259] S1101, in response to the user's operation #1, the electronic device is bound to the remote control of the home device #1.

[0260] For example, as Figure 5 shown, the electronic device can be bound to the remote control of the air conditioner.

[0261] In some embodiments, the electronic device can store the image of the home device #1.

[0262] In some embodiments, the electronic device can also upload the image of the home device #1 to the server.

[0263] In some embodiments, the electronic device may not upload the image of the home device #1 to the server, but the server stores the image of the home device #1. In these embodiments, when registering the home device #1 to the server, the electronic device can send the model of the home device #1 to the server, and then the server can determine the image of the home device #1 according to the model of the home device #1.

[0264] S1102, the electronic device responds to the user's operation #2 and displays an image recognition interface.

[0265] S1103, the electronic device determines that the image recognition interface includes home device #1 and displays N controls on the image recognition interface.

[0266] It should be understood that the descriptions for S1102 and S1103 can be referred to the above. For the sake of brevity, they will not be elaborated here.

[0267] In the embodiments of the present application, when the electronic device recognizes that the image recognition interface includes a home device, it can directly display the controls for controlling the home device on the image recognition interface, enabling the user to control the home device on the image recognition interface, reducing the user's operation steps, and effectively improving the user experience.

[0268] In some embodiments, the electronic device can capture the feature data of home device #1 in the image recognition interface and compare it with the feature data of the images in the stored image library of home devices. When a match is determined, it indicates that home device #1 in the image recognition interface is the home device corresponding to the remote control bound to the electronic device, and then the N controls are displayed. The image library of home devices can include images of M home devices, where M≥1 and is an integer. The image library of home devices can be stored in the electronic device or the server, and the home devices corresponding to the image library of home devices are the home devices that have been bound to the electronic device, or the home devices whose remote controls are bound to the electronic device.

[0269] In the embodiments of the present application, when the electronic device recognizes a home device, it can capture the features of the home device in the image recognition interface and compare them with the previously stored images of home devices, which can effectively avoid misrecognition and help improve the user experience.

[0270] In some embodiments, the method further includes:

[0271] S1104, the electronic device sends operation instruction #1 to home device #1.

[0272] Correspondingly, home device #1 receives the operation instruction #1 sent by the electronic device.

[0273] In response to the user's operation on control #1, since the electronic device is bound to the remote control of home device #1, it can send operation instruction #1 to home device #1, and this control #1 is one of the N controls. The electronic device can send operation instruction #1 through infrared, Bluetooth, etc. In other words, the electronic device can send operation instruction #1 to home device #1 in the same way as the remote control of home device #1.

[0274] S1105, home device #1 executes operation instruction #1.

[0275] In the embodiments of the present application, the user can control home appliances by operating the controls on the image recognition interface, without having to search for the home appliance to be controlled in the home application, saving the user's operation steps and improving the user experience.

[0276] The above mainly introduces a method for controlling home appliances provided by the embodiments of the present application from the perspective of an electronic device. It can be understood that in order for the electronic device to implement the above functions, it includes the corresponding hardware structure and / or software module for executing each function. Those skilled in the art should easily realize that, in combination with the algorithm steps of each example described in the embodiments disclosed herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a certain function is executed in the way of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the present application.

[0277] The embodiments of the present application can divide the functional modules (or units) of the processor in the electronic device according to the above method examples. For example, each functional module (or unit) can be divided corresponding to each function, or two or more functions can be integrated into one processing module (or unit). The above integrated module (or unit) can be implemented in the form of hardware or in the form of a software functional module (or unit). It should be noted that the division of modules (or units) in the embodiments of the present application is illustrative, only a logical function division, and there can be other division methods in actual implementation.

[0278] In the case of dividing each functional module (or unit) corresponding to each function, Figure 12 FIG. shows a schematic composition diagram of an electronic device 1200 provided by the embodiments of the present application. As Figure 12 shown, the electronic device 1200 includes: a processing module 1210 and a transceiver module 1220.

[0279] Among them, the processing module 1210 is used to display an image recognition interface in response to a first operation of the user.

[0280] The processing module 1210 is further used to determine that the image recognition interface includes a first home appliance, and display N controls on the image recognition interface, where N≥1 and is an integer, and the N controls are used to control the first home appliance, where the first home appliance is a home appliance bound to the electronic device, or the first home appliance is a home appliance bound to the electronic device by a remote control.

[0281] In some embodiments, the processing module 1210 is specifically configured to obtain an image of a home device on the image recognition interface; when the image of the home device on the image recognition interface matches the image of the first home device in the image library, it is determined that the image recognition interface includes the first home device, where the image library includes images of M home devices, and the M home devices are home devices bound to the electronic device and / or home devices whose remote controls are bound to the electronic device. The M home devices include the first home device, and M≥1 and is an integer.

[0282] In some embodiments, the M home devices include a second home device, and the second home device is the same model as the first home device. The processing module 1210 is specifically configured to display a first option and a second option on the image recognition interface, where the first option corresponds to the first home device and the second option corresponds to the second home device; in response to an operation in which the user selects the first option, it is determined that the image recognition interface includes the first home device.

[0283] In some embodiments, the M home devices include a second home device, and the second home device is the same model as the first home device. The scene where the user is located is the first sub-scene of the first scene, and the first scene further includes a second sub-scene. The first home device is located in the first sub-scene, and the second home device is located in the second sub-scene. The processing module 1210 is specifically configured to detect the position of the user; when it is detected that the position of the user is located in the first sub-scene, it is determined that the image recognition interface includes the first home device.

[0284] In some embodiments, the image library is stored in the server.

[0285] In some embodiments, the type of the first home device is the first type, and the number of home devices of the first type bound to the electronic device is 1.

[0286] In some embodiments, the electronic device is further bound to a second home device, and the second home device is the same type as the first home device. The processing module 1210 is specifically configured to display a first option and a second option on the image recognition interface, where the first option corresponds to the first home device and the second option corresponds to the second home device; in response to an operation in which the user selects the first option, it is determined that the image recognition interface includes the first home device.

[0287] In some embodiments, the electronic device is further bound to a second home device, and the second home device is the same type as the first home device. The scene where the user is located is the first sub-scene of the first scene, and the first scene further includes a second sub-scene. The first home device is located in the first sub-scene, and the second home device is located in the second sub-scene. The processing module 1210 is specifically configured to detect the position of the user; when it is detected that the position of the user is located in the first sub-scene, it is determined that the image recognition interface includes the first home device.

[0288] In some embodiments, the N controls include a first control. The first home device is registered with the server. The transceiver module 1220 is configured to send a first operation instruction to the server in response to a user's operation on the first control, so that the server controls the first home device according to the first operation instruction.

[0289] In some embodiments, the N controls include a first control. The first home device is bound to the smart gateway. The transceiver module 1220 is configured to send a second operation instruction to the smart gateway in response to a user's operation on the first control, so that the smart gateway controls the first home device according to the second operation instruction, where the first home device and the smart gateway are in the same local area network.

[0290] In some embodiments, the N controls include a first control. The transceiver module 1220 is configured to send a third operation instruction to the first home device in response to a user's operation on the first control, so that the first home device performs an operation associated with the third operation instruction according to the third operation instruction.

[0291] The embodiments of the present application provide a computer program product. When the computer program product runs on an electronic device, the electronic device is enabled to execute the technical solutions in the above embodiments. The implementation principle and technical effects are similar to those of the related embodiments of the above method, and will not be elaborated here.

[0292] The embodiments of the present application provide a readable storage medium. The readable storage medium contains instructions. When the instructions run on an electronic device, the electronic device is enabled to execute the technical solutions of the above embodiments. The implementation principle and technical effects are similar, and will not be elaborated here.

[0293] The embodiments of the present application provide a chip. The chip is used to execute instructions. When the chip runs, it executes the technical solutions in the above embodiments. The implementation principle and technical effects are similar, and will not be elaborated here.

[0294] Those of ordinary skill in the art can realize that the units and algorithm steps of each example described in combination with the embodiments disclosed herein can be implemented by electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are executed in hardware or software depends on the specific application and design constraints of the technical solution. Professional technicians can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application.

[0295] Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices, and units described above can refer to the corresponding processes in the foregoing method embodiments, and will not be elaborated here.

[0296] In several embodiments provided by the present application, it should be understood that the disclosed systems, devices, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed couplings or direct couplings or communication connections to each other can be through some interfaces. The indirect couplings or communication connections of the devices or units can be in electrical, mechanical, or other forms.

[0297] The units described as separate components may or may not be physically separated. The components shown as units may or may not be physical units, that is, they can be located in one place or distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0298] In addition, in each embodiment of the present application, the functional units can be integrated into one processing unit, or each unit can exist physically alone, or two or more units can be integrated into one unit.

[0299] If the function is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiments of the present application, in essence, or the part that contributes to the prior art or part of this technical solution can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the embodiments of the present application. The foregoing storage medium includes: various media such as USB flash drives, mobile hard disks, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical discs that can store program codes.

[0300] As described above, it is only the specific implementation manner of the embodiments of the present application, but the protection scope of the embodiments of the present application is not limited thereto. Any person skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the embodiments of the present application and should be covered by the protection scope of the embodiments of the present application. Therefore, the protection scope of the embodiments of the present application should be subject to the protection scope of the claims.

Claims

1. A method for controlling a home device, the method being applied to an electronic device, the method comprising: Responding to a first operation of a user, displaying an image recognition interface; Determining that the image recognition interface includes a first home device, and displaying N controls on the image recognition interface, where N≥1 and is an integer, and the N controls are used to control the first home device, wherein the first home device is a home device bound to the electronic device, or the first home device is a home device bound to the electronic device by a remote control.

2. The method according to claim 1, wherein The determining that the image recognition interface includes a first home device includes: Obtaining an image of the home device on the image recognition interface; When the image of the home device on the image recognition interface matches the image of the first home device in the image library, determining that the image recognition interface includes the first home device, where the image library includes images of M home devices, and the M home devices are home devices bound to the electronic device and / or home devices bound to the electronic device by a remote control, and the M home devices include the first home device, and M≥1 and is an integer.

3. The method according to claim 2, wherein The M home devices include a second home device, and the second home device is a home device of the same model as the first home device. The determining that the image recognition interface includes the first home device includes: Displaying a first option and a second option on the image recognition interface, where the first option corresponds to the first home device and the second option corresponds to the second home device; Responding to an operation in which the user selects the first option, determining that the image recognition interface includes the first home device.

4. The method according to claim 2, wherein The M home devices include a second home device, and the second home device is a home device of the same model as the first home device. The scene where the user is located is the first sub-scene of the first scene, and the first scene further includes a second sub-scene. The first home device is located in the first sub-scene, and the second home device is located in the second sub-scene. The determining that the image recognition interface includes the first home device includes: Detecting the position of the user; When it is detected that the position of the user is located in the first sub-scene, determining that the image recognition interface includes the first home device.

5. The method according to any one of claims 2 to 5, characterized in that The image library is stored in a server.

6. The method according to claim 1, characterized in that, The type of the first home device is the first type, and the number of home devices of the first type bound to the electronic device is 1.

7. The method according to claim 1, characterized in that, The electronic device is further bound to a second home device, and the second home device is a home device of the same type as the first home device. The determining that the image recognition interface includes the first home device includes: Displaying a first option and a second option on the image recognition interface, where the first option corresponds to the first home device and the second option corresponds to the second home device; Responding to an operation in which the user selects the first option, determining that the image recognition interface includes the first home device.

8. The method according to claim 1, wherein The electronic device is also bound to a second home device, which is a home device of the same type as the first home device. The scenario where the user is located is the first sub-scenario of the first scenario. The first scenario further includes a second sub-scenario. The first home device is located in the first sub-scenario, and the second home device is located in the second sub-scenario. Determining that the image recognition interface includes the first home device includes: Detecting the user's location; When it is detected that the user's location is in the first sub-scenario, determining that the image recognition interface includes the first home device.

9. The method according to any one of claims 1 to 8, characterized in that, The N controls include a first control. The first home device has been registered with the server. The method further includes: In response to the user's operation on the first control, sending a first operation instruction to the server so that the server controls the first home device according to the first operation instruction.

10. The method according to any one of claims 1 to 8, characterized in that, The N controls include a first control. The first home device has been bound to the smart gateway. The method further includes: In response to the user's operation on the first control, sending a second operation instruction to the smart gateway so that the smart gateway controls the first home device according to the second operation instruction, where the first home device and the smart gateway are in the same local area network.

11. The method according to any one of claims 1 to 8, characterized in that, The N controls include a first control. The method further includes: In response to the user's operation on the first control, sending a third operation instruction to the first home device so that the first home device performs an operation associated with the third operation instruction according to the third operation instruction.

12. The method according to any one of claims 1 to 11, characterized in that The displaying the image recognition interface in response to the user's first operation includes: In response to an operation to trigger the camera, starting the camera of the electronic device; Obtaining an image of the home device through the camera and displaying the image recognition interface containing the image of the home device.

13. An electronic device, characterized in that, Comprising one or more processors; one or more memories; the one or more memories store one or more computer programs, the one or more computer programs include instructions, when the instructions are executed by the one or more processors, causing the method according to any one of claims 1 to 12 to be executed.

14. A chip, characterized in that, The chip includes a processor and a communication interface. The communication interface is used to receive a signal and transmit the signal to the processor. The processor processes the signal, causing the method according to any one of claims 1 to 12 to be executed.

15. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions, when the computer instructions run on a computer, causing the method according to any one of claims 1 to 12 to be executed.

Images