Device control method, electronic device and storage medium
By capturing images of smart home appliances and combining them with signal strength to identify device types, the problem of existing technologies being unable to distinguish between multiple devices of the same type has been solved, achieving efficient device control and a user-friendly operating experience.
Patent Information
- Application Number
- CN202110861878.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-07-29
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2041-07-29
AI Technical Summary
Existing smart home appliance control applications can only identify the type of device, but cannot distinguish between multiple devices of the same type, resulting in reduced control efficiency and a poor user experience.
By capturing images of smart home appliances and combining them with signal strength to identify the device type, the system identifies the device to be controlled from the candidate devices and displays its control interface.
It improves the recognition and control efficiency of smart home appliances, simplifies user operation, and enhances the user experience.
Smart Images

Figure CN115701032B_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of computer technology, and in particular to a device control method, an electronic device, and a storage medium. Background Art
[0002] With the popularity of smart homes, more and more smart home appliances (for example, smart speakers, smart lighting, smart sockets, smart TVs, smart refrigerators, etc.) have entered households. However, the control of the above-mentioned smart home appliances requires corresponding applications. Usually, users can control smart home appliances through the above-mentioned related applications. In actual applications, the above-mentioned related applications need to identify the smart home appliances to be controlled before controlling the smart home appliances, so that the identified smart home appliances can be controlled. However, the current related applications can only identify the types of smart home appliances. When the user has multiple smart home appliances of the same type at home, the control efficiency of the above-mentioned smart home appliances will be reduced, which brings a bad experience to the user. Summary of the invention
[0003] The embodiments of the present application provide a device control method, an electronic device and a storage medium to provide a method for identifying and controlling smart home appliances based on signal strength, thereby improving the recognition efficiency, and further improving the efficiency of controlling smart home appliances, and improving the user experience.
[0004] In a first aspect, an embodiment of the present application provides a device control method, which is applied to a first device, including:
[0005] Capture an image of the second device; wherein the first device may be a terminal device such as a mobile phone or a tablet, and the second device may be a smart home appliance such as a smart air conditioner, a smart TV, a smart refrigerator, a smart lighting, a smart switch, etc. The first device may obtain an image of the second device by capturing it with a camera.
[0006] The device type of the second device is obtained, where the device type of the second device is determined according to the image of the second device; wherein the device type can be used to distinguish the type of the second device, for example, a television type, a refrigerator type, an air conditioner type, a lighting type, a switch type, etc.
[0007] Match the device type of the second device with the first device list to determine multiple target devices of the same device type as the second device, wherein the first device list includes multiple devices to be selected, each of which has a preset device type; wherein the devices to be selected may be smart home appliances to be selected, and the smart home appliances to be selected may form a local area network, and the smart home appliances connected to the local area network constitute a device list.
[0008] The signal strength between the first device and each target device is obtained, and the signal strength is used to determine the second device from the target device; and the control interface of the second device is displayed. The signal strength between the first device and each target device may be the received signal strength of Bluetooth, and the control interface is used to control the second device.
[0009] In an embodiment of the present application, candidate target smart home appliances are obtained based on the device type, and the smart home appliance to be controlled is determined among the candidate target smart home appliances by signal strength, and the control interface of the smart home appliance to be controlled is displayed, thereby improving the recognition efficiency and further improving the efficiency of controlling the smart home appliances.
[0010] In order to improve the efficiency of image recognition, in one possible implementation, obtaining the device type of the second device includes:
[0011] Perform image recognition on the image of the second device, and determine the device type of the second device according to the recognition result.
[0012] In one possible implementation, obtaining the device type of the second device includes:
[0013] The image of the second device is sent to the third device; the image of the second device is used by the third device to determine the device type of the second device; and the device type of the second device sent by the third device is received.
[0014] In the embodiment of the present application, by sending the image to a third device (cloud server) for recognition and processing, the computing burden of the first device can be reduced, and the accuracy of image recognition can be improved through the powerful computing power of the third device.
[0015] In order to facilitate the user's shooting operation and improve the user experience, in one possible implementation manner, the first device is installed with a first application; before shooting the image of the second device, the method further includes:
[0016] The first application starts a device-controlled shooting mode in response to a detected first operation of the user;
[0017] Capturing an image of the second device includes:
[0018] The first application captures an image of the second device in response to a detected second operation of the user in the device control shooting mode.
[0019] In one possible implementation, displaying the control interface of the second device includes:
[0020] Displaying a second device list based on the signal strength; wherein the second device list includes part or all of the multiple target devices;
[0021] In response to a detected operation of a user selecting a second device in the second device list, a control interface of the second device is displayed.
[0022] In an embodiment of the present application, candidate smart home appliances are screened by signal strength, and a user selects a second device from the smart home appliances obtained after the screening to control the second device, thereby improving the accuracy of controlling the second device.
[0023] In one possible implementation, displaying the control interface of the second device includes:
[0024] The target device corresponding to the highest signal strength is determined as the second device, and the control interface of the second device is displayed.
[0025] In an embodiment of the present application, by automatically determining the second device from the candidate smart home appliances and displaying the control interface of the second device, the efficiency of identification can be improved and the user's operation can be simplified.
[0026] In order to improve the efficiency of selecting the second device in the device list, in one possible implementation, displaying the second device list based on the signal strength includes:
[0027] Acquire a first target device corresponding to the highest signal strength and a second target device corresponding to the second highest signal strength;
[0028] Based on the order of signal strength from strong to weak, the first target device and the second target device are displayed in sequence.
[0029] In one possible implementation, sequentially displaying the first target device and the second target device based on the order of signal strength from strong to weak includes:
[0030] If the difference between the signal strength of the first target device and the signal strength of the second target device is greater than or equal to the preset signal strength threshold, the first target device and the second target device are displayed sequentially based on the order of signal strength from strong to weak.
[0031] In an embodiment of the present application, when the signal strength difference between the target devices is large, determining the candidate smart home appliance devices based on the signal strength can improve the accuracy of identifying the second device.
[0032] In one possible implementation, displaying the second device list based on the signal strength includes:
[0033] Acquire a first target device corresponding to the highest signal strength and a second target device corresponding to the second highest signal strength;
[0034] If the difference between the signal strength of the first target device and the signal strength of the second target device is less than a preset signal strength threshold, the first target device and the second target device are displayed according to the user's historical usage habits, wherein the user's historical usage habits are used to characterize statistical information of the user's use of the first target device and the second target device.
[0035] In the embodiment of the present application,
[0036] In one possible implementation, when the signal strength difference between the target devices is small, the candidate smart home appliance devices are determined according to the user's historical usage habits, which can improve the accuracy of identifying the second device.
[0037] In one possible implementation, displaying the first target device and the second target device according to the user's historical usage habits includes:
[0038] The usage probabilities corresponding to the first target device and the second target device are determined according to the user's historical usage habits; and the first target device and the second target device are displayed in sequence based on the order of usage probabilities from high to low.
[0039] In one possible implementation manner, the first device is a mobile terminal.
[0040] In one possible implementation, the second device is a smart home appliance.
[0041] In a second aspect, an embodiment of the present application provides a device control apparatus, applied to a first device, including:
[0042] A shooting module, used for shooting an image of the second device;
[0043] an acquisition module, used to acquire a device type of the second device, where the device type of the second device is determined according to an image of the second device;
[0044] A matching module, used for matching the device type of the second device with the first device list, and determining a plurality of target devices of the same device type as the second device, wherein the first device list includes a plurality of to-be-selected devices, each of which has a preset device type;
[0045] The display module is used to obtain the signal strength between the first device and each target device, the signal strength is used to determine the second device from the target devices; and display the control interface of the second device.
[0046] In one possible implementation manner, the acquisition module is further used to perform image recognition on the image of the second device, and determine the device type of the second device according to the recognition result.
[0047] In one possible implementation, the acquisition module is further used to send the image of the second device to a third device, and the image of the second device is used by the third device to determine the device type of the second device; and receive the device type of the second device sent by the third device.
[0048] In one possible implementation manner, the first device is installed with a first application, and the apparatus further includes:
[0049] An enabling module, configured for enabling a device to control a shooting mode in response to a detected first operation of a user by a first application;
[0050] The shooting module is also used for the first application to capture an image of the second device in response to a detected second operation of the user in the device control shooting mode.
[0051] In one possible implementation, the display module is further configured to display a second device list based on the signal strength; wherein the second device list includes part or all of the multiple target devices;
[0052] In response to a detected operation of a user selecting a second device in the second device list, a control interface of the second device is displayed.
[0053] In one possible implementation manner, the display module is further configured to determine a target device corresponding to the highest signal strength as the second device, and display a control interface of the second device.
[0054] In one possible implementation, the display module is further used to obtain a first target device corresponding to the highest signal strength and a second target device corresponding to the second highest signal strength;
[0055] Based on the order of signal strength from strong to weak, the first target device and the second target device are displayed in sequence.
[0056] In one possible implementation, the display module is also used to display the first target device and the second target device sequentially based on the order of signal strength from strong to weak if the difference between the signal strength of the first target device and the signal strength of the second target device is greater than or equal to a preset signal strength threshold.
[0057] In one possible implementation, the display module is further used to obtain a first target device corresponding to the highest signal strength and a second target device corresponding to the second highest signal strength;
[0058] If the difference between the signal strength of the first target device and the signal strength of the second target device is less than a preset signal strength threshold, the first target device and the second target device are displayed according to the user's historical usage habits, wherein the user's historical usage habits are used to characterize statistical information of the user's use of the first target device and the second target device.
[0059] In one possible implementation, the display module is further used to determine the usage probabilities corresponding to the first target device and the second target device respectively according to the user's historical usage habits;
[0060] The first target device and the second target device are displayed in sequence based on the order of usage probability from high to low.
[0061] In one possible implementation manner, the first device is a mobile terminal.
[0062] In one possible implementation, the second device is a smart home appliance.
[0063] In a third aspect, an embodiment of the present application provides a first device, including:
[0064] A memory, the memory is used to store computer program codes, the computer program codes include instructions, when the first device reads the instructions from the memory, the first device performs the following steps:
[0065] capturing an image of the second device;
[0066] Acquire a device type of the second device, where the device type of the second device is determined according to the image of the second device;
[0067] Matching the device type of the second device with the first device list to determine a plurality of target devices of the same device type as the second device, wherein the first device list includes a plurality of to-be-selected devices, each of which has a preset device type;
[0068] Acquire a signal strength between the first device and each target device, the signal strength being used to determine the second device from the target devices;
[0069] The control interface of the second device is displayed.
[0070] In one possible implementation manner, when the instruction is executed by the first device, causing the first device to execute a step of obtaining a device type of the second device includes:
[0071] Perform image recognition on the image of the second device, and determine the device type of the second device according to the recognition result.
[0072] In one possible implementation manner, when the instruction is executed by the first device, causing the first device to execute a step of obtaining a device type of the second device includes:
[0073] Sending the image of the second device to the third device, where the image of the second device is used by the third device to determine the device type of the second device;
[0074] The device type of the second device sent by the third device is received.
[0075] In one possible implementation, the first device is installed with a first application; when the instruction is executed by the first device, before the first device executes the step of capturing an image of the second device, the first device further includes the following steps:
[0076] The first application starts a device-controlled shooting mode in response to a detected first operation of the user;
[0077] The step of causing the first device to execute capturing an image of the second device includes:
[0078] The first application captures an image of the second device in response to a detected second operation of the user in the device control shooting mode.
[0079] In one possible implementation manner, when the instruction is executed by the first device, the step of causing the first device to display the control interface of the second device includes:
[0080] Displaying a second device list based on the signal strength; wherein the second device list includes part or all of the multiple target devices;
[0081] In response to a detected operation of a user selecting a second device in the second device list, a control interface of the second device is displayed.
[0082] In one possible implementation manner, when the instruction is executed by the first device, the step of causing the first device to display the control interface of the second device includes:
[0083] The target device corresponding to the highest signal strength is determined as the second device, and the control interface of the second device is displayed.
[0084] In one possible implementation manner, when the instruction is executed by the first device, the step of causing the first device to display a list of second devices based on signal strength includes:
[0085] Acquire a first target device corresponding to the highest signal strength and a second target device corresponding to the second highest signal strength;
[0086] Based on the order of signal strength from strong to weak, the first target device and the second target device are displayed in sequence.
[0087] In one possible implementation, when the instruction is executed by the first device, the first device performs the step of sequentially displaying the first target device and the second target device in descending order of signal strength, including:
[0088] If the difference between the signal strength of the first target device and the signal strength of the second target device is greater than or equal to the preset signal strength threshold, the first target device and the second target device are displayed sequentially based on the order of signal strength from strong to weak.
[0089] In one possible implementation manner, when the instruction is executed by the first device, the step of causing the first device to display a list of second devices based on signal strength includes:
[0090] Acquire a first target device corresponding to the highest signal strength and a second target device corresponding to the second highest signal strength;
[0091] If the difference between the signal strength of the first target device and the signal strength of the second target device is less than a preset signal strength threshold, the first target device and the second target device are displayed according to the user's historical usage habits, wherein the user's historical usage habits are used to characterize statistical information of the user's use of the first target device and the second target device.
[0092] In one possible implementation, when the instruction is executed by the first device, the first device is caused to perform the step of displaying the first target device and the second target device according to the user's historical usage habits, including:
[0093] Determine usage probabilities corresponding to the first target device and the second target device respectively according to the user's historical usage habits;
[0094] The first target device and the second target device are displayed in sequence based on the order of usage probability from high to low.
[0095] In one possible implementation manner, the first device is a mobile terminal.
[0096] In one possible implementation, the second device is a smart home appliance.
[0097] In a fourth aspect, an embodiment of the present application provides a device control system, comprising the first device and the third device as described in the third aspect, wherein:
[0098] The third device is used to receive the image of the second device sent by the first device, perform image recognition on the image, and send the recognition result to the first device, where the recognition result is the device type of the second device.
[0099] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium, in which a computer program is stored. When the computer-readable storage medium is run on a computer, the computer executes the method described in the first aspect.
[0100] In a sixth aspect, an embodiment of the present application provides a computer program, which, when executed by a computer, is used to execute the method described in the first aspect.
[0101] In one possible design, the program in the sixth aspect may be stored in whole or in part on a storage medium packaged together with the processor, or may be stored in whole or in part on a memory not packaged together with the processor. BRIEF DESCRIPTION OF THE DRAWINGS
[0102] Figure 1 A schematic diagram of an application scenario provided for an embodiment of the present application;
[0103] Figure 2 A schematic diagram of the software architecture of an electronic device provided in an embodiment of the present application;
[0104] Figure 3 A schematic diagram of the hardware structure of an electronic device provided in an embodiment of the present application;
[0105] Figure 4 A schematic diagram of a flow chart of an embodiment of a device control method provided by the present application;
[0106] Figure 5a and Figure 5b A schematic diagram of the device control shooting mode setting provided in an embodiment of the present application;
[0107] Figure 6a-6c A schematic diagram showing a control interface provided in an embodiment of the present application;
[0108] Figure 7 A schematic diagram of the structure of an embodiment of the device control device provided in this application. DETAILED DESCRIPTION
[0109] The technical solution in the embodiment of the present application will be described below in conjunction with the drawings in the embodiment of the present application. In the description of the embodiment of the present application, unless otherwise specified, " / " means or, for example, A / B can mean A or B; "and / or" in this article is only a description of the association relationship of associated objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone.
[0110] In the following, the terms "first" and "second" are used for descriptive purposes only and are not to be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the features. In the description of the embodiments of the present application, unless otherwise specified, "plurality" means two or more.
[0111] With the popularity of smart homes, more and more smart home appliances (for example, smart speakers, smart light bulbs, smart sockets, smart TVs, smart refrigerators, etc.) have entered households. However, the control of these smart home appliances requires corresponding applications. Usually, users can control smart home appliances through the above-mentioned related applications. In actual applications, before controlling the smart home appliances, the above-mentioned related applications need to identify the smart home appliances to be controlled, so that the identified smart home appliances can be controlled. However, the current related applications can only identify the types of smart home appliances. When the user has multiple smart home appliances of the same type at home, the control efficiency of the above-mentioned smart home appliances will be reduced, which brings a bad experience to the user.
[0112] In order to solve the above problems, the embodiment of the present application proposes a device control method, which is applied to the first device 10. Among them, the first device 10 can be a mobile terminal with a camera and a display screen. The first device 10 can also be called a terminal device, a user equipment (User Equipment, UE), an access terminal, a user unit, a user station, a mobile station, a mobile station, a remote station, a remote terminal, a mobile device, a user terminal, a terminal, a wireless communication device, a user agent or a user device. The first device 10 can be a cellular phone, a cordless phone, a personal digital assistant (Personal Digital Assistant, PDA) device, a handheld device with a wireless communication function, a computing device or other processing device connected to a wireless modem, a computer, a laptop computer, a handheld communication device, a handheld computing device, a satellite wireless device, a customer premises equipment (Customer Premise Equipment, CPE) and / or other devices for communicating on a wireless system and a next-generation communication system, for example, a mobile terminal in a 5G network or a mobile terminal in a future evolved public land mobile network (Public Land Mobile Network, PLMN) network. The embodiment of the present application does not specifically limit the form of the above-mentioned first device 10.
[0113] Figure 1 For application scenarios of the above device control method, such as Figure 1As shown, the above application scenario includes a first device 10, multiple second devices 20 and a third device 30. Among them, the second device 20 can be a smart home appliance, such as a smart speaker, smart lighting, smart socket, smart TV, smart refrigerator, etc. The embodiment of the present application does not specifically limit the form of the above second device 20. The third device 30 can be a cloud device (for example, a cloud server).
[0114] The first device 10 and the second device 20 can be connected wirelessly, wherein the wireless connection method can be WIFI or Bluetooth, or other wireless communication methods. The embodiment of the present application does not specifically limit the wireless connection method between the first device 10 and the second device 20. The first device 10 and the third device 30 can be connected via WIFI. For example, the first device 10 can be connected to a WIFI router and establish a connection with the third device 30 through the WIFI router. Optionally, the first device 10 can also communicate with a base station through a mobile communication network (for example, 4G, 5G, etc.), and establish a connection with the third device 30 through the base station. The embodiment of the present application does not specifically limit the wireless connection method between the first device 10 and the third device 30.
[0115] Figure 2 FIG. 1 is a schematic diagram of the software architecture of the first device 10. Figure 2 As shown, the first device 10 includes a hardware layer 11, a driver layer 12, a hardware abstraction layer 13 and an application layer 14.
[0116] The hardware layer 11 may be used to drive a device (eg, a camera) in the first device 10 to capture an image. For example, when the first device 10 is in the smart shooting mode, the hardware layer 11 may drive the camera to capture an image.
[0117] The driver layer 12 can be used to obtain the image captured by the hardware layer 11 and upload the image to the hardware abstraction layer 13 .
[0118] The hardware abstraction layer 13 may be used to receive an image uploaded by the driver layer 12 , and may upload the image to the application layer 14 .
[0119] The application layer 14 can be used to identify the image uploaded by the hardware abstraction layer 13, and can display the control interface of the second device 20 based on the identification result, or send the image uploaded by the hardware abstraction layer 13 to the third device 30, which identifies the above image and sends the identification result to the first device 10. When the first device 10 receives the identification result sent by the third device 30, it can display the control interface of the second device 20 based on the identification result, so as to realize control of the second device 20.
[0120] Combine the following Figure 3First, an exemplary electronic device provided in the following embodiments of the present application is introduced. Figure 3 A schematic structural diagram of an electronic device 100 is shown, and the electronic device 100 may be the first device 10 mentioned above.
[0121] The electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, an earphone interface 170D, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, an air pressure sensor 180C, a magnetic sensor 180D, an acceleration sensor 180E, a distance sensor 180F, a proximity light sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.
[0122] It is to be understood that the structure illustrated in the embodiment of the present invention does not constitute a specific limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or separate some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.
[0123] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.
[0124] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.
[0125] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.
[0126] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.
[0127] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 110 may include multiple groups of I2C buses. The processor 110 may be coupled to the touch sensor 180K, the charger, the flash, the camera 193, etc. through different I2C bus interfaces. For example: the processor 110 may be coupled to the touch sensor 180K through the I2C interface, so that the processor 110 communicates with the touch sensor 180K through the I2C bus interface, thereby realizing the touch function of the electronic device 100.
[0128] The I2S interface can be used for audio communication. In some embodiments, the processor 110 can include multiple I2S buses. The processor 110 can be coupled to the audio module 170 via the I2S bus to achieve communication between the processor 110 and the audio module 170. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 via the I2S interface to achieve the function of answering a call through a Bluetooth headset.
[0129] The PCM interface can also be used for audio communication, sampling, quantizing and encoding analog signals. In some embodiments, the audio module 170 and the wireless communication module 160 can be coupled via a PCM bus interface. In some embodiments, the audio module 170 can also transmit audio signals to the wireless communication module 160 via the PCM interface to realize the function of answering calls via a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.
[0130] The UART interface is a universal serial data bus for asynchronous communication. The bus can be a bidirectional communication bus. It converts the data to be transmitted between serial communication and parallel communication. In some embodiments, the UART interface is generally used to connect the processor 110 and the wireless communication module 160. For example, the processor 110 communicates with the Bluetooth module in the wireless communication module 160 through the UART interface to implement the Bluetooth function. In some embodiments, the audio module 170 can transmit an audio signal to the wireless communication module 160 through the UART interface to implement the function of playing music through a Bluetooth headset.
[0131] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the electronic device 100.
[0132] The GPIO interface can be configured by software. The GPIO interface can be configured as a control signal or as a data signal. In some embodiments, the GPIO interface can be used to connect the processor 110 with the camera 193, the display 194, the wireless communication module 160, the audio module 170, the sensor module 180, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.
[0133] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the electronic device 100, and can also be used to transfer data between the electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.
[0134] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present invention is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.
[0135] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.
[0136] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.
[0137] The wireless communication function of the electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.
[0138] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.
[0139] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G, etc., applied to the electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.
[0140] The modem processor may include a modulator and a demodulator. Among them, the modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device (not limited to a speaker 170A, a receiver 170B, etc.), or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.
[0141] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., which are applied to the electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and performs filtering, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.
[0142] In some embodiments, the antenna 1 of the electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).
[0143] The electronic device 100 implements the display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.
[0144] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Miniled, MicroLed, Micro-oLed, a quantum dot light-emitting diode (QLED), etc. In some embodiments, the electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.
[0145] The electronic device 100 can realize the shooting function through ISP, camera 193, video codec, GPU, display screen 194 and application processor.
[0146] ISP is used to process the data fed back by camera 193. For example, when taking a photo, the shutter is opened, and the light is transmitted to the camera photosensitive element through the lens. The light signal is converted into an electrical signal, and the camera photosensitive element transmits the electrical signal to ISP for processing and converts it into an image visible to the naked eye. ISP can also perform algorithm optimization on the noise, brightness, and skin color of the image. ISP can also optimize the exposure, color temperature and other parameters of the shooting scene. In some embodiments, ISP can be set in camera 193.
[0147] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal oxide semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.
[0148] The digital signal processor is used to process digital signals, and can process not only digital image signals but also other digital signals. For example, when the electronic device 100 is selecting a frequency point, the digital signal processor is used to perform Fourier transform on the frequency point energy.
[0149] Video codecs are used to compress or decompress digital videos. The electronic device 100 may support one or more video codecs. Thus, the electronic device 100 may play or record videos in a variety of coding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.
[0150] NPU is a neural network (NN) computing processor. By drawing on the structure of biological neural networks, such as the transmission mode between neurons in the human brain, it can quickly process input information and can also continuously self-learn. Through NPU, applications such as intelligent cognition of electronic device 100 can be realized, such as image recognition, face recognition, voice recognition, text understanding, etc.
[0151] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.
[0152] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.
[0153] The electronic device 100 can implement audio functions such as music playing and recording through the audio module 170, the speaker 170A, the receiver 170B, the microphone 170C, the headphone jack 170D, and the application processor.
[0154] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signals. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110.
[0155] The speaker 170A, also called a "speaker", is used to convert an audio electrical signal into a sound signal. The electronic device 100 can listen to music or listen to a hands-free call through the speaker 170A.
[0156] The receiver 170B, also called a "earpiece", is used to convert audio electrical signals into sound signals. When the electronic device 100 receives a call or voice message, the voice can be received by placing the receiver 170B close to the human ear.
[0157] Microphone 170C, also called "microphone" or "microphone", is used to convert sound signals into electrical signals. When making a call or sending a voice message, the user can speak by putting their mouth close to microphone 170C to input the sound signal into microphone 170C. The electronic device 100 can be provided with at least one microphone 170C. In other embodiments, the electronic device 100 can be provided with two microphones 170C, which can not only collect sound signals but also realize noise reduction function. In other embodiments, the electronic device 100 can also be provided with three, four or more microphones 170C to collect sound signals, reduce noise, identify the sound source, realize directional recording function, etc.
[0158] The earphone interface 170D is used to connect a wired earphone and can be a USB interface 130 or a 3.5 mm open mobile terminal platform (OMTP) standard interface or a cellular telecommunications industry association of the USA (CTIA) standard interface.
[0159] The pressure sensor 180A is used to sense the pressure signal and convert the pressure signal into an electrical signal. In some embodiments, the pressure sensor 180A can be disposed on the display screen 194.
[0160] There are many types, such as resistive pressure sensors, inductive pressure sensors, capacitive pressure sensors, etc. Capacitive pressure sensors can be at least two parallel plates with conductive materials. When a force acts on the pressure sensor 180A, the capacitance between the electrodes changes. The electronic device 100 determines the intensity of the pressure based on the change in capacitance. When a touch operation acts on the display screen 194, the electronic device 100 detects the touch operation intensity according to the pressure sensor 180A. The electronic device 100 can also calculate the touch position based on the detection signal of the pressure sensor 180A. In some embodiments, touch operations acting on the same touch position but with different touch operation intensities can correspond to different operation instructions. For example: when a touch operation with a touch operation intensity less than the first pressure threshold acts on the short message application icon, an instruction to view the short message is executed. When a touch operation with a touch operation intensity greater than or equal to the first pressure threshold acts on the short message application icon, an instruction to create a new short message is executed.
[0161] The gyro sensor 180B can be used to determine the motion posture of the electronic device 100. In some embodiments, the angular velocity of the electronic device 100 around three axes (i.e., x, y, and z axes) can be determined by the gyro sensor 180B. The gyro sensor 180B can be used for anti-shake shooting. For example, when the shutter is pressed, the gyro sensor 180B detects the angle of the electronic device 100 shaking, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to offset the shaking of the electronic device 100 through reverse movement to achieve anti-shake. The gyro sensor 180B can also be used for navigation and somatosensory game scenes.
[0162] The air pressure sensor 180C is used to measure air pressure. In some embodiments, the electronic device 100 calculates the altitude through the air pressure value measured by the air pressure sensor 180C to assist positioning and navigation.
[0163] The magnetic sensor 180D includes a Hall sensor. The electronic device 100 can use the magnetic sensor 180D to detect the opening and closing of the flip leather case. In some embodiments, when the electronic device 100 is a flip phone, the electronic device 100 can detect the opening and closing of the flip cover according to the magnetic sensor 180D. Then, according to the detected opening and closing state of the leather case or the opening and closing state of the flip cover, the flip cover can be automatically unlocked.
[0164] The acceleration sensor 180E can detect the magnitude of the acceleration of the electronic device 100 in all directions (generally three axes). When the electronic device 100 is stationary, the magnitude and direction of gravity can be detected. It can also be used to identify the posture of the electronic device and is applied to applications such as horizontal and vertical screen switching and pedometers.
[0165] The distance sensor 180F is used to measure the distance. The electronic device 100 can measure the distance by infrared or laser. In some embodiments, when shooting a scene, the electronic device 100 can use the distance sensor 180F to measure the distance to achieve fast focusing.
[0166] The proximity light sensor 180G may include, for example, a light emitting diode (LED) and a light detector, such as a photodiode. The light emitting diode may be an infrared light emitting diode. The electronic device 100 emits infrared light outward through the light emitting diode. The electronic device 100 uses a photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that there is an object near the electronic device 100. When insufficient reflected light is detected, the electronic device 100 can determine that there is no object near the electronic device 100. The electronic device 100 can use the proximity light sensor 180G to detect that the user holds the electronic device 100 close to the ear to talk, so as to automatically turn off the screen to save power. The proximity light sensor 180G can also be used in leather case mode and pocket mode to automatically unlock and lock the screen.
[0167] The ambient light sensor 180L is used to sense the ambient light brightness. The electronic device 100 can adaptively adjust the brightness of the display screen 194 according to the perceived ambient light brightness. The ambient light sensor 180L can also be used to automatically adjust the white balance when taking pictures. The ambient light sensor 180L can also cooperate with the proximity light sensor 180G to detect whether the electronic device 100 is in a pocket to prevent accidental touches.
[0168] The fingerprint sensor 180H is used to collect fingerprints. The electronic device 100 can use the collected fingerprint characteristics to implement fingerprint unlocking, access application locks, fingerprint photography, fingerprint call answering, etc.
[0169] The temperature sensor 180J is used to detect temperature. In some embodiments, the electronic device 100 uses the temperature detected by the temperature sensor 180J to execute a temperature processing strategy. For example, when the temperature reported by the temperature sensor 180J exceeds a threshold, the electronic device 100 reduces the performance of a processor located near the temperature sensor 180J to reduce power consumption and implement thermal protection. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 heats the battery 142 to avoid abnormal shutdown of the electronic device 100 due to low temperature. In other embodiments, when the temperature is lower than another threshold, the electronic device 100 performs a boost on the output voltage of the battery 142 to avoid abnormal shutdown caused by low temperature.
[0170] The touch sensor 180K is also called a "touch control device". The touch sensor 180K can be set on the display screen 194. The touch sensor 180K and the display screen 194 form a touch screen, also called a "touch control screen". The touch sensor 180K is used to detect touch operations acting on or near it. The touch sensor can pass the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through the display screen 194. In other embodiments, the touch sensor 180K can also be set on the surface of the electronic device 100, which is different from the position of the display screen 194.
[0171] The bone conduction sensor 180M can obtain a vibration signal. In some embodiments, the bone conduction sensor 180M can obtain a vibration signal of a vibrating bone block of the vocal part of the human body. The bone conduction sensor 180M can also contact the human pulse to receive a blood pressure beat signal. In some embodiments, the bone conduction sensor 180M can also be set in an earphone and combined into a bone conduction earphone. The audio module 170 can parse out a voice signal based on the vibration signal of the vibrating bone block of the vocal part obtained by the bone conduction sensor 180M to realize a voice function. The application processor can parse the heart rate information based on the blood pressure beat signal obtained by the bone conduction sensor 180M to realize a heart rate detection function.
[0172] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The electronic device 100 may receive key input and generate key signal input related to user settings and function control of the electronic device 100.
[0173] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. For example, touch operations acting on different applications (such as taking pictures, audio playback, etc.) can correspond to different vibration feedback effects. For touch operations acting on different areas of the display screen 194, motor 191 can also correspond to different vibration feedback effects. Different application scenarios (for example: time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also support customization.
[0174] Indicator 192 may be an indicator light, which may be used to indicate charging status, power changes, messages, missed calls, notifications, etc.
[0175] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to and separated from the electronic device 100 by inserting it into the SIM card interface 195 or pulling it out from the SIM card interface 195. The electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 195 can support Nano SIM cards, Micro SIM cards, SIM cards, and the like. Multiple cards can be inserted into the same SIM card interface 195 at the same time. The types of the multiple cards can be the same or different. The SIM card interface 195 can also be compatible with different types of SIM cards. The SIM card interface 195 can also be compatible with external memory cards. The electronic device 100 interacts with the network through the SIM card to implement functions such as calls and data communications. In some embodiments, the electronic device 100 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 100 and cannot be separated from the electronic device 100.
[0176] Figure 4 A flow chart of an embodiment of a smart device control method provided in an embodiment of the present application includes:
[0177] Step 401: In response to a detected first operation of a user, the first device 10 turns on a device control shooting mode.
[0178] Specifically, the user can operate on the display screen 194 of the first device 10 to turn on the device control shooting mode of the first device 10. Among them, the device control shooting mode can be called the smart shooting mode. Exemplarily, the first device 10 can pre-install a first application. Among them, the first application can be a system application or a third-party application. The embodiment of the present application does not specifically limit the type of the above-mentioned first application. , The first application can be used to provide smart shooting services. In specific implementations, the first application can be called a smart remote control application. The user can click on the icon of the above-mentioned first application on the desktop of the first device 10 to open the first application and enter the device control shooting mode setting page. It can be understood that the above example only exemplarily shows the way in which the user opens the first application by clicking, and does not constitute a limitation on the embodiments of the present application. In some embodiments, the above-mentioned first application can also be opened in other ways, such as double-clicking, sliding, etc.
[0179] Next, the user can turn on the device control shooting mode in the device control shooting mode setting page. Exemplarily, the user can turn on the switch of the smart remote control to turn on the device control shooting mode.
[0180] Now combined Figure 5a and Figure 5bThe opening of the device control shooting mode is explained. When the user opens the first application and enters the device control shooting mode setting page, the user can obtain Figure 5a The page 500 shown is a device control shooting mode setting page. Figure 5a As described above, the page 500 includes a smart remote control switch 501. At this time, the smart remote control switch 501 is in a closed state, that is, as described above, the device control shooting mode is not turned on at this time. Then, the user can click the smart remote control switch 501 on the page 500, thereby obtaining Figure 5b Page 510 is shown. Figure 5b As shown, the smart remote control switch 501 is in the on state, that is, at this time, the first device 10 enters the device control shooting mode.
[0181] Step 402 : In response to a detected second operation of the user in the device-controlled shooting mode, the first device 10 acquires an image of the second device 20 .
[0182] Specifically, the user can return to the desktop in the first device 10, and can aim the camera 193 of the first device 10 at the object to be photographed (for example, the second device 20), so as to obtain the image of the second device 20. In specific implementation, since the first device 10 has already turned on the device control shooting mode in step 301, the user does not need to manually open the camera application. When the user holds the first device 10 upright and aims it at the second device 20, the first device 10 can automatically turn on the camera and obtain the image of the second device 20 through shooting, thereby improving the convenience of the user in shooting images, and further improving the user experience.
[0183] Step 403 : The first device 10 obtains the recognition result of the image of the second device 20 .
[0184] Specifically, after the first device 10 acquires the image of the second device 20, the recognition result can be acquired based on the image of the second device 20. The above-mentioned method of acquiring the recognition result can be that the first device 10 locally recognizes the image of the second device 20. Exemplarily, the image of the second device 20 can be recognized using the image recognition algorithm in the first device 10, thereby obtaining the recognition result.
[0185] Optionally, the above-mentioned method of obtaining the recognition result can also be that the first device 10 uploads the image of the second device 20 to the third device 30, and recognizes the image of the second device 20 through the powerful computing power of the third device 30, thereby obtaining the recognition result. Then, the third device 30 can return the recognition result to the first device 10, thereby reducing the computing burden of the first device 10.
[0186] It is understandable that the above recognition result may be the device type of the smart home appliance. For example, the device type may be a TV type, a lighting type, an air conditioner type, etc. Taking the TV type as an example, a user may have multiple TVs at home, for example, one in the living room, one in the master bedroom, and one in the second bedroom. By recognizing the image of the above second device 20, if the device type of the second device 20 is identified as a TV type, since it is unknown which TV it is, it is necessary to further determine which TV the second device 20 is, so that the second device 20 can be controlled.
[0187] Step 404: The first device 10 obtains a preset device list.
[0188] Specifically, the preset device list may include multiple smart home appliances to be selected. It is understandable that the above-mentioned smart home appliances can be interconnected via WIFI. Among them, each smart home appliance to be selected has a preset device type. Exemplarily, the above-mentioned smart home appliances can pre-establish a local area network. The local area network can be a combination of one or more types of bus type, ring type, star type and tree type. The embodiment of the present application does not specifically limit the type of the above-mentioned local area network.
[0189] Then, the first device 10 can obtain the above-mentioned preset device list. Among them, the above-mentioned way for the first device 10 to obtain the preset device list may include the following two. First, the first device 10 can connect to the local area network via WIFI, and can obtain the above-mentioned preset device list through the local area network. That is to say, the first device 10 can query the selected smart home appliances that have been connected to the above-mentioned local area network. Second, after the above-mentioned smart home appliances are connected to the local area network, the above-mentioned preset device list can be sent to the third device 30. The first device 10 can send a device list request to the third device 30 to request the device list of the selected smart home appliances. When the third device 30 receives the device list request of the first device 10, it can send the device list of the selected smart home appliances to the first device 10.
[0190] It is understandable that when any smart home appliance to be selected exits the local area network, for example, the smart home appliance is offline, the smart home appliance no longer appears in the preset device list. When any smart home appliance is connected to the local area network, the smart home appliance becomes a smart home appliance to be selected and appears in the preset device list. When any smart home appliance to be selected exits the local area network or any smart home appliance joins the local area network, the preset device list can be synchronized with the third device 30.
[0191] Step 405 : The first device 10 determines the number of target devices in the preset device list that are of the same device type as the above identification result.
[0192] Specifically, the first device 10 can match the identified device type with the device type of each selected smart home appliance in the above preset device list, thereby obtaining the number of target devices in the local area network with the same device type as the second device 20. For example, assume that the user has multiple TVs at home, for example, one in the living room, one in the master bedroom, and one in the second bedroom. At this time, if the device type of the second device 20 is identified as a TV, it can be determined that the number of target devices (TVs) is 3 (one in the living room, one in the master bedroom, and one in the second bedroom).
[0193] Step 406 : The first device 10 displays the control interface of the second device 20 .
[0194] Specifically, after the first device 10 determines the number of target devices, it can determine the second device 20 in the target devices and display the control interface of the second device 20. In specific implementation, the control interface can be implemented by a control card, each control card can correspond to a smart home appliance, that is, each control card can be used to control a smart home appliance, and the control card can include a function menu and a control for controlling the corresponding smart home appliance. The above-mentioned display control interface can include but is not limited to the following two methods.
[0195] Method 1: Manual display
[0196] Specifically, the first device 10 may first determine the priorities of the multiple target devices, and display a device list of the target devices in order of priority on the desktop of the first device 10, wherein the device list of the target devices may include multiple target devices. When the user selects one of the devices in the device list of the target devices, the interface of the first device 10 may display a corresponding control interface for controlling the second device 20.
[0197] In a specific implementation, after the first device 10 obtains the number of target devices, the number can be compared with a preset first threshold, thereby determining whether to directly display the device list of the target devices on the interface of the first device 10 .
[0198] Exemplarily, the above-mentioned preset first threshold can be set to 2. When the number of target devices is less than 2, that is, if the first device 10 only matches one target device, at this time, the first device 10 can directly display the control interface on the interface.
[0199] When the number of target devices is greater than or equal to 2, that is, the first device 10 matches multiple target devices, at this time, the first device 10 can further determine the priorities of the above multiple target devices, and can display the device list in order of priority. Among them, the above priorities can be used to distinguish the display order of the target devices. For example, target devices with high priorities can be displayed preferentially in the above device list. In specific implementation, the above method of determining the priority of the target device can be implemented by the received signal strength (Received Signal Strength Indication, RSSI), and the above RSSI is used to characterize the received signal strength of Bluetooth between the first device 10 and the target device.
[0200] The first device 10 can obtain the RSSI of each target device and compare the RSSI of all target devices. If the RSSI difference between the target devices is obvious, the priority can be determined according to the RSSI value of each target device, and can be sorted according to the priority. It can be understood that the above method of determining the RSSI difference can be: obtaining the maximum RSSI and the second largest RSSI, and calculating the difference between the maximum RSSI and the second largest RSSI. If the difference is greater than or equal to the preset second threshold, it can be determined that the RSSI difference between the target devices is obvious. If the difference is less than the preset second threshold, it can be determined that the RSSI difference between the target devices is not obvious.
[0201] Next, if the RSSI difference between the target devices is obvious, the priority of the target device can be determined according to the RSSI of each target. For example, the target device corresponding to the highest RSSI (for the convenience of explanation, the "target device corresponding to the highest RSSI" is referred to as the "first target device" below) can be the highest priority, and the target device corresponding to the second highest RSSI (for the convenience of explanation, the "target device corresponding to the second highest RSSI" is referred to as the "second target device" below) can be the second highest priority, and so on. And according to the order of the above priorities, the device list including the above-mentioned first target device and the second target device can be displayed on the interface of the first device 10. It can be understood that the above example only illustrates the scenario of the first target device and the second target device, but is not limited to displaying only the above-mentioned first target device and the second target device. In addition, the display method of the device list of the above-mentioned target devices can be in the form of a pop-up window, and the embodiment of the present application does not specifically limit the display method of the above-mentioned device list.
[0202] Now combined Figure 6a-6c , taking two target devices (eg, a first target device and a second target device) as an example for explanation. Figure 6aThe first device 10 displays the interface before the device list of the target device. After the first device 10 compares the RSSI, it is determined that the RSSI value of the first target device is much greater than the RSSI value of the second target device. For example, the RSSI difference between the first target device and the second target device is greater than the preset second threshold. In other words, the RSSI difference between the first target device and the second target device is obvious, and the priority of the first target device is higher than that of the second target device. At this time, the first device 10 can display the following Figure 6b The interface 600 shown. Figure 6b As shown, the interface 600 may include a device list 601 of a first target device 6011 and a second target device 6012. Since the priority of the first target device 6011 is higher than that of the second target device 6012, according to the usual usage habits of people from left to right and from top to bottom, the position of the first target device 6011 in the device list 601 may be located on the left side of the second target device 6012. Optionally, according to the usual usage habits of people from top to bottom, the position of the first target device 6011 in the device list 601 may be located on the upper side of the second target device 6012. It is to be understood that the above-mentioned display position in the device list to reflect the priority of the target device is only an exemplary description and does not constitute a limitation on the embodiments of the present application. In some embodiments, the priority of the target device may also be reflected by other display methods.
[0203] Next, the user can operate the first target device 6011 or the second target device 6012 in the device list 601, such as clicking or sliding the first target device 6011 or the second target device 6012, so that the control interface corresponding to the above target device can be displayed. Taking the user clicking the first target device 6011 as an example, the following can be obtained: Figure 6c The interface 610 shown is as follows: Figure 6c As shown, the interface 610 may include a control interface 611 corresponding to the first target device 6011 , and the control interface 611 may be used to control the first target device 6011 (ie, the second device 20 ).
[0204] If the RSSI difference between the target devices (e.g., the first target device 6011 and the second target device 6012) is not obvious, the priority of the target device can be determined according to the user's historical usage habits, wherein the user's historical usage habits can be used to characterize the statistical information of the user's use of the target devices (e.g., the first target device 6011 and the second target device 6012). Exemplarily, the statistical information may include which smart home appliance the user uses at which time, and the statistical information may be statistical information obtained by collecting statistics on user data within a period (e.g., one month, half a year, or one year, etc.). In a specific implementation, a user prediction model can be pre-trained, and the user prediction model can be used to predict the probability of a user using a certain target device at a specific time. For example, the current time can be input into the above-mentioned user prediction model, thereby obtaining the probability of the user using all the smart home appliances at home at the current time, and then the first device 10 can filter out the probability of use of the target device corresponding to the currently identified device type from the usage probabilities of all the above-mentioned smart home appliances.
[0205] Optionally, the current time and device type may also be input into the user prediction model, thereby obtaining the probability that the user uses the target device corresponding to the device type at the current time.
[0206] It is understandable that the above determination of the use probability of the target device according to the time is only an exemplary description and does not constitute a limitation on the embodiments of the present application. In some embodiments, the current user's identity can also be input into the above user prediction model to comprehensively decide the probability of the current user using the target device. The probability of different users using the same target device at the same time is different. For example, the probability of the elderly at home turning on the radio in the morning is higher than that of children, and the probability of the middle-aged and young people at home turning on the TV in the evening is higher than that of the elderly. Through the comprehensive decision of user identity and time, the accuracy of the prediction can be improved.
[0207] Next, the priority of the target device may be determined according to the usage probability corresponding to each target device. For example, the target device with the highest usage probability has the highest priority, the target device with the second highest usage probability has the second highest priority, and so on.
[0208] Optionally, user prediction models are usually trained using neural networks, that is, the above-mentioned user prediction model is usually a neural network model. When using the above-mentioned user prediction model for calculation, a large amount of computing power of the first device 10 will be consumed, which will burden the system operation of the first device 10 and cause delays. Therefore, the first device 10 can also preset the mapping relationship between the time and / or user identification and the target device, so that the target device that the user may use at the current time can be determined according to the time and / or user identification. For example, the elderly at home usually turn on the radio at 7 o'clock in the morning, and the young and middle-aged people at home usually turn on the TV at 8 o'clock in the evening, etc. Therefore, through simple mapping, the target device corresponding to the current time and / or user can be output, thereby reducing delays and system consumption.
[0209] It is understandable that the above mapping relationship can be a one-to-one mapping, for example, a time and user identification can be mapped to a target device. Optionally, the above mapping relationship can be a one-to-many mapping, for example, a time and user identification can be mapped to multiple target devices, wherein each mapped target device can have a corresponding usage probability. Exemplarily, the young and middle-aged people at home turn on the TV at 8 o'clock in the evening, wherein the probability of turning on the TV in the living room is 80%, and the probability of turning on the TV in the bedroom is 20%. Then, the priority of the target device can be determined according to the usage probability of the above target device, for example, the target device with a high usage probability has a high priority.
[0210] After determining the priority of the target device, the device list of the target device can be further displayed in the order of priority on the interface of the first device 10. And after the user selects the target device in the device list, the control interface corresponding to the second device 20 can be displayed on the interface of the first device 10. The specific process of displaying the device list of the target device and the control interface of the second device 20 on the interface of the first device 10 can be referred to. Figure 6a-6c , I will not go into details here.
[0211] Method 2: Automatic display
[0212] Specifically, the first device 10 can first determine the priorities of multiple target devices, and can display the control interface of the highest priority target device on the interface of the first device 10. The control interface of the highest priority target device can be considered as the control interface of the second device 20, which is used to control the second device 20.
[0213] In a specific implementation, after the first device 10 obtains the number of target devices, the number can be compared with a preset first threshold, thereby determining whether to directly display the control interface of the second device 20 on the interface of the first device 10 .
[0214] Exemplarily, the above-mentioned preset first threshold can be set to 2. When the number of target devices is less than 2, that is, if the first device 10 only matches one target device, at this time, the first device 10 can directly display the control interface of the target device on the interface, and the control interface of the target device can be used to control the second device 20.
[0215] When the number of target devices is greater than or equal to 2, that is, the first device 10 matches multiple target devices, then the first device 10 can further determine the priorities of the multiple target devices and display the control interface of the target device with the highest priority on the interface of the first device 10, thereby facilitating the user's operation and improving the control efficiency of the smart home appliance. For the specific display effect of the control interface, please refer to Figure 6c , which will not be described in detail here. In specific implementation, the above method of determining the priority of the target device can be implemented through RSSI, or according to the user's historical usage habits. The specific process of determining the priority of the target device through RSSI and the user's historical usage habits can refer to method 1, which will not be described in detail here.
[0216] It can be understood that in the above embodiment, steps 401 to 406 are all optional steps. The present application only provides one feasible embodiment, which may also include more or fewer steps than steps 401 to 406, and the present application does not limit this.
[0217] Figure 7 This is a schematic diagram of the structure of an embodiment of the device control device of the present application, such as Figure 7 As shown, the device control device 70 is applied to the first device and may include: a shooting module 71, an acquisition module 72, a matching module 73 and a display module 74; wherein,
[0218] A shooting module 71, used for shooting an image of a second device;
[0219] An acquisition module 72, configured to acquire a device type of the second device, where the device type of the second device is determined according to an image of the second device;
[0220] A matching module 73, configured to match the device type of the second device with the first device list, and determine a plurality of target devices of the same device type as the second device, wherein the first device list includes a plurality of to-be-selected devices, each of which has a preset device type;
[0221] The display module 74 is used to obtain the signal strength between the first device and each target device, the signal strength is used to determine the second device from the target devices; and display the control interface of the second device.
[0222] In one possible implementation, the acquisition module 72 is further configured to perform image recognition on the image of the second device, and determine the device type of the second device according to the recognition result.
[0223] In one possible implementation, the acquisition module 72 is further used to send the image of the second device to a third device, and the image of the second device is used by the third device to determine the device type of the second device; and receive the device type of the second device sent by the third device.
[0224] In one possible implementation, the first device is installed with a first application, and the device control apparatus 70 further includes:
[0225] An enabling module 75, configured for enabling, by the first application, a device-controlled shooting mode in response to a detected first operation of a user;
[0226] The shooting module 71 is also used by the first application to capture an image of the second device in response to a detected second operation of the user in the device control shooting mode.
[0227] In one possible implementation, the display module 74 is further configured to display a second device list based on the signal strength; wherein the second device list includes part or all of the multiple target devices;
[0228] In response to a detected operation of a user selecting a second device in the second device list, a control interface of the second device is displayed.
[0229] In one possible implementation, the display module 74 is further configured to determine the target device corresponding to the highest signal strength as the second device, and display a control interface of the second device.
[0230] In one possible implementation, the display module 74 is further used to obtain a first target device corresponding to the highest signal strength and a second target device corresponding to the second highest signal strength;
[0231] Based on the order of signal strength from strong to weak, the first target device and the second target device are displayed in sequence.
[0232] In one possible implementation, the display module 74 is also used to display the first target device and the second target device sequentially based on the order of signal strength from strong to weak if the difference between the signal strength of the first target device and the signal strength of the second target device is greater than or equal to a preset signal strength threshold.
[0233] In one possible implementation, the display module 74 is further used to obtain a first target device corresponding to the highest signal strength and a second target device corresponding to the second highest signal strength;
[0234] If the difference between the signal strength of the first target device and the signal strength of the second target device is less than a preset signal strength threshold, the first target device and the second target device are displayed according to the user's historical usage habits, wherein the user's historical usage habits are used to characterize statistical information of the user's use of the first target device and the second target device.
[0235] In one possible implementation, the display module 74 is further used to determine the usage probabilities corresponding to the first target device and the second target device respectively according to the user's historical usage habits;
[0236] The first target device and the second target device are displayed in sequence based on the order of usage probability from high to low.
[0237] In one possible implementation manner, the first device is a mobile terminal.
[0238] In one possible implementation, the second device is a smart home appliance.
[0239] Figure 7 The device control device 70 provided in the embodiment shown can be used to execute the present application Figure 1 -The technical solution of the method embodiment shown in FIG6 , its implementation principle and technical effects can be further referred to the relevant description in the method embodiment.
[0240] It should be understood that the above Figure 7 The division of the various modules of the device control device 70 shown is only a division of logical functions. In actual implementation, they can be fully or partially integrated into one physical entity, or they can be physically separated. And these modules can all be implemented in the form of software calling through processing elements; they can also be all implemented in the form of hardware; some modules can also be implemented in the form of software calling through processing elements, and some modules can be implemented in the form of hardware. For example, the detection module can be a separately established processing element, or it can be integrated in a chip of an electronic device. The implementation of other modules is similar. In addition, all or part of these modules can be integrated together, or they can be implemented independently. In the implementation process, each step of the above method or each of the above modules can be completed by an integrated logic circuit of hardware in the processor element or instructions in the form of software.
[0241] For example, the above modules may be one or more integrated circuits configured to implement the above methods, such as one or more application specific integrated circuits (ASIC), or one or more microprocessors (DSP), or one or more field programmable gate arrays (FPGA). For another example, these modules may be integrated together and implemented in the form of a system-on-a-chip (SOC).
[0242] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the electronic device 100. In other embodiments of the present application, the electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.
[0243] It is understandable that, in order to realize the above functions, the above-mentioned electronic devices, etc. include hardware structures and / or software modules corresponding to the execution of each function. Those skilled in the art should easily realize that, in combination with the units and algorithm steps of each example described in the embodiments disclosed herein, the embodiments of the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to exceed the scope of the embodiments of the present application.
[0244] The embodiment of the present application can divide the functional modules of the above-mentioned electronic device etc. according to the above-mentioned method example. For example, each functional module can be divided corresponding to each function, or two or more functions can be integrated into one processing module. The above-mentioned integrated module can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of modules in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.
[0245] Through the description of the above implementation methods, technicians in the relevant field can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.
[0246] Each functional unit in each embodiment of the present application can be integrated into a processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units.
[0247] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the embodiment of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) or a processor to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as flash memory, mobile hard disk, read-only memory, random access memory, disk or optical disk.
[0248] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. A device control method, applied to a first device, characterized in that: The method comprises: capturing an image of the second device; Acquire a device type of the second device, where the device type of the second device is determined according to the image of the second device; Matching the device type of the second device with the first device list to determine a plurality of target devices of the same device type as the second device, wherein the first device list includes a plurality of to-be-selected devices, the plurality of to-be-selected devices include the second device, and each of the to-be-selected devices has a preset device type; Acquire a signal strength between the first device and each of the target devices, wherein the signal strength is used to determine the second device from the target devices; Displaying a second device list based on the signal strength; wherein the second device list includes part or all of the multiple target devices; In response to a detected operation of the user selecting the second device from the second device list, displaying a control interface of the second device; The displaying of the second device list based on the signal strength includes: Obtain a first target device corresponding to the highest signal strength, and a second target device corresponding to the second highest signal strength; if the difference between the signal strength of the first target device and the signal strength of the second target device is greater than or equal to a preset signal strength threshold, display the first target device and the second target device in sequence based on the order of the signal strengths from strong to weak; if the difference between the signal strength of the first target device and the signal strength of the second target device is less than the preset signal strength threshold, display the first target device and the second target device according to the user's historical usage habits, wherein the user's historical usage habits are used to characterize statistical information on the user's use of the first target device and the second target device.
2. The method according to claim 1, characterized in that The acquiring the device type of the second device includes: Perform image recognition on the image of the second device, and determine the device type of the second device according to the recognition result.
3. The method according to claim 1, characterized in that The acquiring the device type of the second device includes: Sending the image of the second device to a third device, where the image of the second device is used by the third device to determine the device type of the second device; Receive the device type of the second device sent by the third device.
4. The method according to any one of claims 1 to 3, characterized in that: The first device is installed with a first application; Before capturing the image of the second device, the method further includes: the first application, in response to a detected first operation of the user, turning on a device control capture mode; The capturing of the image of the second device comprises: The first application captures an image of the second device in response to a detected second operation of the user in the device-controlled shooting mode.
5. The method according to any one of claims 1 to 3, characterized in that: The displaying of the first target device and the second target device according to the user's historical usage habits comprises: Determine usage probabilities corresponding to the first target device and the second target device respectively according to the user's historical usage habits; Based on the order of the usage probability from high to low, the first target device and the second target device are displayed in sequence.
6. The method according to any one of claims 1 to 3, characterized in that: The first device is a mobile terminal.
7. The method according to any one of claims 1 to 3, characterized in that: The second device is a smart home appliance.
8. A first device, characterized in that: include: A memory, wherein the memory is used to store computer program codes, wherein the computer program codes include instructions, and when the first device reads the instructions from the memory, the first device executes the method according to any one of claims 1 to 7.
9. A device control system, characterized in that: The method comprises the first device and the third device as claimed in claim 8, wherein: The third device is used to receive the image of the second device sent by the first device, perform image recognition on the image, and send the recognition result to the first device, where the recognition result is the device type of the second device.
10. A computer-readable storage medium, characterized in that: The method comprises computer instructions, which, when executed on the first device, cause the first device to execute the method according to any one of claims 1 to 7.
Citation Information
Patent Citations
Method for controlling household electrical appliance through mobile terminal, mobile terminal, and storage device
CN107682237A
Device voice control method and system, and smart terminal
CN109410943A
Household equipment control method and device for simplifying user operation and household equipment
CN110262264A
Smart home control method and device
CN110908340A
Equipment control method and device, storage medium and mobile terminal
CN112671620A
