Equipment searching method and electronic equipment

Abstract

The invention provides an equipment searching method and electronic equipment, and relates to the technical field of terminals. In the process that the electronic device displays the device search page including the display element associated with the wall surface of the room, the user intention can be determined according to the user operation, so that the user is helped to quickly search the target device meeting the user intention. The method comprises the steps that in response to a first operation of a user, a first electronic device displays a device search page, and the device search page comprises display elements associated with the wall surface of a room; in response to a second operation of the user on the device searching page and / or a first pointing operation of the user, the first electronic device determines a first target device searched by the user, the first target device is an object associated with the wall orientation, and the first pointing operation is an operation that the user rotates the first electronic device to point to the first target device. And then, the first electronic equipment presents or selects an identifier corresponding to the first target equipment.

Description

Technical Field

[0001] The present application relates to the field of terminal technology, and in particular to a device search method and an electronic device. Background Art

[0002] With the development of terminal technology, more and more devices support intelligence, and users can meet various needs in life through different smart devices. For example, smart devices include smart screens, Bluetooth speakers, air purifiers and other smart home devices, thus providing users with a smart home life.

[0003] Generally, users can control all the smart devices in the house through a certain control device. However, as the number of smart devices increases, it becomes increasingly difficult for users to find the smart devices they want to control through the control device, which affects user usage. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides a device search method and an electronic device. The technical solution provided by the present application can determine the user's intention according to the user's operation during the process of the electronic device displaying a device search page including display elements associated with the wall of the room, thereby helping the user to quickly find the target device that meets the user's intention.

[0005] In order to achieve the above technical objectives, this application provides the following technical solutions:

[0006] In a first aspect, a device search method is provided, which is applied to a first electronic device. The method includes: in response to a first operation of a user, displaying a device search page, wherein the device search page includes a display element associated with a wall of a room. In response to a second operation of the user on the device search page and / or a first pointing operation of the user, determining a first target device searched by the user, wherein the first target device is an object associated with a wall orientation, and the first pointing operation is an operation in which the user rotates the first electronic device to point to the first target device. Presenting or selecting an identifier corresponding to the first target device.

[0007] The display elements associated with the walls of the room include, for example, the name of the wall, a schematic mark of the wall (such as a line segment used to indicate the wall), and the like.

[0008] For example, in response to the user's first operation, the first electronic device locates the current room as the living room, and can display a device search page for searching for devices in the living room. For example, the device search page displays the TV wall and background wall included in the living room, as well as line segments used to represent the wall.

[0009] In this way, the first electronic device displays a device search page including display elements associated with the wall of the room, and the user only needs to perform simple interface operations and / or pointing operations to achieve device search.

[0010] According to the first aspect, in response to the user's second operation on the device search page and / or the user's first pointing operation, determining the first target device the user is looking for includes: in response to the user's second operation on the device search page and / or the user's first pointing operation, determining that the wall where the device the user is looking for is located is the first wall.

[0011] Exemplarily, during the process of displaying the device search page, the first electronic device detects a second operation of the user moving the position control to a display element associated with the TV wall (such as a TV wall segment), and determines that the wall where the device the user is searching for is located is the TV wall.

[0012] As another example, during the process of displaying the device search page, the first electronic device detects a first pointing operation of the user turning the first electronic device to point to the TV wall, and determines that the wall where the device searched by the user is located is the TV wall.

[0013] In this way, the first electronic device first determines the wall according to the user operation, and then determines that the device the user instructs to find is one or several devices on the wall.

[0014] According to the first aspect, or any implementation of the first aspect above, the method further includes: in response to a third operation of the user on the device search page and / or a second pointing operation of the user, determining that the wall where the device searched by the user is located is a second wall, and a second target device searched by the user on the second wall. Presenting or selecting an identifier corresponding to the second target device.

[0015] The third operation and the second pointing operation include a single operation or a series of sub-operations. For example, after the user searches for the second wall through the third operation, the user searches for the second target device through the second pointing operation. Alternatively, the user searches for the second wall through the second pointing operation first, and then searches for the second target device through the third operation. Alternatively, the user searches for the second wall and the second target device through the second operation. Alternatively, the user searches for the second wall and the second target device through the second pointing operation.

[0016] In this way, based on the anchoring relationship between the wall and the device, the user can quickly find the desired target device through simple interface operations and / or pointing operations.

[0017] According to the first aspect, or any implementation of the first aspect above, the method further includes: in response to a fourth operation of the user on the first wall and / or a third pointing operation of the user on the device search page, determining a third target device of the user, and presenting or selecting an identifier corresponding to the third target device.

[0018] In this way, after finding the desired first target device, the user can continue to search for other desired third target devices through interface operations and / or pointing operations.

[0019] According to the first aspect, or any implementation of the first aspect above, the identifier of the first target device is a controllable control, and the method further includes: in response to a user operation on the identifier corresponding to the first target device, sending instruction information to the first target device, the instruction information is used to control the first target device. Alternatively, in response to a user operation on the identifier corresponding to the first target device, displaying a control page of the first target device, the control page is used to control the first target device.

[0020] Optionally, the identifier corresponding to the first target device is a controllable identifier or an uncontrollable identifier.

[0021] In this way, users can find and control devices through simple operations, improving the user experience.

[0022] According to the first aspect, or any implementation of the first aspect above, presenting or selecting an identifier corresponding to the first target device includes: in response to a second operation of a user lingering on a first display element associated with a first wall, presenting an identifier of a fourth target device. During the process of detecting the second operation of the user lingering on the first display element, a first pointing operation of the user shaking the first electronic device up and down is detected, and the presenting of the identifier of the fourth target device is switched to the presenting of the identifier of the first target device.

[0023] In this way, the user can combine interface operations and pointing operations to flexibly find the desired target device.

[0024] According to the first aspect, or any implementation of the first aspect above, in response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the first target device that the user is searching for includes: in response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the type of the user's search operation. In the case where the search operation type is a location search operation type, displaying an identifier of a device to be searched. In the case where the search operation type is an area search operation type, displaying multiple identifiers of devices to be searched. Wherein, the first target device is one or more devices among the devices to be searched.

[0025] In this way, the first electronic device can determine whether the user instructs to search for one target device or multiple target devices based on the user's interface operation on the device search page or the user's pointing operation of rotating the first electronic device.

[0026] According to the first aspect, or any implementation of the first aspect above, in response to the user's second operation on the device search page and / or the user's first pointing operation, determining the first target device searched by the user includes: in response to the user's second operation on the device search page and / or the user's first pointing operation, determining that the user's search operation type is a function search operation type.

[0027] It should be understood that the location search operation type, area search operation type, and function search operation type are merely names used to distinguish operation types. There may also be other naming methods, and the embodiments of the present application do not limit this.

[0028] According to the first aspect, or any implementation of the first aspect above, in response to the user's second operation on the device search page and / or the user's first pointing operation, determining the user's search operation type includes: detecting the user's operation of moving the location control displayed on the device search page to the first display element associated with the first wall, and determining the search operation type to be a location search operation type. Alternatively, detecting the user's operation of moving the area control displayed on the device search page to the first display element, and determining the search operation type to be an area search operation type. Alternatively, detecting the user's operation of pointing the first electronic device to the first wall and not moving at a certain position on the first wall or moving within a range less than a preset range, and determining the search operation type to be a location search operation type. Alternatively, detecting the user's operation of pointing the first electronic device to the first wall and moving within a range on the first wall greater than a preset range, and determining the search operation type to be an area search operation type.

[0029] According to the first aspect, or any implementation of the first aspect above, in response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the first target device that the user is looking for includes: detecting a fifth operation of the user moving the position control displayed on the device search page to the first display element associated with the first wall, and displaying the identifier of the first device corresponding to the resting position of the position control on the first display element. Detecting a sixth operation of the user moving the position control on the first display element, displaying the identifier of the second device corresponding to the final resting position, or switching to display the identifiers of multiple devices corresponding to the movement trajectory during the movement. Determining the second device corresponding to the final resting position as the first target device.

[0030] In this way, if the user only remembers the wall where the required device is located, but forgets or remembers the specific location of the device on the wall incorrectly, the user can also quickly find the required target device through interface operation and / or pointing operation, thereby further reducing the user's memory difficulty and improving the user's usage experience.

[0031] According to the first aspect, or any implementation of the first aspect above, in response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the first target device that the user is looking for includes: detecting a seventh operation in which the user moves the area control displayed on the device search page to the first display element associated with the first wall, and displaying the identifiers of multiple devices to be found corresponding to the first wall. In response to the first pointing operation of the user, determining the area range of the first pointing operation. Matching the area range and multiple devices to be found, and determining some or all of the multiple devices to be found as the first target device.

[0032] According to the first aspect, or any implementation of the first aspect above, after detecting a seventh operation in which a user moves a region control displayed on a device search page to a first display element associated with a first wall, and displaying the identifiers of multiple devices to be found corresponding to the first wall, the method further includes: detecting an eighth operation in which a user moves the region control on the first display element, and determining a starting position and an ending position of the eighth operation. Determine multiple first target devices covered by the starting position and the ending position among the multiple devices to be found.

[0033] In this way, while achieving group control, the user's personalized and flexible control needs can be met.

[0034] According to the first aspect, or any implementation of the first aspect above, in response to the user's second operation on the device search page and / or the user's first pointing operation, determining the first target device the user is looking for includes: detecting a ninth operation in which the user moves the area control displayed on the device search page toward the first display element associated with the first wall, displaying a subsystem identifier and / or a scene identifier, the multiple third devices corresponding to the subsystem identifier are devices of the same subsystem among the multiple devices to be found, the multiple fourth devices corresponding to the scene identifier are devices in the same scene among the multiple devices to be found, and the multiple devices to be found are devices corresponding to the first wall.

[0035] According to the first aspect, or any implementation of the first aspect above, the method further includes: detecting a tenth operation of the user on the subsystem identifier, determining that the plurality of third devices are first target devices. Detecting an eleventh operation of the user on the scene identifier, determining that the plurality of fourth devices are first target devices.

[0036] In this way, after the first electronic device determines the device data of multiple smart home devices corresponding to the wall indicated by the user according to the user's interface operation or pointing operation, it can determine the control subsystem and / or scene corresponding to the current wall to display the corresponding group control controls to better meet the user's personalized needs.

[0037] According to the first aspect, or any implementation of the first aspect above, in response to the user's second operation on the device search page and / or the user's first pointing operation, determining the first target device the user is looking for includes: determining the first wall and / or the first target device based on the first pointing angle at which the user rotates the first electronic device to point to the wall.

[0038] According to the first aspect, or any implementation of the first aspect above, in response to the user's second operation on the device search page and / or the user's first pointing operation, determining the first target device the user is looking for includes: determining a second pointing angle based on the stop position of the second operation on the first display element associated with the first wall, and the second pointing angle is used to determine the first target device.

[0039] According to the first aspect, or any implementation of the first aspect above, in response to a first operation of a user, displaying a device search page includes: in response to the first operation of the user, acquiring a spatial data model, the spatial data model including house data, room data, wall data, and device data. Displaying the device search page based on the spatial data model and the positioning data of the first electronic device.

[0040] According to the first aspect, or any implementation of the first aspect above, the house data is used to locate the house where the first target device is located, the room data is used to locate the room where the first target device is located, the wall data is used to locate the wall where the first target device is located, and the device data is used to locate the position of the first target device on the wall.

[0041] In this way, the first electronic device displays a corresponding device search page through a preset space data model in combination with the space where the first electronic device is currently located, thereby realizing device search in the space corresponding to the device search page.

[0042] Moreover, combined with the spatial data model, there is no need to pre-configure base stations in space, which effectively reduces hardware costs.

[0043] In addition, the first electronic device filters devices that are not in the current house through house data, filters devices that are not in the current room through room data, filters devices that are not anchored on the current wall through wall data, and matches the device closest to the current angle through device data, thereby determining the target device through four layers of filtering and improving the accuracy of device search and control.

[0044] According to the first aspect, or any implementation of the first aspect above, the room data includes a reference anchor point, and the device data includes a first azimuth angle of the device relative to the reference anchor point; in response to a second operation of the user on the device search page and / or a first pointing operation of the user, determining the first target device searched by the user includes: detecting the first pointing operation, obtaining a second azimuth angle corresponding to the first pointing operation, and converting the second azimuth angle into the first azimuth angle to determine the first target device pointed to by the first azimuth angle.

[0045] Among them, the position of the user in the process of searching for the device is not necessarily the reference anchor point position in the room. Different users may have different heights, resulting in different heights of the position where the first electronic device is operated. This leads to a deviation between the operating position and the reference anchor point position, affecting the pointing angle. Therefore, in order to ensure the accuracy of the target device finally determined, the first electronic device can determine the device actually pointed to by the user through conversion calculation based on the currently determined position data and spatial data model.

[0046] According to the first aspect, or any implementation of the first aspect above, the method further includes: detecting a user's operation of adding a device, instructing the user to point the first electronic device to the newly added device, and displaying a prompt message that the newly added device has been successfully added.

[0047] Optionally, the first electronic device instructs the user to stand at a position where the first electronic device is located at a reference anchor point, so that the first electronic device points from the reference anchor point to the newly added device.

[0048] In this way, the first electronic device can add a new searchable device according to the user operation to meet the user's demand for adding a new searchable device.

[0049] In a second aspect, an electronic device is provided. The electronic device includes: a processor, a memory and a display screen, the memory and the display screen are coupled to the processor, the memory is used to store computer program code, the computer program code includes computer instructions, when the processor reads the computer instructions from the memory, the electronic device executes: in response to the user's first operation, a device search page is displayed, and the device search page includes display elements associated with the wall of the room. In response to the user's second operation on the device search page and / or the user's first pointing operation, the first target device searched by the user is determined, the first target device is an object associated with the wall orientation, and the first pointing operation is an operation in which the user rotates the first electronic device to point to the first target device. An identifier corresponding to the first target device is presented or selected.

[0050] According to the second aspect, in response to the user's second operation on the device search page and / or the user's first pointing operation, determining the first target device the user is looking for includes: in response to the user's second operation on the device search page and / or the user's first pointing operation, determining that the wall where the device the user is looking for is located is the first wall.

[0051] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instruction from the memory, the electronic device is further caused to execute: in response to the user's third operation on the device search page and / or the user's second pointing operation, determine that the wall where the device searched by the user is located is the second wall, and the second target device searched by the user on the second wall. Present or select an identifier corresponding to the second target device.

[0052] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instruction from the memory, the electronic device is further caused to execute: in response to the fourth operation of the user on the first wall and / or the third pointing operation of the user on the device search page, determine the third target device of the user. Present or select an identifier corresponding to the third target device.

[0053] According to the second aspect, or any implementation of the second aspect above, the identifier of the first target device is a controllable control, and when the processor reads the computer instruction from the memory, it also causes the electronic device to execute: in response to the user's operation on the identifier corresponding to the first target device, send instruction information to the first target device, and the instruction information is used to control the first target device. Or, in response to the user's operation on the identifier corresponding to the first target device, display a control page of the first target device, and the control page is used to control the first target device.

[0054] According to the second aspect, or any implementation of the second aspect above, presenting or selecting an identifier corresponding to the first target device includes: in response to a second operation of a user lingering on a first display element associated with a first wall, presenting an identifier of a fourth target device. During the process of detecting the second operation of the user lingering on the first display element, a first pointing operation of the user shaking the first electronic device up and down is detected, and presenting the identifier of the fourth target device is switched to presenting the identifier of the first target device.

[0055] According to the second aspect, or any implementation of the second aspect above, in response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the first target device that the user is searching for includes: in response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the type of the user's search operation. In the case where the search operation type is a location search operation type, displaying an identifier of a device to be searched. In the case where the search operation type is an area search operation type, displaying multiple identifiers of devices to be searched. Wherein, the first target device is one or more devices among the devices to be searched.

[0056] According to the second aspect, or any implementation of the second aspect above, in response to the user's second operation on the device search page and / or the user's first pointing operation, determining the first target device searched by the user includes: in response to the user's second operation on the device search page and / or the user's first pointing operation, determining that the user's search operation type is a function search operation type.

[0057] According to the second aspect, or any implementation of the second aspect above, in response to the user's second operation on the device search page and / or the user's first pointing operation, determining the user's search operation type includes: detecting the user's operation of moving the location control displayed on the device search page to the first display element associated with the first wall, and determining that the search operation type is a location search operation type. Or, detecting the user's operation of moving the area control displayed on the device search page to the first display element, and determining that the search operation type is an area search operation type. Or, detecting the user's operation of pointing the first electronic device to the first wall and not moving at a certain position on the first wall or the moving range is less than a preset range, determining that the search operation type is a location search operation type. Or, detecting the user's operation of pointing the first electronic device to the first wall and the moving range on the first wall is greater than a preset range, and determining that the search operation type is an area search operation type.

[0058] According to the second aspect, or any implementation of the second aspect above, in response to the user's second operation on the device search page and / or the user's first pointing operation, determining the first target device that the user is looking for includes: detecting a fifth operation in which the user moves the position control displayed on the device search page to the first display element associated with the first wall, and displaying the identifier of the first device corresponding to the resting position of the position control on the first display element. Detecting a sixth operation in which the user moves the position control on the first display element, displaying the identifier of the second device corresponding to the final resting position, or switching to display the identifiers of multiple devices corresponding to the movement trajectory during the movement. Determining the second device corresponding to the final resting position as the first target device.

[0059] According to the second aspect, or any implementation of the second aspect above, in response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the first target device that the user is looking for includes: detecting a seventh operation in which the user moves the area control displayed on the device search page to the first display element associated with the first wall, and displaying the identifiers of multiple devices to be found corresponding to the first wall. In response to the first pointing operation of the user, determining the area range of the first pointing operation. Matching the area range and multiple devices to be found, and determining some or all of the multiple devices to be found as the first target device.

[0060] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instruction from the memory, the electronic device is further caused to execute: detecting an eighth operation of the user moving the area control on the first display element, determining the starting position and the ending position of the eighth operation. Determining multiple first target devices covered by the starting position and the ending position among multiple devices to be found.

[0061] According to the second aspect, or any implementation of the second aspect above, in response to the user's second operation on the device search page and / or the user's first pointing operation, determining the first target device the user is looking for includes: detecting a ninth operation in which the user moves the area control displayed on the device search page toward the first display element associated with the first wall, displaying a subsystem identifier and / or a scene identifier, the multiple third devices corresponding to the subsystem identifier are devices of the same subsystem among the multiple devices to be found, the multiple fourth devices corresponding to the scene identifier are devices in the same scene among the multiple devices to be found, and the multiple devices to be found are devices corresponding to the first wall.

[0062] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instruction from the memory, the electronic device is further caused to execute: detecting the user's tenth operation on the subsystem identifier, determining that the plurality of third devices are first target devices. Detecting the user's eleventh operation on the scene identifier, determining that the plurality of fourth devices are first target devices.

[0063] According to the second aspect, or any implementation of the second aspect above, in response to the user's second operation on the device search page and / or the user's first pointing operation, determining the first target device the user is looking for includes: determining the first wall and / or the first target device based on the first pointing angle at which the user rotates the first electronic device to point to the wall.

[0064] According to the second aspect, or any implementation of the second aspect above, in response to the user's second operation on the device search page and / or the user's first pointing operation, determining the first target device the user is looking for includes: determining a second pointing angle based on the stop position of the second operation on the first display element associated with the first wall, and the second pointing angle is used to determine the first target device.

[0065] According to the second aspect, or any implementation of the second aspect above, in response to a first operation of a user, displaying a device search page includes: in response to the first operation of the user, acquiring a spatial data model, the spatial data model including house data, room data, wall data, and device data. Displaying the device search page based on the spatial data model and the positioning data of the first electronic device.

[0066] According to the second aspect, or any implementation of the second aspect above, the house data is used to locate the house where the first target device is located, the room data is used to locate the room where the first target device is located, the wall data is used to locate the wall where the first target device is located, and the device data is used to locate the position of the first target device on the wall.

[0067] According to the second aspect, or any implementation of the second aspect above, the room data includes a reference anchor point, and the device data includes a first azimuth angle of the device relative to the reference anchor point; in response to a second operation of the user on the device search page and / or a first pointing operation of the user, determining the first target device searched by the user includes: detecting the first pointing operation, obtaining a second azimuth angle corresponding to the first pointing operation, and converting the second azimuth angle into the first azimuth angle to determine the first target device pointed to by the first azimuth angle.

[0068] According to the second aspect, or any implementation of the second aspect above, when the processor reads the computer instruction from the memory, the electronic device is further caused to execute: upon detecting the user's operation of adding a new device, instructing the user to point the first electronic device to the new device, and displaying a prompt message that the new device has been successfully added.

[0069] In a third aspect, an electronic device is provided, which has the function of implementing the device search method described in the first aspect and any possible implementation thereof. The function can be implemented by hardware, or by hardware executing corresponding software. The hardware or software includes one or more modules corresponding to the above functions.

[0070] In a fourth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program (also referred to as an instruction or code), and when the computer program is executed by an electronic device, the electronic device executes the method of the first aspect or any one of the implementations of the first aspect.

[0071] According to a fifth aspect, a computer program product is provided. When the computer program product is run on an electronic device, the electronic device executes the method according to the first aspect or any one of the implementations of the first aspect.

[0072] According to a sixth aspect, a circuit system is provided, the circuit system comprising a processing circuit, wherein the processing circuit is configured to execute the method according to the first aspect or any one of the embodiments of the first aspect.

[0073] In the seventh aspect, a chip system is provided, comprising at least one processor and at least one interface circuit, wherein the at least one interface circuit is used to perform transceiver functions and send instructions to the at least one processor, and when the at least one processor executes the instructions, the at least one processor executes the method of the first aspect or any one of the embodiments of the first aspect.

[0074] The technical effects of the aforementioned aspects can be referenced to each other and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0075] Figure 1 Schematic diagram of the smart home device management interface provided in the embodiment of the present application Figure 1 ;

[0076] Figure 2 Schematic diagram of the smart home device management interface provided in the embodiment of the present application Figure 2 ;

[0077] Figure 3-1 A schematic diagram of the principle of a pointing control device provided in an embodiment of the present application;

[0078] Figure 3-2 A schematic diagram of a pointing control device scenario provided in an embodiment of the present application;

[0079] Figure 4 A schematic diagram of a communication system to which the device search method provided in an embodiment of the present application is applied;

[0080] Figure 5 A schematic diagram of the hardware structure of a first electronic device provided in an embodiment of the present application;

[0081] Figure 6 A schematic diagram of a home scene provided in an embodiment of the present application;

[0082] Figure 7 A schematic diagram of a reference anchor point provided in an embodiment of the present application;

[0083] Figure 8 A schematic diagram of the device locations provided for the embodiments of the present application;

[0084] Fig. 9A schematic diagram of the overall process of the device search scenario provided in the embodiment of the present application;

[0085] Fig.10 A schematic diagram of a pointing angle conversion scenario provided in an embodiment of the present application;

[0086] Fig.11 A schematic diagram of the pointing angle conversion principle provided in an embodiment of the present application;

[0087] Fig.12 A schematic diagram of smart home devices included in the device search scenario provided in an embodiment of the present application;

[0088] Fig.13 Schematic diagram of the device search function startup interface provided in the embodiment of the present application Figure 1 ;

[0089] Fig.14 Schematic diagram of the device search function startup interface provided in the embodiment of the present application Figure 2 ;

[0090] Fig.15 Schematic diagram of a device search scenario in a location search operation type provided in an embodiment of the present application Figure 1 ;

[0091] Fig.16 Schematic diagram of a device search scenario in a location search operation type provided in an embodiment of the present application Figure 2 ;

[0092] Fig.17 Schematic diagram 3 of a device search scenario in a location search operation type provided in an embodiment of the present application;

[0093] Fig.18 Schematic diagram of a device search scenario in a location search operation type provided in an embodiment of the present application Figure 4 ;

[0094] Fig.19 Schematic diagram of a device search scenario in a location search operation type provided in an embodiment of the present application Figure 5 ;

[0095] Fig. 20 Schematic diagram of a device search scenario in a location search operation type provided in an embodiment of the present application Figure 6 ;

[0096] Fig.21 Schematic diagram of a device search scenario in a location search operation type provided in an embodiment of the present application Figure 7 ;

[0097] Fig. 22 Schematic diagram of a device search scenario in a zone search operation type provided in an embodiment of the present application Figure 1;

[0098] Fig.23 Schematic diagram of a device search scenario in a zone search operation type provided in an embodiment of the present application Figure 2 ;

[0099] Fig.24 Schematic diagram 3 of a device search scenario in a zone search operation type provided in an embodiment of the present application;

[0100] Fig.25 Schematic diagram of a device search scenario in a zone search operation type provided in an embodiment of the present application Figure 4 ;

[0101] Fig.26 Schematic diagram of a device search scenario in a zone search operation type provided in an embodiment of the present application Figure 5 ;

[0102] Fig. 27 A schematic diagram of a device search scenario in a function search operation type provided in an embodiment of the present application;

[0103] Fig.28 A schematic diagram of a device search method flow chart provided in an embodiment of the present application;

[0104] Fig.29 A schematic diagram of the structure of a first electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0105] The technical solutions in the embodiments of the present application are described below in conjunction with the accompanying drawings in the embodiments of the present application. Wherein, in the description of the embodiments of the present application, the terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and the appended claims of the present application, the singular expressions "one", "a kind of", "said", "above", "the" and "this" are intended to include expressions such as "one or more", unless there is a clear indication to the contrary in the context. It should also be understood that in the following embodiments of the present application, "at least one", "one or more" refer to one or more (including two).

[0106] References to "one embodiment" or "some embodiments" etc. described in this specification mean that one or more embodiments of the present application include specific features, structures or characteristics described in conjunction with the embodiment. Thus, the statements "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear in different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways. The term "connection" includes direct connection and indirect connection, unless otherwise specified. "First" and "second" are used for descriptive purposes only and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated.

[0107] In the embodiments of the present application, the words "exemplarily" or "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplarily" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "exemplarily" or "for example" is intended to present related concepts in a specific way.

[0108] In some embodiments, the control device may display one or more smart devices managed by the user account after logging into the user account in response to the user operation. In this way, the user may control multiple smart devices through one control device.

[0109] For example, Figure 1 As shown, the electronic device displays the smart life application interface in response to the user operation. On the homepage of the smart life application, multiple smart devices managed under the currently logged-in user account can be displayed. For example, the TV in the living room, the TV in the master bedroom, the speakers in the master bedroom, etc. The user can select the smart device he wants to control in the current interface.

[0110] However, with the development of terminal technology, the number of smart devices is increasing, and it is becoming increasingly difficult for users to find the smart devices they want to control in the smart home application interface.

[0111] In view of the above situation, the smart home application supports classifying smart devices by room, so that when the user searches for a smart device, he can first determine the room where the device to be found is located, and then search for the smart device to be found in the interface corresponding to the room.

[0112] For example, Figure 2As shown, the electronic device displays multiple room cards on the homepage of the smart life application, and the multiple room cards include information about multiple smart devices in the corresponding rooms. In this way, when the user needs to find a smart device, he can open the corresponding room card and then search for the smart device in the room corresponding to the room card. For example, if the user needs to turn on the light in the living room, he can click the living room card to instruct the electronic device to display the device search interface corresponding to the living room. After that, the user can find the control of the light to be controlled in the device search interface.

[0113] Although the above method can reduce the difficulty of users finding smart devices to a certain extent. However, if there are many rooms or a single room contains many smart devices, it is still difficult for users to find a smart device. Figure 2 As shown, in the current scenario, the living room includes 67 smart devices. It is obviously very difficult for a user to find a smart device that he needs to control among these 67 smart devices.

[0114] In addition, users need to remember which room the smart device they need to control is in, which also requires a certain level of memory from the user.

[0115] In response to the above situation, the electronic device also supports a pointing control function. When the user points the electronic device at the smart device to be controlled, the electronic device can directly display the control interface of the smart device, making it easier for the user to control the smart device without having to search for the device on the device search interface.

[0116] In some examples, a positioning base station with known coordinates is pre-configured in the room, and the positioning base station is used to locate the position of the smart device in the room. After that, when the user brings an electronic device equipped with ultra wide band (UWB) positioning function into the room, the electronic device can emit pulses at a certain frequency and continuously interact with the base station with known coordinates. In this way, the base station can obtain the distance and angle relationship between the electronic device and the smart device through a preset algorithm, thereby achieving control.

[0117] For example, Figure 3-1 As shown in Figure 1, different smart devices in the room can form a certain distance and angle relationship with the mobile phone carried by the user. Figure 3-2 As shown in (a), the user holds the phone and changes the direction of the phone until Figure 3-2As shown in (b), the mobile phone points to the smart screen to be controlled. During this process, the mobile phone transmits pulses at a certain frequency and continuously interacts with the base station with known coordinates. Then, after the mobile phone points to the smart screen, the base station can determine that the smart device pointed at by the user after stopping turning the mobile phone is the smart screen based on the distance and angle relationship between the mobile phone and the smart screen. The base station can send the device information of the smart screen to the mobile phone, so that the mobile phone can display the control interface of the smart screen. Figure 3-2 As shown in (c), in response to the user's operation on the control interface, the mobile phone can send a startup signal to the smart screen to instruct the smart screen to start up. Therefore, in the above process, the user only needs to point the mobile phone at the smart screen to control it, which is easy to operate.

[0118] However, the realization of the pointing control function of the above electronic device requires the pre-configuration of a positioning base station, which has a high hardware cost and supports fewer smart devices.

[0119] In addition, in the current scenario, users, control devices (such as mobile phones), and smart devices to be controlled (such as smart screens) need to be in the same room, and the geographical location restrictions are high, resulting in limited application scenarios.

[0120] In order to solve the technical problems mentioned above, an embodiment of the present application provides a device search method. By pre-building a spatial data model, the electronic device can determine the user's intention based on the user operation and the spatial data model, thereby helping the user to quickly find the target device that meets the user's intention.

[0121] Figure 4 A schematic diagram of a communication system to which the device search method provided in an embodiment of the present application is applied. Figure 4 As shown, the communication system includes a first electronic device 100 and a second electronic device 200 .

[0122] Optionally, the first electronic device 100 is a terminal device that can be used to control the second electronic device 200. For example, the first electronic device 100 can be a terminal device with a display function, such as a mobile phone, a tablet computer, a laptop computer, an ultra-mobile personal computer (UMPC), a netbook, a personal digital assistant (PDA), a wearable device, an artificial intelligence (AI) device, etc. The operating system installed in the first device 100 includes but is not limited to Or other operating systems. This application does not limit the specific type of the first device 100 and the installed operating system.

[0123] Optionally, the second electronic device 200 may be, for example, a terminal device such as a smart home device, a mobile phone, a tablet computer, a laptop computer, a netbook, a PDA, a wearable device, an AI device, etc. Among them, the smart home device includes, for example, a smart home control panel, audio and video equipment (such as a smart screen, a speaker, etc.), lighting system equipment (such as a desk lamp, a lamp, etc.), environmental control equipment (such as an air purifier, a sweeping robot, etc.), security system equipment (such as a camera, a smart door lock, etc.), kitchen appliances (such as an oven, a dishwasher, etc.), smart sensors, smart sockets, etc.

[0124] The operating system installed on the second electronic device 200 includes but is not limited to Or other operating systems. The second electronic device 200 may not be installed with an operating system. This application does not limit the specific type of the second electronic device 200, whether an operating system is installed, and the type of operating system if an operating system is installed.

[0125] In some embodiments, the first electronic device 100 can establish a wireless communication connection with the second electronic device 200 through wireless communication technology. The wireless communication technology includes but is not limited to at least one of the following: wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) network), power line communication (PLC), Bluetooth (BT) (for example, traditional Bluetooth or low energy (BLE) Bluetooth), near field communication (NFC), Zigbee, frequency modulation (FM), infrared (IR), UWB connection, etc.

[0126] In some embodiments, the operating system is, for example, a smart home system, and the second electronic device 200 is a smart home device. Alternatively, the operating system is a smart office system, and the second electronic device 200 is a smart office device. Alternatively, the communication system may also be other communication systems such as a smart campus communication system, which is not limited in the embodiments of the present application. The following takes the operating system as, for example, a smart home system, and the second electronic device 200 as a smart home device as an example to introduce the device search method provided in the embodiments of the present application in detail.

[0127] Optionally, the first electronic device 100 and the second electronic device 200 in the embodiment of the present application may be implemented by different devices, and the different devices may have the same, similar or somewhat different hardware structures, such as Figure 5 The hardware structure shown.

[0128] For example, the first electronic device 100 has Figure 5 Take the hardware structure shown in the figure as an example. Figure 5 The hardware structure shown is described.

[0129] like Figure 5 As shown, the first electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna 1, an antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a sensor module 180, a button 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc.

[0130] It is to be understood that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the first electronic device 100. In other embodiments of the present application, the first electronic device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0131] The processor 110 may include one or more processing units, for example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0132] The controller can generate operation control signals according to the instruction operation code and timing signal to complete the control of instruction fetching and execution.

[0133] The processor 110 may also be provided with a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. The memory may store instructions or data that the processor 110 has just used or cyclically used. If the processor 110 needs to use the instruction or data again, it may be directly called from the memory. This avoids repeated access, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0134] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0135] The MIPI interface can be used to connect the processor 110 with peripheral devices such as the display screen 194 and the camera 193. The MIPI interface includes a camera serial interface (CSI), a display serial interface (DSI), etc. In some embodiments, the processor 110 and the camera 193 communicate via the CSI interface to implement the shooting function of the first electronic device 100. The processor 110 and the display screen 194 communicate via the DSI interface to implement the display function of the first electronic device 100.

[0136] The USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the first electronic device 100, and can also be used to transfer data between the first electronic device 100 and a peripheral device. It can also be used to connect headphones to play audio through the headphones. The interface can also be used to connect other electronic devices, such as AR devices, etc.

[0137] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the first electronic device 100. In other embodiments of the present application, the first electronic device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0138] The charging management module 140 is used to receive charging input from a charger. The charger may be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 140 may receive charging input from a wired charger through the USB interface 130. In some wireless charging embodiments, the charging management module 140 may receive wireless charging input through a wireless charging coil of the first electronic device 100. While the charging management module 140 is charging the battery 142, it may also power the electronic device through the power management module 141.

[0139] The power management module 141 is used to connect the battery 142, the charging management module 140 and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, and supplies power to the processor 110, the internal memory 121, the display screen 194, the camera 193, and the wireless communication module 160. The power management module 141 can also be used to monitor parameters such as battery capacity, battery cycle number, battery health status (leakage, impedance), etc. In some other embodiments, the power management module 141 can also be set in the processor 110. In other embodiments, the power management module 141 and the charging management module 140 can also be set in the same device.

[0140] The wireless communication function of the first electronic device 100 can be implemented through the antenna 1, the antenna 2, the mobile communication module 150, the wireless communication module 160, the modem processor and the baseband processor.

[0141] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in the first electronic device 100 can be used to cover a single or multiple communication frequency bands. Different antennas can also be reused to improve the utilization of the antennas. For example, antenna 1 can be reused as a diversity antenna for a wireless local area network. In some other embodiments, the antenna can be used in combination with a tuning switch.

[0142] The mobile communication module 150 can provide solutions for wireless communications including 2G / 3G / 4G / 5G applied to the first electronic device 100. The mobile communication module 150 may include at least one filter, a switch, a power amplifier, a low noise amplifier (LNA), etc. The mobile communication module 150 can receive electromagnetic waves from the antenna 1, and filter, amplify, and process the received electromagnetic waves, and transmit them to the modulation and demodulation processor for demodulation. The mobile communication module 150 can also amplify the signal modulated by the modulation and demodulation processor, and convert it into electromagnetic waves for radiation through the antenna 1. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the processor 110. In some embodiments, at least some of the functional modules of the mobile communication module 150 can be set in the same device as at least some of the modules of the processor 110.

[0143] The modem processor may include a modulator and a demodulator. The modulator is used to modulate the low-frequency baseband signal to be sent into a medium-high frequency signal. The demodulator is used to demodulate the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After the low-frequency baseband signal is processed by the baseband processor, it is passed to the application processor. The application processor outputs a sound signal through an audio device, or displays an image or video through a display screen 194. In some embodiments, the modem processor may be an independent device. In other embodiments, the modem processor may be independent of the processor 110 and be set in the same device as the mobile communication module 150 or other functional modules.

[0144] The wireless communication module 160 can provide wireless communication solutions including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), infrared (IR), etc., applied on the first electronic device 100. The wireless communication module 160 can be one or more devices integrating at least one communication processing module. The wireless communication module 160 receives electromagnetic waves via the antenna 2, modulates the frequency of the electromagnetic wave signal and filters it, and sends the processed signal to the processor 110. The wireless communication module 160 can also receive the signal to be sent from the processor 110, modulate the frequency of it, amplify it, and convert it into electromagnetic waves for radiation through the antenna 2.

[0145] In some embodiments, the antenna 1 of the first electronic device 100 is coupled to the mobile communication module 150, and the antenna 2 is coupled to the wireless communication module 160, so that the first electronic device 100 can communicate with the network and other devices through wireless communication technology. The wireless communication technology may include global system for mobile communications (GSM), general packet radio service (GPRS), code division multiple access (CDMA), wideband code division multiple access (WCDMA), time-division code division multiple access (TD-SCDMA), long term evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technology, etc. The GNSS may include a global positioning system (GPS), a global navigation satellite system (GLONASS), a Beidou navigation satellite system (BDS), a quasi-zenith satellite system (QZSS) and / or a satellite based augmentation system (SBAS).

[0146] The first electronic device 100 implements a display function through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, which connects the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations for graphics rendering. The processor 110 may include one or more GPUs that execute program instructions to generate or change display information.

[0147] The display screen 194 is used to display images, videos, etc. The display screen 194 includes a display panel. The display panel can be manufactured using a liquid crystal display (LCD), such as an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode or an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), Mini-led, Micro-led, Micro-oled, quantum dot light emitting diodes (QLED), etc. In some embodiments, the first electronic device 100 may include 1 or N display screens 194, where N is a positive integer greater than 1.

[0148] The camera 193 is used to capture still images or videos. The object generates an optical image through the lens and projects it onto the photosensitive element. The photosensitive element can be a charge coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the optical signal into an electrical signal, and then passes the electrical signal to the ISP to be converted into a digital image signal. The ISP outputs the digital image signal to the DSP for processing. The DSP converts the digital image signal into an image signal in a standard RGB, YUV or other format. In some embodiments, the first electronic device 100 may include 1 or N cameras 193, where N is a positive integer greater than 1.

[0149] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the first electronic device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0150] The internal memory 121 can be used to store computer executable program codes, which include instructions. The internal memory 121 may include a program storage area and a data storage area. Among them, the program storage area may store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc. The data storage area may store data created during the use of the first electronic device 100 (such as audio data, a phone book, etc.), etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as at least one disk storage device, a flash memory device, a universal flash storage (UFS), etc. The processor 110 executes various functional applications and data processing of the first electronic device 100 by running instructions stored in the internal memory 121, and / or instructions stored in a memory provided in the processor.

[0151] The audio module 170 is used to convert digital audio information into analog audio signal output, and is also used to convert analog audio input into digital audio signal. The audio module 170 can also be used to encode and decode audio signals. In some embodiments, the audio module 170 can be arranged in the processor 110, or some functional modules of the audio module 170 can be arranged in the processor 110. The first electronic device 100 can use the audio module 170, such as music playing, recording, etc. The audio module 170 can include a speaker, a receiver, a microphone, a headphone interface, and an application processor to implement audio functions.

[0152] The sensor module 180 may include a pressure sensor, a gyro sensor, an air pressure sensor, a magnetic sensor, an acceleration sensor, a distance sensor, a proximity light sensor, a fingerprint sensor, a temperature sensor, a touch sensor, an ambient light sensor, a bone conduction sensor, and the like.

[0153] The key 190 includes a power key, a volume key, etc. The key 190 may be a mechanical key or a touch key. The first electronic device 100 may receive key input and generate key signal input related to user settings and function control of the first electronic device 100.

[0154] Motor 191 can generate vibration prompts. Motor 191 can be used for incoming call vibration prompts, and can also be used for touch vibration feedback. Motor 191 can also correspond to different vibration feedback effects for touch operations on different areas of display screen 194. Different application scenarios (for example: time reminder, receiving information, alarm clock, game, etc.) can also correspond to different vibration feedback effects.

[0155] The indicator 192 may be an indicator light, which may be used to indicate the charging status, power changes, messages, missed calls, notifications, etc.

[0156] The SIM card interface 195 is used to connect a SIM card. The SIM card can be connected to or disconnected from the first electronic device 100 by inserting or removing the SIM card interface 195. The first electronic device 100 can support 1 or N SIM card interfaces, where N is a positive integer greater than 1.

[0157] The following takes the first electronic device 100 as a control device and the second electronic device 200 as a smart home device as an example to introduce the device search method provided in the embodiment of the present application in detail. There are multiple smart home devices.

[0158] In some embodiments, the user lives in a home space environment and naturally forms a memory palace of the location of items in the home space in the brain. Then, the control device displays a device search page in response to the user operation, and the device search page may correspond to the actual space where the user is located. In this way, the user indicates the corresponding location on the device search page, and the control device can determine the smart home devices arranged at the location.

[0159] For example, the control device determines the room where the user is located based on its own positioning information. Afterwards, the control device can display the display elements corresponding to the current room on the device search page, such as the room name, the name of one or more walls, and schematic identifiers (such as line segments used to indicate walls, etc.). In this way, the user can perform touch operations on the device search page, such as clicking on a certain position on the wall schematic identifier, and the control device can determine the smart home device corresponding to the wall position corresponding to the operation position. Alternatively, the user directly turns the control device to point to the smart home device to be controlled, and the control device detects data through sensors to determine the wall pointed to by the user and the target smart home device.

[0160] In this way, users can quickly find devices through simple touch or pointing operations.

[0161] In some embodiments, the control device may determine the information of the current space in a variety of ways. For example, the control device locates the current location through a positioning module. Alternatively, the control device determines the spatial locations of different smart home devices by pre-establishing a spatial data model, so as to determine the target smart home device that the user is looking for in response to the user's operation.

[0162] The following introduces the device search method by taking the configuration of a spatial data model in a control device as an example.

[0163] In some embodiments, in a home scenario, such as Figure 6As shown, a house includes multiple rooms, each room includes multiple walls, and each wall can be placed with one or more smart home devices. Therefore, during the pre-installation process, technicians can build a spatial data model based on the house data, room data, wall data, and device data. The spatial data model is used to describe the positional relationship between the rooms, walls, and smart home devices in the current home space. For example, a TV is installed on the TV wall side of the living room, and smart speakers are installed on both sides of the TV.

[0164] The house data, room data, wall data, and equipment data in the spatial data model are introduced as follows.

[0165] Exemplarily, the technician configures the house data in the spatial data model through the following code. As shown in Table 1, houseId is used to represent the unique identifier of the house, such as houseId_1. Position is used to represent the location of the house, such as identifying the location of the house by the longitude and latitude coordinates of the house. In some examples, one or more spatial data models may be configured in the control device, and the control device may match the user's location with the house's location, thereby automatically switching the house where the current user is located, so as to facilitate the subsequent control of the smart home devices in the house. Rooms is used to represent a list of all rooms in the current house, such as the current houseId_1 includes roomId_1, roomId_2, and roomId_3.

[0166]

[0167] Table 1

[0168] Data name describe houseId The unique identifier of the house Position Indicates the location of the room Rooms Represents a list of all the rooms in the current house

[0169] Exemplarily, the technician configures the room data in the spatial data model through the following code. Among them, as shown in Table 2, roomId represents the unique identifier of the room, such as roomId_2. Position is used to indicate the position of the room, such as identifying the position of the room by the latitude and longitude coordinates of the room. In some examples, a house may include multiple rooms, and the control device can match the user's position with the room's position, thereby automatically switching the room where the current user is located, so as to facilitate the subsequent control of the smart home devices in the room. SpaceInfo is used to represent room data, such as the length, width, and height data of the room. In some examples, in the subsequent smart home device control process, the control device can determine the pointing angle of the control device according to SpaceInfo. Among them, the user can point the control device to the smart home device to be controlled, and the control device determines the pointing angle according to the spatial data model to determine the smart home device to be controlled indicated by the user. Among them, the specific pointing control method is described in detail in the relevant embodiment below, which will not be repeated here.

[0170] BaseAnchor is used to represent the reference anchor point data. In some examples, BaseAnchor is used as a reference to measure the position data of the control device in space. For example, when the user's spatial position changes (such as the user moves in the room), the control device can recalculate the pointing angle of the control device based on the relative relationship between the user's current position data and the reference anchor point, and then match the smart home device that the control device is pointing to. Among them, the method of re-matching the pointing angle according to the change of the user's position is described in detail in the relevant embodiment below, which will not be repeated here.

[0171] For example, Figure 7 As shown, the technician first constructs a coordinate system in the space of the current room. For example, the technician determines the stable origin position of the coordinate system according to specific rules. For example, a certain corner of the ground wall in the current room is used as the origin of the coordinate system to create a spatial coordinate system. After that, a certain point is determined in the spatial coordinate system as a reference anchor point. For example, based on the distance from the point to each wall, the spatial center point in the current room is determined as the reference anchor point.

[0172]

[0173]

[0174] Table 2

[0175] Data name describe roomId The unique identifier of the room Position Indicates the location of the room SpaceInfo Room data, such as the length, width, and height of the room BaseAnchor The reference anchor point is used to measure the position data of the control device in space. WallList Represents a list of all walls in the room

[0176] Exemplarily, the technician configures the wall data in the spatial data model through the following code. As shown in Table 3, wallId represents the unique identifier of the wall, such as wallId1. direction is used to represent the azimuth angle from the inside of the room to the wall, which is used to match the direction of the wall pointed by the control device. deviceList is used to represent the list of all smart home devices anchored to the wall. For example, smart home devices deviceId_1 and deviceId_2 are anchored on the wall with an azimuth angle of 56°.

[0177]

[0178] Table 3

[0179] Data name describe wallId The unique identifier of the wall direction Indicates the azimuth angle from the inside of the room to the wall, used to match the direction of the wall that the control device is pointing to deviceList Represents a list of all smart home devices anchored to the wall

[0180] Exemplarily, the technician configures the device data in the spatial data model through the following code. As shown in Table 4, deviceId represents the unique identifier of the smart home device, such as deviceId_1. supportOperate is used to indicate the supported search types. For example, the supported search types include area search and location search. Among them, area search is used to indicate the search for one or more smart home devices included in the area indicated by the user, and location search is used to indicate the search for a smart home device at the location indicated by the user. The specific implementation methods of area search and location search are detailed in the relevant embodiments below and will not be repeated here. Anchor is used to indicate the anchor point data of the smart home device on the wall. For example, Figure 8 As shown, the direction from the reference anchor point points to the smart home device, angleX represents the horizontal angle of the smart home device relative to the reference anchor point, and angleY ​​represents the vertical angle of the smart home device relative to the reference anchor point. Subsequently, the control device matches the pointing angle with the anchor point data in the device data to determine the smart home device pointed to by the user.

[0181]

[0182] Table 4

[0183] Data name describe deviceId The unique identifier of a smart home device. supportOperate Supported search types, such as region search and location search Anchor Represents the anchor point data of smart home devices on the wall

[0184] Therefore, the technicians use the above code to complete the entry of house data, room data, wall data, and equipment data to build a spatial data model corresponding to the house.

[0185] In some embodiments, after completing the construction of the spatial data model, the technician can upload the spatial data model to the cloud server. Then, when a subsequent user uses a control device to search for smart home devices in a house for the first time, the control device can instruct the control device to download the spatial data model from the cloud server. For example, the control device responds to user operations, logs into the user account, and requests the cloud server to download one or more spatial data models managed under the user account. Alternatively, the control device responds to user operations, logs into the user account, and requests the cloud server to download information about the spatial data model under the user account. In the case where the number of spatial data models under the user account is one, the cloud server is requested to download the one spatial data model. In the case where the number of spatial data models under the user account is multiple, the information of multiple houses corresponding to the multiple spatial data models is displayed. Afterwards, the control device determines the spatial data model corresponding to the house that the user instructs to operate according to the user's choice, and then requests the cloud server to download the spatial data model.

[0186] In some examples, after downloading the spatial data model, the control device can control one or more smart home devices corresponding to the spatial data model according to the user operation. In the subsequent device search process, the control device does not need to download the spatial data model again. In some examples, after determining that the spatial data model is updated, the cloud server can send an update instruction to the control device, and the control device can determine whether to update the spatial data model according to the update instruction. For example, the control device updates the spatial data model in response to the user operation.

[0187] In this way, users can use the same or different control devices as needed and control smart home devices by logging into their user accounts.

[0188] In some examples, technicians update the spatial data model based on changes in smart home devices in the house. In other examples, users update the spatial data model based on changes in smart home devices in the house during use.

[0189] For example, when a technician or user determines that a new smart home device needs to be added to a room, the handheld control device is located at the reference anchor point position shown in Table 2 above. Afterwards, the technician or user selects the device information of the smart home device that needs to be entered, and points the control device to the newly added smart home device. Among them, the control device obtains spatial information according to the user's pointing operation. In response to the user's instruction to enter the operation (such as clicking the entry control operation, etc.), the control device can establish an association between the device information and the spatial information, complete the information entry of the smart home device, so that the smart home device can be controlled subsequently. In some examples, the device information includes, for example, the identification of the smart home device (such as deviceId), the media access control (MAC) address, etc. The spatial information includes, for example, the room data of the smart home device determined by the control device in combination with the spatial data model, the wall data, the horizontal angle of the smart home device relative to the reference anchor point, the vertical angle of the smart home device relative to the reference anchor point, and other information.

[0190] In other embodiments, after completing the construction of the spatial data model, the technician can directly pre-set the spatial data model into the control device. For example, the technician completes the construction of the spatial data model through the control device. In some examples, the control device is, for example, a central control device in a house, such as a smart home control panel.

[0191] The above describes the construction and configuration method of the spatial data model. The following describes in detail the process of controlling the device to search for devices based on the spatial data model.

[0192] In some embodiments, the user lives in a home space environment, so it is easy for him to form a memory palace of the location of items in the home space in his brain. Then, the position relationship of smart home devices in the human brain and the smart home position relationship indicated in the spatial data model can be consistent. Therefore, when the user needs to control a smart home device, he directly enters the room and selects the smart home device by pointing or using the control device interface. The control device can determine the user's intention based on the user's operation and the spatial data model, and then determine the smart home device to be controlled. In this process, the difficulty of users finding the target smart home device among multiple smart home devices is reduced, and memory is reduced.

[0193] For example, Fig. 9As shown, the control device displays the device search interface according to the user operation. In some examples, the control device determines the house and room where the user is currently located through various methods such as the global positioning system (GPS), AI super-sensing sensor detection data, user interface input data, and user voice input data. Afterwards, the control device may display the device search interface corresponding to the current room, which is used to find and control smart home devices in the current room. Afterwards, the control device obtains the corresponding UI operation data and sensor data based on the user's operation on the device search interface and / or the pointing operation on the control device, thereby determining the user intent data. The control device combines the user intent data and the spatial data model, and through a preset algorithm, it can determine the target smart home device that the user indicates to control.

[0194] For example, when a user points a handheld control device at a TV wall, the control device can determine that the wall pointed by the user is a TV wall based on the sensor data, and then determine the device data of the smart home device anchored on the TV wall. Afterwards, the control device determines the change in the pointing angle of the control device relative to the TV wall based on the change in the sensor data. Afterwards, the control device determines the smart home device that the user finally points to by combining the horizontal angle and vertical angle included in the anchor data (Anchor) of different smart home devices in the device data already defined in the spatial data model. This smart home device is the target smart home device to be found by the user.

[0195] In this way, users only need simple UI operations and pointing operations to find and control smart home devices. In addition, combined with the spatial data model, there is no need to pre-configure base stations in the space, effectively reducing hardware costs.

[0196] In addition, the control device filters smart home devices that are not in the current house through house data, filters smart home devices that are not in the current room through room data, filters smart home devices that are not anchored on the current wall through wall data, and matches the device closest to the current angle through device data. Through four layers of filtering, the target smart home device is determined, thereby improving the accuracy of device search and control.

[0197] In some examples, the position of the user in the process of searching for the device is not necessarily the reference anchor point position in the room. Different users may have different heights, which may result in different heights of the operating control device. This will cause a deviation between the operating position and the reference anchor point position, affecting the pointing angle. Therefore, in order to ensure the accuracy of the target smart home device finally determined, the control device can determine the device actually pointed by the user through conversion calculation based on the currently determined position data and spatial data model.

[0198] For example, Fig.10 In the scenarios shown in (a)-(b), the user's position in space or the user operating the device changes, and the pointing angle of the user's position relative to the same smart home device changes. Fig.11 As shown, during the smart home device search process, when the user's spatial position changes (such as the user moves in the room, etc.), the control device can match the current pointing angle with angleX and angleY ​​in the device position (Anchor) according to the user's current room data (SpaceInfo), the reference anchor point (BaseAnchor), the user's current position data, and the device position (Anchor) through conversion calculation, and according to the relative relationship between the user's current position data and the reference anchor point, recalculate the pointing angle of the control device, and then match the smart home device pointed by the control device.

[0199] As follows Fig.12 The example scenario shown in the figure introduces the specific implementation process of the device search process in detail. Fig.12 As shown, in the current room, a row of smart spotlights are anchored on the top wall of the TV wall, a smart screen is anchored on the middle wall, and smart speakers are located on both sides of the smart screen; a smart air purifier and a smart socket are anchored on the bottom wall of the background wall.

[0200] In some embodiments, the control device is configured with a device search function, and the user can start the device search function in a variety of ways (such as UI operation, voice operation, etc.). In some examples, the device search function is used to find smart home devices. In some examples, after the control device starts the device search function, the device search interface can be displayed. In some examples, the control device can display the device search interface in full screen, or display a device search card, or display a device search floating window, etc.

[0201] In some examples, the control device responds to the activation of the device search function to determine the current house, room, and user location to display the corresponding device search interface. Among them, the data of the user's current room (CurrentRoom) can be obtained through GPS, AI super-sensing sensors, user interface input, user voice input, etc. The user's current location (CurrentUserPosition) is used to describe the user's relative position in the current room, such as orientation, coordinates, etc. The control device can obtain the user's current location through AI super-sensing sensors, user interface input, etc.

[0202] For example, the control device detects the user's operation on the smart life application icon, starts the smart life application, determines the current house, room, and user location, and displays the following information: Fig.13The device search interface 1301 is shown. A device search area 131 (also described as a device search card) is displayed on the device search interface 1301. The control device can determine the target smart home device that the user is looking for by detecting the user's operation on the device search area 131.

[0203] As another example, Fig.14 As shown, the control device displays a smart life card 141 on the desktop, and a device search area is displayed on the smart life card 141. The control device can determine the target smart home device that the user is looking for by detecting the user's operation on the smart life card 141. In some examples, as the user moves in the space, the display content on the smart life card 141 changes. For example, after the user moves to another room, the smart life card 141 displays the device search area corresponding to the moved room.

[0204] It should be understood that the control device can also display the device search interface in other ways. For example, the control device displays a device search card on the negative one screen. For another example, the control device displays a floating window on the desktop, and the floating window is used to display the device search interface. The embodiment of the present application does not limit this.

[0205] The following describes the device search process by taking the example of controlling the device to start the device search function through the smart life application and displaying the device search interface in the smart life application.

[0206] In some embodiments, the control device determines the target smart home device that the user indicates to search for by detecting the user's UI operation on the device search interface and / or the pointing operation of the control device in combination with the spatial data model.

[0207] In some examples, the control device determines, based on the user operation, that the user instructs to operate a smart home device or determines that the user instructs to operate multiple smart home devices. That is, the control device determines, based on the user operation, that the user needs to control a smart home device individually, or needs to control the smart home devices in a group. In some examples, the user's single control or group control needs are distinguished by the search operation type. For example, the search operation type includes a location search operation type and an area search operation type. The location search operation type is used to indicate that the user needs to find a smart home device corresponding to the current location, and the area search operation type is used to indicate that the user needs to find one or more smart home devices included in the current area.

[0208] For example, Fig.15In the interface 1501 shown in (a), in response to the user starting the device search function, the control device determines the house (such as the house where the control device is currently located) and the room (such as the room where the control device is currently located) that the user is searching for based on the spatial data model, and automatically obtains the current house data and room data, and the relative position of the user in the room to display the current room card 151. As shown in the room card 151, the room where the user is currently located is the living room, corresponding to Fig.12 In the scene shown, smart home devices are anchored on the TV wall and background wall of the living room. A region control 152 and a position control 153 are displayed on the room card 151. The user can select a region search operation type or a position search operation type by operating the region control 152 or the position control 153. The room card 151 also displays the wall name, and the user can determine his or her own position according to the wall name, and indicate the wall where the target smart home device to be searched is anchored.

[0209] For example, as shown in interface 1501, the control device detects that the user long presses the position control 153 and moves to the TV wall, and can display the following Fig.15 Interface 1502 shown in (b). In addition, the control device can determine that the search operation type indicated by the user is a location search operation type, and the smart home device being searched is a smart home device anchored to the TV wall. Then, the control device can determine the information of the wall pointed to by the current user (CurrentPointingWall), such as the wall data of the TV wall. In addition, the control device can determine the horizontal pointing angle (CurrnetAngleX) and the vertical pointing angle (CurrnetAngleY) of the position indicated by the current user according to the stop position of the position control 153 on the TV wall. Through CurrnetAngleX and CurrnetAngleY, the control device can match the target smart home device with the corresponding pointing angle. For example, as shown in interface 1502, the stop position of the position control 153 on the TV wall is a Y-axis 0 degree angle pointing to the TV wall. The control device determines that the target smart home device is a smart screen based on the pointing angle.

[0210] In some examples, the control device may determine CurrentPointingWall based on the user's UI operation. Alternatively, the control device may determine CurrentPointingWall based on sensor data and position data. For example, the preset position data includes that the TV wall is located at a position 15° north. The control device determines the direction pointed by the control device as the position 15° north based on the sensor data, and may determine that the wall pointed by the current user is the TV wall. The control device may determine CurrnetAngleX and CurrnetAngleY ​​based on the user's UI operation or sensor data.

[0211] In some examples, after determining the target smart home device, the control device may display information of the target smart home device so that the user can determine whether the currently found target smart home device is the smart home device the user wants to find. Optionally, the information of the smart home device displayed by the control device is, for example, one or more of the smart home device icon, name, nickname, etc.

[0212] For example, Fig.15 In the interface 1502 shown in (b), after the control device determines that the target smart home device is a smart screen, it displays a smart screen icon 154 near the position control 153. Therefore, the user can intuitively determine whether the search result of the smart home device is correct based on the smart screen icon 154. In some examples, if the search result is correct, in response to the user's operation on the smart screen icon 154, the control device may display a smart screen control interface (or a smart screen control card) so that the user can directly control the smart screen on the smart screen control interface, such as turning on the smart screen, turning up the volume of the smart screen, changing channels, etc. If the search result is wrong, the control device can also continue to search for the target smart home device according to the user's operation.

[0213] In some examples, the user may also select the search operation type and the wall surface in other ways, such as indicating the search operation type and the wall surface through UI operation and / or pointing operation.

[0214] For example, the user selects the search operation type and the wall through the UI operation. Fig.16 In the interface 1601 shown in (a), the control device detects the user's operation on the position control 161 (such as single click, double click, etc.) during the display of the device search interface, and determines that the search operation type indicated by the user is a position search operation type. Fig.16 In the interface 1602 shown in (b), the control device detects the user's operation on the TV wall and can determine that the smart home device that the user indicates to find has been anchored to the TV wall. In addition, the control device can determine the target smart home device corresponding to the operation position 162 according to the user's operation position 162 on the TV wall. If the operation position 162 is at a 45-degree angle on the Y axis pointing to the TV wall, the control device determines that the target smart home device is a smart spotlight based on the pointing angle. Afterwards, the control device can display the smart spotlight icon 163. It should be understood that the user can also select the wall first and then select the search operation type.

[0215] For another example, the user selects the search operation type and the wall through the pointing operation. For example, the user holds the control device and points to the smart home device to be found. The control device determines the wall data of the wall pointed by the user and the device data corresponding to the wall data according to the sensor data and the spatial data model. After that, the control device determines the smart home device pointed by the user according to the pointing angle, and determines the smart home device as the target smart home device. Optionally, the control device can display the information of the target smart home device so that the user can confirm whether the current search result is correct. Optionally, after the control device determines that the user does not move at a certain position on the wall or the moving range is within a preset range, it can determine that the search operation type indicated by the user is a position search operation type, and the smart home device at the corresponding position can be determined as the target smart home device. Alternatively, the control device determines that the user moves beyond the preset range after pointing to the wall, and can determine that the search operation type indicated by the user is an area search operation type, and can determine the search area according to the actual moving range, so as to determine the smart home devices in the area as the target smart home devices.

[0216] For another example, the user selects the search operation type and the wall through UI operation and pointing operation.

[0217] For example, Fig.17 In the interface 1701 shown in (a), the control device detects the user's long press of the position control 171 during the display of the device search interface, and determines that the search operation type is a position search operation type. Fig.17 In the interface 1702 shown in (b), the control device detects that the user long presses the position control 171 to move to the position of the TV wall 172, and determines that the wall anchored by the device searched by the user is the TV wall. The control device can obtain the wall data and device data of the TV wall. Fig.17 As shown in (c), the user holds the control device and moves the control device to find the desired smart home device. The control device determines the pointing angle corresponding to the TV wall according to the user's pointing operation, thereby determining the target smart home device. Optionally, as shown in interface 1703, the control device displays the information of the smart home device.

[0218] For example, Fig.18 In the interface 1801 shown in (a), the control device detects the user clicking on the TV wall 181 during the display of the device search interface, and determines that the wall anchored by the device searched by the user is the TV wall. Fig.18As shown in (b), the control device determines that the target smart home device that the user is looking for is a target smart home device at a certain location, or determines that the smart home device that the user is looking for is a smart home device within a certain area according to the user's pointing operation. Alternatively, the control device determines that the default search operation type is a location search operation type or an area search operation type, and determines the target smart home device according to the user's pointing operation. Optionally, as shown in interface 1802, the control device displays information about the smart home device.

[0219] In this way, the user can find the target smart home device through simple UI operations and / or pointing operations, effectively reducing the difficulty for the user to find the device.

[0220] In some embodiments, the target smart home device determined by the control device may not be the smart home device required by the user, or the user cannot directly determine the location of the required smart home device on the wall. Therefore, the control device can switch to display the information of the corresponding smart home device according to the user's UI operation and / or pointing operation, so that the user can quickly find the required smart home device.

[0221] For example, Fig.19 In the interface 1901 shown in (a), the control device detects the user's operation of selecting the position control 191 and moving it to the TV wall 192 during the process of displaying the device search interface, and can obtain the wall data of the TV wall and the device data of the smart home device anchored to the TV wall. Fig.19 In the interface 1902 shown in (b), the control device determines the target smart home device according to the final stop position of the position control 191 on the wall 192, and displays the icon 193 of the target smart home device. If the user confirms that the smart home device is not the smart home device he needs, the user can find the required smart home device by moving the position control horizontally on the TV wall. Fig.19 In the interface 1903 shown in (c), the control device detects that the user moves the position control 191 on the TV wall 192 in the direction indicated by the arrow, and determines that the user instructs to find a new smart home device. Fig.19 In the interface 1904 shown in (d), the control device determines that the smart home device corresponding to the position where the position control 191 finally stops on the TV wall 192 is the new target smart home device indicated by the user, and displays the icon 195 of the new target smart home device for user confirmation.

[0222] As another example, Fig. 20 In the interface 2001 shown in (a), the control device determines that the user indicates to search for a new target smart home device according to the user's operation of moving the position control 202 horizontally to the right along the direction indicated by the arrow on the TV wall 201. Fig. 20In the interface 2002 shown in (b), there are multiple smart home devices anchored on the TV wall. During the movement of the position control 201, the control device can change the display of the icons of the smart home devices corresponding to different moving positions according to the change of the moving position. As shown in interface 2002, in response to the user's operation of moving the position control 202 horizontally to the right, the control device matches the corresponding moving position according to the anchor position of the smart home device on the TV wall during the movement, and displays the smart home device icon A corresponding to the moving position 1, the smart home device icon B corresponding to the moving position 2, and the smart home device icon C corresponding to the moving position 3. In this way, during the movement of the position control, the user can also determine whether the required smart home device has been found according to the changes in the icons. It should be understood that after the control device determines that the user has moved the position control away from the current moving position, it will no longer display the smart home device icon corresponding to the current moving position. Fig. 20 As shown in (c), the user can repeatedly move the position control 202 horizontally to confirm the final target smart home device. As shown in interface 2003, the control device detects that the user moves the position control 202 horizontally to the right and then moves the position control 202 horizontally to the left. Similarly, the smart home device icon corresponding to the moving position can be displayed according to the change of the moving position.

[0223] For example, Fig.21 In the interface 2101 shown in (a), the control device detects the user's selection of the position control 211 and the operation of moving to the TV wall 212 during the display of the device search interface, and can obtain the wall data of the TV wall and the device data of the smart home device anchored to the TV wall. Fig.21 In the interface 2102 shown in (b), the control device determines that the target smart home device is a smart screen according to the final stop position of the position control 211 on the wall 212, and the pointing angle is Y=0°, so the control device can display the smart screen icon 213. Fig.12 In the scenario shown, the smart home devices anchored to the TV wall include a smart screen and a smart spotlight, wherein the Y-axis angles of the pointing angles of the smart screen and one smart spotlight are both 0° (i.e., Y=0°). In some examples, when the control device determines that the pointing angle is 0° and the Y-axis angle is 0°, it prioritizes determining that the target smart home device is the smart screen located in the center of the TV wall. However, the smart screen may not be the smart home device that the user needs to find. In addition, since the current device search interface is displayed in 2D, users cannot directly operate the UI on the current interface to indicate smart home devices with the same vertical angle. Therefore, if Fig.21As shown in (c), the user can change the vertical angle by shaking the control device vertically up and down while keeping the long press position control 211 on the TV wall 212 unchanged, thereby switching the smart home devices in the vertical direction. As shown in interface 2103, the control device displays the corresponding spotlight icon 214 according to the change of the pointing angle in the vertical direction, so as to facilitate the user to confirm whether the required smart home device is found. Optionally, when the user shakes the control device in the vertical direction, the control device changes the display of the corresponding smart home device icon according to the change of the pointing angle.

[0224] In this way, if the user only remembers the wall where the required smart home device is located, but forgets or remembers the specific location of the smart home device on the wall incorrectly, the user can also quickly find the required smart home device through UI operation and / or pointing operation, thereby further reducing the user's memory difficulty and improving the user's experience.

[0225] In some embodiments, to simplify user operations, users can also directly search for multiple smart home devices included in a region through the region search operation type, thereby achieving group control of smart home devices. Fig.12 In the scene shown, there are multiple smart spotlights on the top of the TV wall. If the user wants to control multiple smart spotlights, if the user searches for the operation type by location, it is necessary to perform multiple operations to find the multiple smart spotlights to be controlled one by one, which is cumbersome. Therefore, the user can search for the operation type by area to find multiple smart spotlights at one time.

[0226] The implementation process of the regional operation type is introduced as follows.

[0227] In some examples, the control device first determines the device data of multiple smart home devices corresponding to the wall indicated by the user according to the user's UI operation. Then, the control device determines the target smart home device selected by the user according to the user's pointing operation.

[0228] For example, Fig. 22 In the interface 2201 shown in (a), the control device detects the user's operation of selecting the area control 221 and moving it toward the TV wall 222 during the process of displaying the device search interface, and can obtain the wall data and device data of the TV wall. Fig. 22 In the interface 2202 shown in (b), the control device can display icons of multiple smart home devices anchored to the TV wall, such as icons of multiple smart spotlights. Fig. 22As shown in (b), the user can select the smart spotlight to be controlled by shaking the control device horizontally. As shown in reference numeral 223, the control device determines the change of the pointing angle according to the pointing operation of the user, and then determines the multiple target smart spotlights selected by the user, and distinguishes and displays the smart spotlight icons selected by the user and the smart spotlight icons not selected by the user in a preset manner, so as to facilitate the user to confirm whether the device search is completed.

[0229] In some examples, the control device first determines the device data of multiple smart home devices corresponding to the wall indicated by the user according to the user's UI operation. Optionally, the control device can determine the device type of these smart home devices based on the device data, group these smart home devices according to the device type, and display the corresponding group control controls. In this way, the user can directly find multiple smart home devices in groups after selecting the wall, reducing subsequent pointing operations and further reducing the difficulty of user operation. Optionally, the control device can classify smart home devices of different device types according to the smart home device control subsystem. For example, smart spotlights belong to the lighting subsystem.

[0230] Optionally, the control subsystem is a grouping of smart home devices pre-configured by a technician according to device types, and the control device can determine the control subsystem to which the smart home device belongs according to the device type of the smart home device. For example, the control subsystem includes a lighting subsystem, a temperature control subsystem, and the like.

[0231] For example, Fig.23 In the interface 2301 shown in (a), the control device detects the user's operation of selecting the area control 231 and moving it to the TV wall 232 during the process of displaying the device search interface, and can obtain the wall data and device data of the TV wall. Fig.23 In the interface 2302 shown in (b), the control device determines that the multiple smart spotlights corresponding to the current wall belong to the lighting subsystem, so the control device can display the lighting control 233. Among them, through the lighting control 233, the user can directly control the multiple smart spotlights located on the top of the TV wall, such as turning on, off, and adjusting the brightness.

[0232] In some examples, users can customize smart home device scenes and add smart home devices that need to be uniformly controlled in different scenes. For example, users create audio-visual entertainment scenes, including smart screens and smart speakers. Therefore, after the control device determines the device data of multiple smart home devices corresponding to the wall indicated by the user according to the user's UI operation, it can determine the control subsystem and / or scene corresponding to the current wall to display the corresponding group control controls. In this way, the personalized needs of users can be better met.

[0233] For example, Fig.24In the interface 2401 shown in (a), the control device detects the user's operation of selecting the area control 241 and moving it to the TV wall 242 during the process of displaying the device search interface, and can obtain the wall data and device data of the TV wall. Fig.24 In the interface 2402 shown in (b), the control device determines that the multiple smart spotlights corresponding to the current wall belong to the lighting subsystem, and the smart screen and smart speaker belong to the audio-visual entertainment scene. Therefore, the control device can display the lighting control 243 and the audio-visual entertainment control 244. In this way, through the lighting control 243, the user can directly control the multiple smart spotlights located at the top of the TV wall, such as turning on, off, brightness adjustment, etc. Through the audio-visual entertainment control 244, the user can directly control the smart screen and smart speaker anchored to the TV wall, such as turning on, off, volume adjustment, etc.

[0234] Optionally, as shown in reference numeral 245, the control device also displays icons of corresponding smart home devices, thereby facilitating the user to determine the smart home device to be controlled corresponding to the control subsystem control or scene control to avoid misoperation.

[0235] In some examples, although the multiple smart home devices corresponding to the wall may belong to the control subsystem or scene, the user may need to control some of the smart home devices. Therefore, the control device can also select multiple smart home devices to be controlled according to the user's pointing operation or UI operation. Optionally, after determining the multiple smart home devices that the user wants to control, the control device can also confirm whether these smart home devices can also be group controlled to display the corresponding group control controls for user convenience. In this way, while realizing group control, the personalized needs of users are met.

[0236] For example, Fig.25 In the interface 2501 shown in (a), the control device detects the user's operation of selecting the area control 251 and moving it to the TV wall 252 during the process of displaying the device search interface, and can obtain the wall data and device data of the TV wall. After that, the control device displays the icons of multiple smart home devices anchored on the wall. The user can select some or all of the smart home devices by shaking the control device. Fig.25 In the interface 2502 shown in (b), the control device determines that the user selects 4 smart spotlights from all the smart spotlights according to the user's pointing operation and the change of the pointing angle. The control device determines that the device types of the multiple smart spotlights selected by the user are the same and all belong to the lighting subsystem. Then, as shown in the interface 2502, the control device can display the lighting control 253 to facilitate the user to group control the selected 4 smart spotlights.

[0237] As another example, Fig.26In the interface 2601 shown in (a), the control device detects the user's operation of selecting the area control 261 and moving it to the TV wall 262 during the process of displaying the device search interface, and can obtain the wall data and device data of the TV wall. Afterwards, the control device displays the icons of multiple smart home devices anchored on the wall. Fig.26 In the interface 2602 shown in (b), the control device detects the user's movement operation on the area control 261, and can determine the multiple smart home devices selected by the user based on the starting position and the ending position of the movement. For example, the control device determines that the user selects 4 smart spotlights from all smart spotlights based on the user's UI operation. The control device determines that the device types of the multiple smart spotlights selected by the user are the same and all belong to the lighting subsystem. Then, as shown in the interface 2602, the control device can display the lighting control 263 to facilitate the user to group control the selected 4 smart spotlights.

[0238] In some embodiments, during the device search process of the above-mentioned area device search type, the control device can determine the change of the pointing angle based on the sensor data, and then determine one or more smart home devices included in the range of the pointing angle change.

[0239] For example, the control device determines that the user is shaking the control device horizontally based on the sensor data, and determines the range of the user's horizontal pointing angle based on the horizontal starting angle (StartAngleX) and the horizontal ending angle (EndAngleX). Afterwards, the control device can determine one or more smart home devices within the corresponding pointing angle range based on the device data in the spatial data model.

[0240] For another example, the control device determines that the user is shaking the control device vertically based on the sensor data, and determines the range of the user's vertical pointing angle based on the vertical direction starting angle (StartAngleY) and the vertical direction ending angle (EndAngleY). Afterwards, the control device can determine one or more smart home devices within the corresponding pointing angle range based on the device data in the spatial data model.

[0241] In this way, the control device helps the user to quickly find smart home devices according to the user's UI operation and / or pointing operation, effectively reducing the user's operation difficulty and low learning cost.

[0242] In some embodiments, the device search interface displayed by the control device is a 2D interface, such as Fig.13In the interface 1301 shown, the control device displays the device search interface corresponding to the living room, which only includes the TV wall, the background wall, and the two walls between the TV wall and the background wall. However, in actual application, the room may include more walls (such as the roof, etc.). In this case, the control device can switch the room angle to display the device search interface according to the user's pointing operation, gesture operation, voice operation, etc., so that the user can find smart home devices on other walls.

[0243] In some embodiments, the above description takes the case where the control device and the smart home device that the user needs to find are in the same space as an example to introduce the device search process. In some examples, the user can also remotely search for smart home devices through the control device.

[0244] Exemplarily, the control device can determine the house where the user indicates to search for the device according to the user's UI operation and / or voice operation, and then obtain the corresponding spatial data model, and search for the target smart home device based on the spatial data model. For example, the control device can determine the room where the user indicates to search for the device according to the user's UI operation and / or voice operation, and display the device search interface corresponding to the room. In the remote control process, the control device can assume that the user is at the reference anchor point position by default, and the user's position in the room is the default position. Afterwards, the control device determines the wall indicated by the user according to the user's UI operation or voice operation, and determines the target smart home device through the above-mentioned location search operation type or area search operation type, combined with the user's UI operation or voice operation, wherein the UI operation can be, for example, an operation of selecting a smart home device in the interface or an operation of selecting a smart home device by a handheld device; the voice operation can be, for example, an operation of the user describing the name of a selected smart home device by voice, so as to realize multimodal control interaction and improve the convenience of user operation.

[0245] Thus, compared with the existing device search method for pointing control functions, the device search method provided in the embodiment of the present application can realize remote device search by combining the spatial data model, without being limited to the scenario where the control device and the target smart home device are in the same space. It enriches the application scenarios and meets the remote control needs of users.

[0246] In some embodiments, the control device may display a function control on the device search interface, through which the user may directly control the corresponding smart home device to perform a specific function.

[0247] Optionally, the search operation type also includes, for example, a function search operation type. Optionally, the specific function is, for example, open, close, open and adjust to preset parameters, and the like.

[0248] For example, Fig. 27In the interface 2701 shown in (a), in response to the user starting the device search function, the control device determines the house (such as the house where the control device is currently located) and the room (such as the room where the control device is currently located) that the user is searching for according to the spatial data model, and automatically obtains the current house data and room data, and the relative position of the user in the room to display the current room card 271. As shown in the room card 271, the room where the current user is located is the living room. Among them, an on control 272 and an off control 273 are displayed on the room card 271. The user can select the function search operation type by operating the on control 272 or the off control 273. For example, the control device detects that the user long presses the on control 272 and moves it to the TV wall. The control device can determine that the search operation type indicated by the user is a function search operation type, and the smart home device being searched is a smart home device anchored to the TV wall.

[0249] For example, the control device determines all the smart home devices anchored to the TV wall and sends a start instruction to these smart home devices to turn on these smart home devices. In this way, the user can achieve batch control of smart home devices by simply dragging the function control, without having to search for each smart home device one by one and then control each smart home device one by one.

[0250] For another example, the control device determines all the smart home devices anchored to the TV wall, and determines that these smart home devices belong to the same type of smart home devices, such as these smart home devices are all smart spotlights. Then, the control device sends a start instruction to these smart spotlights, and carries preset parameters in the start instruction, such as brightness value, color temperature, etc., so that these smart spotlights are turned on according to the preset parameters. Optionally, the preset parameters can be parameters set by the user, parameters pre-configured in the control device, commonly used parameters, parameters used last time, etc. In this way, the user can achieve batch control of smart home devices according to user needs by simply dragging the function control, thereby improving the user experience.

[0251] For another example, the control device stores multiple groups of parameters corresponding to the control, and different groups of parameters correspond to different device types. In this way, when the control device determines that the smart home devices anchored to the TV wall are of multiple types, it can send a start instruction with corresponding parameters to the smart home devices of the corresponding types according to the preset multiple groups of parameters, and / or send a start instruction to the smart home devices of the device type that are not configured with parameters, so as to realize flexible control of different devices. Fig.12In the scenario shown, multiple smart spotlights, smart screens, and smart speakers are anchored on the TV wall. Parameter 1 corresponding to the smart spotlight and parameter 2 corresponding to the smart speaker are configured in the control device. Then, in response to the user's long press of the open control 272 and moving it toward the TV wall, the control device may send a start instruction carrying parameter 1 to the smart spotlight, a start instruction carrying parameter 2 to the smart speaker, and a start instruction to the smart screen, thereby turning on the smart spotlight, smart speaker, and smart screen, and making the smart spotlight run according to parameter 1, and making the smart speaker run according to parameter 2. In this way, the user can realize batch control of smart home devices of different device types according to user needs by simply dragging the function control, thereby improving the user experience.

[0252] Optionally, after controlling the smart home device, the control device may display Fig. 27 The interface 2702 shown in (b) is used to facilitate the user to determine the execution status of the control operation.

[0253] As another example, in response to the user long pressing the off control 273 and moving it toward the TV wall, the control device can control the smart home device anchored to the TV wall to turn off.

[0254] In some examples, the control device may display at least one of a function control, a position control, and an area control, thereby implementing at least one search operation type among a function search operation type, a position search operation type, and an area search operation type.

[0255] The above description uses the example of a user searching for a smart home device as an example to illustrate the search process. Based on the same principle, the device search method provided in the embodiment of the present application can also be used to search for any item in a space such as a non-intelligent electronic device or a non-electronic device. The following description is a detailed description of the implementation of the control device responding to a user operation to search for any item in a space.

[0256] In some embodiments, the technician updates the spatial data model according to the changes of the items in the house. In other examples, the user updates the spatial data model according to the changes of the items in the house during use.

[0257] For example, when a technician or user determines that a new item needs to be added to a room, the handheld control device is located at the reference anchor point position as shown in Table 2 above. Afterwards, the technician or user selects the information of the item to be entered and points the control device to the newly added item. Among them, the control device obtains spatial information according to the user's pointing operation. In response to the user's instruction to enter the operation (such as clicking the entry control operation, etc.), the control device can establish an association between the item information and the spatial information, complete the entry of the item information, so that the item can be controlled subsequently. In some examples, the item information includes, for example, the item identification, the name of the item, etc. The spatial information includes, for example, the room data of the item determined by the control device in combination with the spatial data model, the wall data, the horizontal angle of the item relative to the reference anchor point, the vertical angle of the item relative to the reference anchor point, and other information.

[0258] In this way, the subsequent control device can find the required items in response to the user's operation on the search interface.

[0259] For example, the control device displays a search interface in response to a user operation, and the search interface corresponds to the room the control device is currently in. Afterwards, in response to a user touch operation on the search interface, or a user pointing operation by rotating the control device, the control device can determine the item that the user indicates to search for, such as a bookshelf.

[0260] In some examples, the control device may also anchor an object to a non-wall location in response to a user operation, such as anchoring an object to another object. For example, in response to a user operation, the control device anchors a desk lamp to a certain layer of a bookshelf. Subsequently, the control device may also determine the location of the object indicated by the user in response to the user's pointing operation.

[0261] In this way, users can search for any object in the space, such as smart home devices, non-smart electronic devices, non-electronic devices, etc., through simple touch operations or pointing operations, meeting users' item search needs and further improving users' usage experience.

[0262] Fig.28 This is a flow chart of a device search method provided in an embodiment of the present application. It should be noted that this method does not Fig.28 The specific order described below is a limitation, and it should be understood that in other embodiments, the order of some steps in the method can be interchanged according to actual needs, or some steps can be omitted or deleted. The method includes the following steps:

[0263] S2801. In response to a first operation of a user, a first electronic device displays a device search page, where the device search page includes a display element associated with a wall of a room.

[0264] The display elements associated with the walls of the room include, for example, the name of the wall, a schematic mark of the wall, and the like.

[0265] Optionally, the device search page is, for example, any one or more of the following: a device search interface displayed in full screen, a device search card displayed on the desktop, the negative one screen, an application, etc., a device search area displayed in an application, etc.

[0266] For example, Fig.13 As shown, in response to the user's first operation, the first electronic device locates the current room as the living room, and can display a device search page, which is used to search for devices in the living room. For example, the device search page displays the TV wall and background wall included in the living room, as well as line segments used to represent the wall surface.

[0267] In some embodiments, the first electronic device displays a device search page corresponding to the space where the first electronic device is currently located based on the spatial data model. For example, in response to the user's first operation, the first electronic device obtains a spatial data model, which includes house data, room data, wall data, and device data. Afterwards, the first electronic device displays a device search page based on the spatial data model and the positioning data of the first electronic device.

[0268] Optionally, the house data is used to locate the house where the first target device is located, the room data is used to locate the room where the first target device is located, the wall data is used to locate the wall where the first target device is located, and the device data is used to locate the position of the first target device on the wall.

[0269] In this way, the first electronic device displays a corresponding device search page through a preset space data model in combination with the space where the first electronic device is currently located, thereby realizing device search in the space corresponding to the device search page.

[0270] S2802: In response to the user's second operation on the device search page and / or the user's first pointing operation, the first electronic device determines the first target device that the user is searching for.

[0271] The first target device is an object associated with the wall orientation. The first pointing operation is an operation in which the user rotates the first electronic device to point to the first target device.

[0272] For example, the first target device is in the direction of the TV wall, anchored to the TV wall, installed on the TV wall, etc.

[0273] In some embodiments, in response to the user's second operation on the device search page and / or the user's first pointing operation, the first electronic device determines that the wall where the device searched by the user is located is the first wall.

[0274] For example, Fig.15 As shown in (a), during the process of displaying the device search page, the first electronic device detects a second operation of the user moving the position control 153 to a display element associated with the TV wall (such as a TV wall segment), and determines that the wall where the device the user is searching for is located is the TV wall.

[0275] As another example, Fig.15 As shown in (a), during the process of displaying the device search page, the first electronic device detects a first pointing operation of the user turning the first electronic device to point to the TV wall, and determines that the wall where the device searched by the user is located is the TV wall.

[0276] In this way, the first electronic device first determines the wall according to the user operation, and then determines that the device the user instructs to find is one or several devices on the wall.

[0277] In some embodiments, in response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the first target device that the user is searching for includes: determining the type of the user's search operation in response to the second operation of the user on the device search page and / or the first pointing operation of the user. In the case where the search operation type is a location search operation type, an identifier of a device to be searched is displayed. In the case where the search operation type is an area search operation type, multiple identifiers of devices to be searched are displayed. The first target device is one or more devices among the devices to be searched.

[0278] For example, Fig.15 In the interface 1501 shown in (a), the first electronic device detects a second operation of the user moving the position control 153 to a display element associated with the TV wall (such as a TV wall line segment) during the process of displaying the device search page, and determines a device to be found that the user is looking for according to the stop position of the second operation, and displays the following Fig.15 (b) shows an interface 1502. As shown in the interface 1502, the first electronic device displays the icon 154 of the device to be found.

[0279] For example, Fig. 22 In the interface 2201 shown in (a), the first electronic device detects a second operation of the user selecting the area control 221 and moving it to the TV wall 222 during the process of searching for a page on the display device. Fig. 22 In the interface 2202 shown in (b), the first electronic device can display multiple icons of devices to be found anchored to the TV wall, such as icons of multiple smart spotlights.

[0280] In some examples, the first electronic device detects an operation in which a user moves a location control displayed on a device search page toward a first display element associated with a first wall, and determines that the search operation type is a location search operation type. Alternatively, the first electronic device detects an operation in which a user moves an area control displayed on a device search page toward a first display element, and determines that the search operation type is an area search operation type. Alternatively, the first electronic device detects an operation in which a user points the first electronic device at a first wall and does not move at a certain position on the first wall or moves within a range smaller than a preset range, and determines that the search operation type is a location search operation type. Alternatively, the first electronic device detects an operation in which a user points the first electronic device at a first wall and moves within a range larger than a preset range on the first wall, and determines that the search operation type is an area search operation type.

[0281] In this way, the first electronic device can determine whether the user instructs to search for one target device or multiple target devices based on the user's touch operation on the device search page or the user's pointing operation of rotating the first electronic device.

[0282] In other embodiments, in response to the user's second operation on the device search page and / or the user's first pointing operation, determining the first target device the user is searching for includes: in response to the user's second operation on the device search page and / or the user's first pointing operation, determining that the user's search operation type is a function search operation type.

[0283] For example, Fig. 27 In the interface 2701 shown in (a), the first electronic device detects the second operation of the user moving the open control 272 to the display element associated with the TV wall (such as the TV wall segment) during the display device search page, and determines that the user's search operation type is a function search operation type.

[0284] It should be understood that the location search operation type, area search operation type, and function search operation type are merely names used to distinguish operation types. There may also be other naming methods, and the embodiments of the present application do not limit this.

[0285] In some embodiments, the first electronic device detects a fifth operation of the user moving the position control displayed on the device search page to the first display element associated with the first wall, and displays the identifier of the first device corresponding to the stop position of the position control on the first display element. Afterwards, the first electronic device detects a sixth operation of the user moving the position control on the first display element, and displays the identifier of the second device corresponding to the final stop position, or switches to display the identifiers of multiple devices corresponding to the movement trajectory during the movement. Afterwards, the first electronic device determines the second device corresponding to the final stop position as the first target device.

[0286] For example, Fig.19 In the interface 1901 shown in (a), the first electronic device detects the user's selection of the position control 191 and the movement toward the TV wall 192 during the display of the device search page, and can obtain the wall data of the TV wall and the device data of the smart home device anchored to the TV wall. Fig.19 In the interface 1902 shown in (b), the first electronic device determines the first device according to the final stop position of the position control 191 on the wall 192, and displays the icon 193 of the first device. If the user confirms that the first device is not the smart home device he needs, the user can find the required device by moving the position control 191 horizontally on the TV wall. Fig.19 In the interface 1903 shown in (c), the first electronic device detects that the user moves the position control 191 in the direction indicated by the arrow on the TV wall 192, and determines that the user instructs to find a new device. Fig.19 In the interface 1904 shown in (d), the first electronic device determines that the second device corresponding to the position where the position control 191 finally stops on the TV wall 192 is the first target device indicated by the user, and displays the icon 195 of the first target device for user confirmation.

[0287] Optionally, when the user moves the position control 191, the first electronic device may switch to display the identifiers of multiple devices corresponding to the movement track, or only display the icon of the device corresponding to the final stop position.

[0288] In this way, if the user only remembers the wall where the required device is located, but forgets or remembers the specific location of the device on the wall incorrectly, the user can also quickly find the required target device through touch operation and / or pointing operation, thereby further reducing the user's memory difficulty and improving the user's usage experience.

[0289] In some embodiments, the first electronic device detects a seventh operation in which the user moves the area control displayed on the device search page to the first display element associated with the first wall, and displays the identifiers of multiple devices to be found corresponding to the first wall. In response to the user's first pointing operation, the first electronic device determines the area range of the first pointing operation. The first electronic device matches the area range and multiple devices to be found, and determines some or all of the multiple devices to be found as the first target device.

[0290] For example, Fig.25 In the interface 2501 shown in (a), the first electronic device detects the user's operation of selecting the area control 251 and moving it to the TV wall 252 during the process of displaying the device search page, and displays the icons of multiple devices anchored on the wall. The user can select some or all of the devices by shaking the first electronic device. Fig.25In the interface 2502 shown in (b), the first electronic device determines that the user selects 4 smart spotlights from all the smart spotlights according to the pointing operation of the user and the change of the pointing angle, that is, determines these 4 smart spotlights as the first target devices.

[0291] In some examples, the first electronic device detects an eighth operation of the user moving the area control on the first display element, determines the starting position and the ending position of the eighth operation, and determines multiple first target devices covered by the starting position and the ending position among multiple devices to be found.

[0292] As another example, Fig.26 In the interface 2601 shown in (a), the first electronic device detects the user's operation of selecting the area control 261 and moving it to the TV wall 262 during the process of displaying the device search page, and can display the icons of multiple devices anchored on the wall. Fig.26 In the interface 2602 shown in (b), the first electronic device detects the user's movement operation on the area control 261, and can determine the multiple smart home devices selected by the user according to the starting position and the ending position of the movement. For example, the first electronic device determines that the user selects 4 smart spotlights from all smart spotlights according to the user's eighth operation.

[0293] In this way, while achieving group control, the user's personalized and flexible control needs can be met.

[0294] In some embodiments, the first electronic device detects a ninth operation of the user moving the area control displayed on the device search page to the first display element associated with the first wall, and displays a subsystem identifier and / or a scene identifier. The plurality of third devices corresponding to the subsystem identifier are devices of the same subsystem among the plurality of devices to be found, the plurality of fourth devices corresponding to the scene identifier are devices in the same scene among the plurality of devices to be found, and the plurality of devices to be found are devices corresponding to the first wall.

[0295] In some examples, the first electronic device detects the user's tenth operation on the subsystem identifier and determines that the plurality of third devices are the first target devices. In some examples, the first electronic device detects the user's eleventh operation on the scene identifier and determines that the plurality of fourth devices are the first target devices.

[0296] For example, Fig.24 In the interface 2401 shown in (a), the first electronic device detects the operation of the user selecting the area control 241 and moving it to the TV wall 242 during the process of searching the page on the display device, such as Fig.24In the interface 2402 shown in (b), the first electronic device determines that the multiple smart spotlights corresponding to the current wall belong to the lighting subsystem, and the smart screen and smart speaker belong to the audio-visual entertainment scene. Therefore, the first electronic device can display the lighting control 243 and the audio-visual entertainment control 244. In this way, through the lighting control 243, the user can directly control the multiple smart spotlights located at the top of the TV wall, such as turning on, off, brightness adjustment, etc. Through the audio-visual entertainment control 244, the user can directly control the smart screen and smart speaker anchored to the TV wall, such as turning on, off, volume adjustment, etc.

[0297] In this way, after the first electronic device determines the device data of multiple smart home devices corresponding to the wall indicated by the user according to the user's touch operation or pointing operation, it can determine the control subsystem and / or scene corresponding to the current wall to display the corresponding group control controls to better meet the user's personalized needs.

[0298] In some embodiments, the first electronic device determines the first wall and / or the first target device according to a first pointing angle at which the user rotates the first electronic device to point toward the wall.

[0299] In some embodiments, the first electronic device determines a second pointing angle according to a stop position of the second operation on the first display element associated with the first wall surface, and the second pointing angle is used to determine the first target device.

[0300] Exemplarily, as shown in Table 4 above, the device data in the spatial data model includes the pointing angle from the reference anchor point to the smart home device. For example, angleX represents the horizontal angle of the smart home device relative to the reference anchor point, and angleY ​​represents the vertical angle of the smart home device relative to the reference anchor point. Then, in the subsequent device search process, the first electronic device can determine the target device that the user is looking for based on the current pointing angle.

[0301] In some examples, the room data includes a reference anchor point, and the device data includes a first azimuth angle of the device relative to the reference anchor point. Then, the first electronic device detects the first pointing operation and obtains a second azimuth angle corresponding to the first pointing operation. Thereafter, the first electronic device converts the second azimuth angle into the first azimuth angle to determine the first target device pointed to by the first azimuth angle.

[0302] Among them, the position of the user in the process of searching for the device is not necessarily the reference anchor point position in the room. Different users may have different heights, resulting in different heights of the position where the first electronic device is operated. This leads to a deviation between the operating position and the reference anchor point position, affecting the pointing angle. Therefore, in order to ensure the accuracy of the target device finally determined, the first electronic device can determine the device actually pointed to by the user through conversion calculation based on the currently determined position data and spatial data model.

[0303] S2803: The first electronic device presents or selects an identifier corresponding to the first target device.

[0304] In some embodiments, after determining the first target device, the first electronic device may display an identifier corresponding to the first target device, so that the user can determine whether the first target device is the device that the user needs to find.

[0305] Optionally, the identifier corresponding to the first target device is a controllable identifier or an uncontrollable identifier.

[0306] For example, the identifier of the first target device is a controllable control, and in response to the user's operation on the identifier corresponding to the first target device, the first electronic device sends instruction information to the first target device, and the instruction information is used to control the first target device. Alternatively, in response to the user's operation on the identifier corresponding to the first target device, the first electronic device displays a control page of the first target device, and the control page is used to control the first target device.

[0307] In other embodiments, after determining the first wall where the user indicates to find the device, the first electronic device may display the identifiers of all devices corresponding to the wall, and display the identifier of the first target device selected by the user among all devices as a selected state according to the user operation. The first target device is part or all of the devices corresponding to the first wall.

[0308] In this way, users only need simple UI operations and pointing operations to find and control the device.

[0309] Moreover, combined with the spatial data model, there is no need to pre-configure base stations in space, which effectively reduces hardware costs.

[0310] In addition, the first electronic device filters devices that are not in the current house through house data, filters devices that are not in the current room through room data, filters devices that are not anchored on the current wall through wall data, and matches the device closest to the current angle through device data, thereby determining the target device through four layers of filtering and improving the accuracy of device search and control.

[0311] In some embodiments, in response to a second operation of a user staying on a first display element associated with a first wall, the first electronic device presents an identification of a fourth target device. During the process of detecting the second operation of a user staying on the first display element, the first electronic device detects a first pointing operation of the user shaking the first electronic device up and down, and switches the presentation of the identification of the fourth target device to the presentation of the identification of the first target device.

[0312] For example, Fig.21In the interface 2101 shown in (a), the first electronic device detects the user's operation of selecting the position control 211 and moving it to the TV wall 212 during the process of searching the page on the display device. Fig.21 In the interface 2102 shown in (b), the first electronic device can determine that the first target device is a smart screen according to the final stop position of the position control 211 on the wall 212, and the pointing angle is Y=0°, so the first electronic device can display the smart screen icon 213. Fig.12 In the scenario shown, the smart home devices anchored to the TV wall include a smart screen and a smart spotlight, wherein the Y-axis angles of the pointing angles of the smart screen and one smart spotlight are both 0° (i.e., Y=0°). In some examples, when the first electronic device determines that the pointing angle is 0° and the Y-axis angle is 0°, it preferentially determines that the first target device is the smart screen located at the center of the TV wall. However, the smart screen may not be the device the user needs to find. In addition, since the current device search page is displayed in 2D, the user cannot directly indicate the device with the same vertical angle through UI operations on the current interface. Therefore, if Fig.21 As shown in (c), the user can change the vertical angle by shaking the first electronic device vertically up and down while keeping the long press position control 211 on the TV wall 212 unchanged, thereby switching the device in the vertical direction. As shown in interface 2103, the first electronic device displays the corresponding spotlight icon 214 according to the change of the pointing angle in the vertical direction, so as to facilitate the user to confirm whether the required device is found. Optionally, when the user shakes the first electronic device in the vertical direction, the first electronic device changes the icon of the corresponding device according to the change of the pointing angle.

[0313] In this way, the user can combine touch operation and pointing operation to flexibly find the desired target device.

[0314] In some embodiments, in response to the user's third operation on the device search page and / or the user's second pointing operation, the first electronic device determines that the wall where the device searched by the user is located is the second wall, and the second target device searched by the user on the second wall. The first electronic device presents or selects an identifier corresponding to the second target device.

[0315] The third operation and the second pointing operation include a single operation or a series of sub-operations. For example, after the user searches for the second wall through the third operation, the user searches for the second target device through the second pointing operation. Alternatively, the user searches for the second wall through the second pointing operation first, and then searches for the second target device through the third operation. Alternatively, the user searches for the second wall and the second target device through the second operation. Alternatively, the user searches for the second wall and the second target device through the second pointing operation.

[0316] In this way, based on the anchoring relationship between the wall and the device, the user can quickly find the desired target device through simple UI operations and / or pointing operations.

[0317] In some embodiments, in response to a fourth operation of the user on the device search page on the first wall and / or a third pointing operation of the user, a third target device of the user is determined. The first electronic device presents or selects an identifier corresponding to the third target device.

[0318] In this way, after finding the desired first target device, the user can continue to find other desired third target devices through UI operations and / or pointing operations.

[0319] In some embodiments, upon detecting the user's operation of adding a new device, the first electronic device instructs the user to point the first electronic device at the newly added device, and the first electronic device displays a prompt message that the newly added device has been successfully entered.

[0320] Optionally, the first electronic device instructs the user to stand at a position where the first electronic device is located at a reference anchor point, so that the first electronic device points from the reference anchor point to the newly added device.

[0321] In this way, the first electronic device can add a new searchable device according to the user operation to meet the user's demand for adding a new searchable device.

[0322] In some examples, the first electronic device may also execute the steps and functions executed by the control device in the above embodiments, thereby implementing the device search method provided in the above embodiments.

[0323] Combination of the above Figure 5-Figure 28 The device search method provided in the embodiment of the present application is described in detail. Fig.29 The first electronic device provided in an embodiment of the present application is described in detail.

[0324] In one possible design, Fig.29 This is a schematic diagram of the structure of a first electronic device provided in an embodiment of the present application. Fig.29 As shown, the first electronic device 2900 may include: a transceiver unit 2901, a processing unit 2902, and a display unit 2903. The first electronic device 2900 may be used to implement the functions of the control device involved in the above method embodiment.

[0325] Optionally, the transceiver unit 2901 is used to support the first electronic device 2900 to execute Fig.28 S2801 and S2802 in.

[0326] Optionally, the processing unit 2902 is used to support the first electronic device 2900 to execute Fig.28 S2802 in.

[0327] Optionally, the display unit 2903 is used to support the first electronic device 2900 to display interface content; and / or to support the first electronic device 2900 to execute Fig.28 S2801 and S2803 in.

[0328] Among them, the transceiver unit may include a receiving unit and a sending unit, which may be implemented by a transceiver or a transceiver-related circuit component, and may be a transceiver or a transceiver module. The operations and / or functions of each unit in the first electronic device 2900 are respectively to implement the corresponding process of the device search method described in the above method embodiment. All relevant contents of each step involved in the above method embodiment can be referred to the functional description of the corresponding functional unit, and for the sake of brevity, they will not be repeated here.

[0329] Optionally, Fig.29 The first electronic device 2900 shown may also include a storage unit ( Fig.29 (not shown), the storage unit stores a program or instruction. When the transceiver unit 2901, the processing unit 2902 and the display unit 2903 execute the program or instruction, Fig.29 The first electronic device 2900 shown can execute the device search method described in the above method embodiment.

[0330] Fig.29 The technical effects of the first electronic device 2900 shown can refer to the technical effects of the device search method described in the above method embodiment, and will not be repeated here.

[0331] In addition to being in the form of the first electronic device 2900, the technical solution provided in the present application may also be a functional unit or chip in the first electronic device, or a device used in conjunction with the first electronic device.

[0332] An embodiment of the present application also provides a chip system, including: a processor, the processor is coupled to a memory, the memory is used to store programs or instructions, when the program or instructions are executed by the processor, the chip system implements the method in any of the above method embodiments.

[0333] Optionally, the processor in the chip system may be one or more. The processor may be implemented by hardware or by software. When implemented by hardware, the processor may be a logic circuit, an integrated circuit, etc. When implemented by software, the processor may be a general-purpose processor implemented by reading software code stored in a memory.

[0334] Optionally, the memory in the chip system may also be one or more. The memory may be integrated with the processor or may be separately arranged with the processor, which is not limited in the embodiments of the present application. Exemplarily, the memory may be a non-transient processor, such as a read-only memory ROM, which may be integrated with the processor on the same chip or may be arranged on different chips respectively. The embodiments of the present application do not specifically limit the type of memory and the arrangement of the memory and the processor.

[0335] Exemplarily, the chip system can be a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a system on chip (SoC), a central processor unit (CPU), a network processor (NP), a digital signal processor (DSP), a microcontroller unit (MCU), a programmable logic device (PLD) or other integrated chips.

[0336] It should be understood that each step in the above method embodiment can be completed by an integrated logic circuit of hardware in a processor or by instructions in the form of software. The method steps disclosed in the embodiments of the present application can be directly embodied as being executed by a hardware processor, or by a combination of hardware and software modules in a processor.

[0337] An embodiment of the present application also provides a computer-readable storage medium, in which a computer program is stored. When the computer program is run on a computer, the computer executes the above-mentioned related steps to implement the device search method in the above-mentioned embodiment.

[0338] The embodiment of the present application also provides a computer program product. When the computer program product is run on a computer, the computer is caused to execute the above-mentioned related steps to implement the device search method in the above-mentioned embodiment.

[0339] In addition, an embodiment of the present application further provides a device. The device may be a component or a module, and the device may include one or more processors and a memory connected to each other. The memory is used to store a computer program. When the computer program is executed by one or more processors, the device performs the device search method in the above-mentioned method embodiments.

[0340] Among them, the device, computer-readable storage medium, computer program product or chip provided in the embodiments of the present application are used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0341] The steps of the method or algorithm described in conjunction with the disclosure of the embodiments of the present application can be implemented in hardware or by executing software instructions by a processor. The software instructions can be composed of corresponding software modules, and the software modules can be stored in random access memory (RAM), flash memory, read only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs or any other form of storage medium known in the art. An exemplary storage medium is coupled to a processor so that the processor can read information from the storage medium and can write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an application specific integrated circuit (ASIC).

[0342] Through the description of the above implementation methods, those skilled in the art can clearly understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed; that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. The specific working process of the system, device and unit described above can refer to the corresponding process in the aforementioned method embodiment, and will not be repeated here.

[0343] In the several embodiments provided in the present application, it should be understood that the disclosed method can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the modules or units is only a logical function division, and there may be other division methods in actual implementation; for example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of modules or units, which can be electrical, mechanical or other forms.

[0344] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0345] Computer-readable storage media include, but are not limited to, any of the following: a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, and other media that can store program codes.

[0346] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A device search method, characterized in that: Applied to a first electronic device, the method includes: In response to a first operation of the user, displaying a device search page, wherein the device search page includes a display element associated with a wall of the room; In response to a second operation of the user on the device search page and / or a first pointing operation of the user, determining a first target device searched by the user, wherein the first target device is an object associated with the wall orientation, and the first pointing operation is an operation of the user rotating the first electronic device to point to the first target device; An identifier corresponding to the first target device is presented or selected.

2. The method according to claim 1, characterized in that In response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the first target device searched by the user includes: In response to a second operation of the user on the device search page and / or a first pointing operation of the user, it is determined that the wall where the device searched by the user is located is the first wall.

3. The method according to claim 1 or 2, characterized in that: The method further comprises: In response to a third operation of the user on the device search page and / or a second pointing operation of the user, determining that the wall where the device searched by the user is located is a second wall, and a second target device searched by the user on the second wall; An identifier corresponding to the second target device is presented or selected.

4. The method according to any one of claims 1 to 3, characterized in that: The method further comprises: In response to a fourth operation of the user on the first wall on the device search page and / or a third pointing operation of the user, determining a third target device of the user; An identifier corresponding to the third target device is presented or selected.

5. The method according to any one of claims 1 to 4, characterized in that: The identification of the first target device is a controllable control, and the method further includes: In response to a user's operation on an identifier corresponding to the first target device, sending instruction information to the first target device, where the instruction information is used to control the first target device; or, In response to a user's operation on the identifier corresponding to the first target device, a control page of the first target device is displayed, where the control page is used to control the first target device.

6. The method according to any one of claims 1 to 5, characterized in that: The presenting or selecting an identifier corresponding to the first target device includes: In response to the second operation of the user pausing on the first display element associated with the first wall surface, presenting the identifier of the fourth target device; During the process of detecting the second operation of the user staying on the first display element, the first pointing operation of the user shaking the first electronic device up and down is detected, and the presentation of the identifier of the fourth target device is switched to the presentation of the identifier of the first target device.

7. The method according to any one of claims 1 to 6, characterized in that: In response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the first target device searched by the user includes: In response to a second operation of the user on the device search page and / or a first pointing operation of the user, determining a search operation type of the user; In case the search operation type is a location search operation type, displaying an identifier of a device to be searched; In the case where the search operation type is an area search operation type, multiple identifiers of devices to be searched are displayed; The first target device is one or more devices among the devices to be found.

8. The method according to claim 7, characterized in that In response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the search operation type of the user includes: Detecting an operation of a user moving a location control displayed on the device search page toward a first display element associated with a first wall surface, and determining that the search operation type is the location search operation type; or, detecting an operation of a user moving an area control displayed on the device search page toward the first display element, and determining that the search operation type is the area search operation type; or, Detecting an operation in which the user points the first electronic device toward the first wall and does not move at a certain position on the first wall or moves within a range smaller than a preset range, determining that the search operation type is the position search operation type; or, It is detected that the user points the first electronic device toward the first wall and the moving range on the first wall is greater than a preset range, and the search operation type is determined to be the area search operation type.

9. The method according to any one of claims 1 to 6, characterized in that: In response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the first target device searched by the user includes: In response to the user's second operation on the device search page and / or the user's first pointing operation, it is determined that the user's search operation type is a function search operation type.

10. The method according to any one of claims 1 to 9, characterized in that: In response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the first target device searched by the user includes: detecting a fifth operation of the user moving the position control displayed on the device search page to a first display element associated with the first wall surface, and displaying an identifier of the first device corresponding to the stop position of the position control on the first display element; A sixth operation of the user moving the position control on the first display element is detected, and an identifier of the second device corresponding to the final stop position is displayed, or identifiers of multiple devices corresponding to the movement track during the movement are switched to be displayed; The second device corresponding to the final stop position is determined as the first target device.

11. The method according to any one of claims 1 to 9, characterized in that: In response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the first target device searched by the user includes: A seventh operation of moving the area control displayed on the device search page to the first display element associated with the first wall is detected, and identifications of multiple to-be-searched devices corresponding to the first wall are displayed; In response to a first pointing operation of a user, determining a region range of the first pointing operation; The area range is matched with the multiple devices to be found, and some or all of the devices to be found are determined as the first target devices.

12. The method according to claim 11, characterized in that After detecting the seventh operation of the user moving the area control displayed on the device search page to the first display element associated with the first wall surface, and displaying the identifiers of multiple devices to be found corresponding to the first wall surface, the method further includes: detecting an eighth operation of the user moving the area control on the first display element, and determining a starting position and an ending position of the eighth operation; Determine a plurality of first target devices covered by the starting position and the ending position among the plurality of devices to be found.

13. The method according to any one of claims 1 to 12, characterized in that: In response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the first target device searched by the user includes: A ninth operation is detected in which the user moves the area control displayed on the device search page to the first display element associated with the first wall, and a subsystem identifier and / or a scene identifier is displayed. The multiple third devices corresponding to the subsystem identifier are devices of the same subsystem among the multiple devices to be found, and the multiple fourth devices corresponding to the scene identifier are devices in the same scene among the multiple devices to be found, and the multiple devices to be found are devices corresponding to the first wall.

14. The method according to claim 13, characterized in that The method further comprises: detecting a tenth operation of the user on the subsystem identifier, and determining that the plurality of third devices are the first target devices; An eleventh operation of the user on the scene identifier is detected, and the multiple fourth devices are determined to be the first target devices.

15. The method according to any one of claims 1 to 14, characterized in that: In response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the first target device searched by the user includes: The first wall and / or the first target device are determined according to a first pointing angle at which the user rotates the first electronic device to point to the wall.

16. The method according to any one of claims 1 to 14, characterized in that: In response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the first target device searched by the user includes: A second pointing angle is determined according to a stop position of the second operation on a first display element associated with a first wall surface, and the second pointing angle is used to determine the first target device.

17. The method according to any one of claims 1 to 16, characterized in that: The method of displaying a device search page in response to a first operation of the user includes: In response to a first operation of the user, a spatial data model is acquired, where the spatial data model includes house data, room data, wall data, and device data; The device search page is displayed according to the spatial data model and the positioning data of the first electronic device.

18. The method according to claim 17, characterized in that The house data is used to locate the house where the first target device is located, the room data is used to locate the room where the first target device is located, the wall data is used to locate the wall where the first target device is located, and the device data is used to locate the position of the first target device on the wall.

19. The method according to claim 17 or 18, characterized in that The room data includes a reference anchor point, and the device data includes a first orientation angle of the device relative to the reference anchor point; In response to the second operation of the user on the device search page and / or the first pointing operation of the user, determining the first target device searched by the user includes: The first pointing operation is detected, and a second azimuth angle corresponding to the first pointing operation is acquired; The second azimuth angle is converted into the first azimuth angle to determine a first target device pointed to by the first azimuth angle.

20. The method according to any one of claims 1 to 19, characterized in that: The method further comprises: Detecting a user's operation of adding a device, instructing the user to point the first electronic device toward the newly added device; A prompt message is displayed indicating that the newly added device has been successfully entered.

21. An electronic device, characterized in that: include: A processor, a memory and a display screen, wherein the memory and the display screen are coupled to the processor, the memory is used to store computer program code, and the computer program code includes computer instructions. When the processor reads the computer instructions from the memory, the electronic device executes the method as described in any one of claims 1 to 20.

22. A computer-readable storage medium, characterized in that: The computer-readable storage medium comprises a computer program, and when the computer program is executed on an electronic device, the electronic device is enabled to execute the method according to any one of claims 1 to 20.

23. A computer program product, characterized in that When the computer program product is executed on a computer, the computer is caused to execute the method according to any one of claims 1 to 20.

Images