Smart home device control system

Abstract

An intelligent home device control system includes a mobile terminal device, a plurality of access points, and a cloud server. The mobile terminal device is configured to measure an azimuth angle and an elevation angle of the mobile terminal device. The access points are configured to locate a plurality of home devices to obtain a plurality of positions of the home devices and to measure a plurality of distances between the access points and the mobile terminal device, respectively. The cloud server is configured to determine whether a position of the mobile terminal device is changed and to calculate the position of the mobile terminal device. The cloud server is configured to determine whether the azimuth angle and the elevation angle of the mobile terminal device are changed and to calculate a pointing direction of the mobile terminal device. The cloud server is configured to control at least one of the home devices corresponding to a control signal from a user interface of the mobile terminal device in response to the control signal.

Description

Technical Field

[0001] This disclosure relates to a smart home device control system. Background Technology

[0002] In recent years, with the rapid development of smart home technology, more and more users are enjoying a convenient and intelligent lifestyle. However, current smart home control methods still suffer from operational complexity, failing to meet users' needs for more intuitive or convenient interaction. Existing smart home control methods still have significant room for improvement in terms of ease of operation and user experience.

[0003] Therefore, how to propose a smart home device control system that can solve the above problems is one of the issues that the industry is currently eager to invest research and development resources to address. Summary of the Invention

[0004] In view of this, one objective of this disclosure is to provide a smart home device control system that can solve the above-mentioned problems.

[0005] To achieve the above objectives, according to one embodiment of this disclosure, a smart home device control system includes a mobile terminal device, several access points, and a cloud server. The mobile terminal device includes an inertial measurement unit configured to measure the azimuth and pitch angles of the mobile terminal device. The access points provide a network. The mobile terminal device communicates with the access points via the network. The access points are configured to: locate several home devices to obtain several positions of the home devices; and respectively measure several distances between the access points and the mobile terminal device. The cloud server communicates with the mobile terminal device via the network. The cloud server includes a positioning device, a pointing calculation device, and a control device. The positioning device is configured to determine whether the position of the mobile terminal device has changed and calculate the position of the mobile terminal device. The pointing calculation device is configured to determine whether the azimuth and pitch angles of the mobile terminal device have changed and calculate the pointing of the mobile terminal device. The control device is configured to control at least one of the home devices corresponding to the control signal in response to a control signal from the user interface of the mobile terminal device. The control signal is associated with the azimuth and pitch angles of the mobile terminal device. Attached Figure Description

[0006] To make the above and other objects, features, advantages and embodiments of this disclosure more apparent and understandable, the accompanying drawings are described below:

[0007] Figure 1 This is a functional block diagram of a smart home device control system according to one embodiment of the present disclosure.

[0008] Figure 2 This is a flowchart of a smart home device control method according to one embodiment of the present disclosure.

[0009] Figure 3 This is a schematic diagram illustrating the location of access points for a door and several home appliances according to one embodiment of this disclosure.

[0010] Figure 4A This is a schematic diagram of a user holding a mobile terminal device in front of a door according to one embodiment of the present disclosure.

[0011] Figure 4B This is a schematic diagram of a user interface of a mobile terminal device generating control signals according to an embodiment of the present disclosure.

[0012] Figure 5 This is a schematic diagram illustrating the measurement of several distances between a mobile terminal device and several access points according to an embodiment of the present disclosure.

[0013] Figure 6 This is a schematic diagram of the pointing, azimuth, pitch, and roll angles of a mobile terminal device according to one embodiment of the present disclosure.

[0014] Figure 7 This is a schematic diagram illustrating the calculation of several distances and several angular differences between a mobile terminal device and several home appliances according to an embodiment of the present disclosure.

[0015] Figure 8 This is a schematic diagram of a list on a user interface that is associated with the sorting of several home appliances according to one embodiment of the present disclosure.

[0016] Figure 9 This is a schematic diagram illustrating a user interface for displaying a home appliance that a user wishes to control, according to one embodiment of the present disclosure.

[0017] Figure 10A This is a schematic diagram illustrating a user's attempt to find lost items according to one embodiment of this disclosure.

[0018] Figure 10B This is a schematic diagram illustrating a user's attempt to find lost items according to another embodiment of this disclosure.

[0019] Figure 11A This is a schematic diagram of a mobile terminal device, a cloud server, and a receiving terminal device according to one embodiment of the present disclosure.

[0020] Figure 11B This is a schematic diagram showing icons and messages associated with a receiving terminal device in a user interface according to an embodiment of the present disclosure.

[0021] Figure 12A This is a schematic diagram of a user performing a scrolling operation on a mobile terminal device according to an embodiment of the present disclosure.

[0022] Figure 12B This is a schematic diagram of a home appliance controlled by a scrolling operation according to one embodiment of the present disclosure.

[0023] Figure 13A This is a schematic diagram of a user performing a scrolling operation on a mobile terminal device according to another embodiment of the present disclosure.

[0024] Figure 13B This is a schematic diagram of a home appliance controlled by a scrolling operation according to another embodiment of the present disclosure.

[0025] Figure Labels

[0026] 100: Smart Home Device Control System

[0027] 110: Mobile terminal devices

[0028] 112, 122: Communication devices

[0029] 114: Latitude and Longitude Calculation Device

[0030] 116: Inertial Measurement Unit

[0031] 120: Cloud server

[0032] 123: Database

[0033] 124: Positioning device

[0034] 126: Points to computing device

[0035] 127: Selection device

[0036] 128: Control device

[0037] 130: Receiving terminal equipment

[0038] AC, CD, CF, DHM, SF, TV: Home appliances

[0039] D 1101 D 1102 D 1103 D AC D CF D KY D TV :distance

[0040] DL: Download Operation

[0041] DR: Door

[0042] ICN, ICN1, ICN2, ICN3, ICN4, ICN5: Illustration

[0043] ITN: Network

[0044] KY: Lost Items

[0045] L110: Location

[0046] LS: List

[0047] M: Method

[0048] M 1101 M 1102 M 1103 M DR1 M DR2 M DR3 M TV1 M TV2 M TV3 Distance measurement operation

[0049] MSG1, MSG2, MSG3, MSG4, MSG5: Messages

[0050] P110: Pointing to

[0051] RSP1, RSP2: Feedback Operations

[0052] S201, S202, S203, S204, S205, S206, S207, S208, S209, S210:

[0053] step

[0054] SG1, SG2, SG3, SG4: Control signals

[0055] UI: User Interface

[0056] UL: Upload Operation

[0057] US: User

[0058] WF1, WF2, WF3: Access Points

[0059] X, Y, Z: Direction

[0060] θ: Pitch angle

[0061] θ AC θ CF θ TV : Angle difference

[0062] φ: Roll angle

[0063] φ1, φ2: Rolling operation

[0064] ψ: Azimuth Detailed Implementation

[0065] The following describes several embodiments of this disclosure with reference to the accompanying drawings. For clarity, many practical details will be described in the following description. However, it should be understood that these practical details should not be used to limit this disclosure. That is, these practical details are not essential in some embodiments of this disclosure. Furthermore, for the sake of simplicity, some well-known and conventional structures and components will be shown in the drawings in a simple schematic manner. The same reference numerals will be used to denote the same or similar components in all the drawings.

[0066] The following will describe in detail the structure, function, and connection relationship between the components included in the smart home device control system 100 of this embodiment.

[0067] Please refer to Figure 1 . Figure 1 This is a functional block diagram of a smart home device control system 100 according to one embodiment of the present disclosure. Figure 1 As shown, in this embodiment, the smart home device control system 100 includes a mobile terminal device 110, a cloud server 120, and a network ITN. The cloud server 120 is communicatively connected to the mobile terminal device 110 via the network ITN. The network ITN can be implemented through several access points. Specifically, several access points provide the network ITN. The mobile terminal device 110 is communicatively connected to the access points via the network ITN. The access points are configured to locate several home devices to obtain several positions of the several home devices. The access points are further configured to measure several distances between the several access points and the mobile terminal device 110 respectively. The mobile terminal device 110 includes a communication device 112, a latitude and longitude calculation device 114, and an inertial measurement device 116. The mobile terminal device 110 is communicatively connected to the network ITN via the communication device 112. The latitude and longitude calculation device 114 is configured to measure several distances between the mobile terminal device 110 and the several access points respectively. The inertial measurement unit 116 is configured to measure the azimuth, pitch, and roll angles of the mobile terminal device 110.

[0068] Please continue to refer to this. Figure 1In this embodiment, the cloud server 120 includes a communication device 122, a database 123, a positioning device 124, a pointing calculation device 126, a selection device 127, and a control device 128. The cloud server 120 is communicatively connected to the network ITN via the communication device 122. The database 123 stores data including the location of access points, the locations of several home appliances, the location of the mobile terminal device 110, its azimuth, pitch, roll, pointing, and distance measurement. The positioning device 124 is configured to determine whether the position of the mobile terminal device 110 has changed and calculate the position of the mobile terminal device 110. The pointing calculation device 126 is configured to determine whether the azimuth and pitch angles of the mobile terminal device 110 have changed and calculate the pointing of the mobile terminal device 110. The pointing calculation device 126 is further configured to calculate the pointing of the mobile terminal device 110 based on its azimuth and pitch angles. The pointing calculation device 126 is further configured to calculate the distance and angle difference between the mobile terminal device 110 and the home appliances, respectively. Selection device 127 is configured to sort several home appliances based on the distance and angle difference between the mobile terminal device 110 and several home appliances. Control device 128 is configured to control at least one of the several home appliances corresponding to the control signal in response to a control signal from the user interface of the mobile terminal device 110. In some embodiments, the control signal is associated with the azimuth and pitch angles of the mobile terminal device 110.

[0069] In some implementations, the combination of communication device 112, communication device 122 and network ITN supports, for example, the 802.11mc protocol, thereby enabling indoor positioning technology for access points, the Internet of Things (IoT) and mobile terminal devices 110.

[0070] Please refer to Figure 2 . Figure 2 This is a flowchart of a smart home device control method M according to one embodiment of the present disclosure. Figure 2 The method M shown includes steps S201, S202, S203, S204, S205, S206, S207, S208, S209, and S210. For a better understanding of step S201, please refer to... Figure 2 as well as Figure 3 For a better understanding of step S202, please refer to [link / reference]. Figure 2 , Figure 4A , Figure 4B as well as Figure 5 For a better understanding of steps S203 and S204, please refer to... Figure 2 as well as Figure 5 For a better understanding of step S205, please refer to [link / reference]. Figure 2 as well as Figure 6 For a better understanding of steps S206 and S207, please refer to... Figure 2 , Figure 4B as well as Figure 7 For a better understanding of steps S208, S209, and S210, please refer to... Figure 2 , Figure 7 , Figure 8 as well as Figure 9 .

[0071] The following details steps S201, S202, S203, S204, S205, S206, S207, S208, S209, and S210.

[0072] Step S201: Locate several home appliances to obtain several locations of the home appliances.

[0073] Please refer to Figure 3 . Figure 3 This is a schematic diagram illustrating the location of a door DR and several home appliances according to an embodiment of this disclosure. Figure 3 As shown, this embodiment provides a room configuration. The room includes a door (DR), a home appliance (TV), a home appliance (AC), a home appliance (CF), a home appliance (CD), and a home appliance (DHM). Access points WF1, WF2, and WF3 are positioned around the room. Access points WF1, WF2, and WF3 provide a network ITN. Access points WF1, WF2, and WF3 each perform a ranging operation M on the door (DR). DR1 Distance measurement operation M DR2 and ranging operation M DR3 And through the ranging operation M DR1 Distance measurement operation M DR2 and ranging operation M DR3 Obtain the location of the door DR. Access points WF1, WF2, and WF3 also perform distance measurement operations on the home appliance TV. TV1、 Distance measurement operation M TV2 and ranging operation M TV3 And through the ranging operation M TV1 Distance measurement operation M TV2 and ranging operation M TV3 Obtain the location of home device TV. Similarly, access points WF1, WF2, and WF3 can also perform ranging operations on home devices AC, CF, CD, and DHM respectively to obtain the locations of several home devices.

[0074] In some implementations, access points WF1, WF2, and WF3 may be, for example, WiFi base stations or other suitable network facilities.

[0075] Step S202: Measure the distances between the mobile terminal device and several access points respectively.

[0076] Please refer to Figure 4A , Figure 4B as well as Figure 5 . Figure 4A This is a schematic diagram of a user US handheld mobile terminal device 110 in front of a room door DR according to an embodiment of the present disclosure. Figure 4B A schematic diagram of a user interface (UI) of a mobile terminal device 110 that generates a control signal SG1 for a user US according to an embodiment of the present disclosure. Figure 5 This is a schematic diagram illustrating the measurement of several distances between a mobile terminal device 110 and several access points according to an embodiment of the present disclosure. Figure 4A As shown, in this embodiment, before performing step S202, the user US holds a mobile terminal device 110 in front of the room door DR to obtain authorization to enter the room. Then, as... Figure 4B As shown, when a user US using a handheld mobile terminal device 110 approaches the door DR, the control device 128 of the cloud server 120 controls the user interface (UI) of the mobile terminal device 110 to display the icon ICN and the message MSG1. In some embodiments, the message MSG1 includes a query asking the user US whether they wish to unlock the door DR. For example, the message MSG1 may contain the words "Open door 1207?". In one use case, the cloud server 120 may be configured to only allow specific devices (e.g., mobile terminal device 110) to be authorized to enter the room. Then, when the user US provides the control signal SG1 to the user interface (UI) in the area where the icon ICN is distributed, the door DR is unlocked, thereby authorizing the user US to enter the room. In some embodiments, the control signal SG1 may be, for example, a touch signal or other suitable type of signal.

[0077] Please continue to refer to this. Figure 5 In this embodiment, after the user US of the handheld mobile terminal device 110 enters the room with authorization, the latitude and longitude calculation device 114 of the mobile terminal device 110 performs a ranging operation M. 1101 The distance D1101 between the mobile terminal device 110 and the access point WF1 is calculated at any time. The latitude and longitude calculation device 114 of the mobile terminal device 110 performs the ranging operation M. 1102The distance D1102 between the mobile terminal device 110 and the access point WF2 is calculated at any time. The latitude and longitude calculation device 114 of the mobile terminal device 110 performs the ranging operation M. 1103 To calculate the distance D between the mobile terminal device 110 and the access point WF3 at any time. 1103 In some implementations, access points WF1, WF2, and WF3 also perform ranging operations on the mobile terminal device 110, respectively. 1101 Distance measurement operation M 1102 and ranging operation M 1103 To calculate distance D at any time 1101 Distance D 1102 and distance D 1103 .

[0078] Step S203: Determine whether the location of the mobile terminal device has changed.

[0079] Please continue to refer to this. Figure 5 When the user US of the handheld mobile terminal device 110 enters the room, the positioning device 124 determines whether the position L110 of the mobile terminal device 110 has changed. Specifically, the mobile terminal device 110, access points WF1, WF2, and WF3 calculate the relevant distance D. 1101 Distance D 1102 and distance D 1103 The data is transmitted to database 123, and the positioning device 124 calculates the distance D at any time. 1101 Distance D 1102 and distance D 1103 To determine whether the position L110 of the mobile terminal device 110 has changed.

[0080] Step S204: Calculate the location of the mobile terminal device.

[0081] Please continue to refer to this. Figure 5 If the positioning device 124 calculates the distance D in step S203... 1101 Distance D 1102 and distance D 1103 When it is determined that the position L110 of the mobile terminal device 110 has changed, the positioning device 124 calculates the position L110 of the mobile terminal device 110. Specifically, the positioning device 124 calculates the position L110 of the mobile terminal device 110, which is essentially calculating the absolute position L110 of the mobile terminal device 110.

[0082] Step S205: Determine whether the azimuth and pitch angles of the mobile terminal device have changed.

[0083] Please refer to Figure 6 . Figure 6This is a schematic diagram showing the pointing P110, azimuth angle ψ, pitch angle θ, and roll angle φ of a mobile terminal device 110 according to an embodiment of the present disclosure. Figure 6 As shown, in this embodiment, if the positioning device 124 calculates the distance D at any time in step S203... 1101 Distance D 1102 and distance D 1103 If the position L110 of the mobile terminal device 110 remains unchanged, the pointing calculation device 126 then determines whether the azimuth angle ψ and pitch angle θ of the mobile terminal device 110 have changed. The azimuth angle ψ, pitch angle θ, and roll angle φ are the angles of rotation in the Z, Y, and X directions, respectively. Specifically, the inertial measurement device 116 of the mobile terminal device 110 continuously measures the azimuth angle ψ and pitch angle θ and transmits the data to the database 123. The pointing calculation device 126 then determines whether the pointing P110 of the mobile terminal device 110 has changed based on the continuously measured azimuth angle ψ and pitch angle θ. In other words, the pointing calculation device 126 can calculate the pointing P110 of the mobile terminal device 110 based solely on the azimuth angle ψ and pitch angle θ. In some embodiments, the inertial measurement device 116 of the mobile terminal device 110 also measures the roll angle φ of the mobile terminal device 110 at any time and transmits the roll angle φ data to the database 123, and the pointing calculation device 126 determines whether the pointing P110 of the mobile terminal device 110 has changed based on the azimuth angle ψ, pitch angle θ and roll angle φ measured at any time.

[0084] Step S206: Calculate the distances and angle differences between the mobile terminal device and several home appliances respectively.

[0085] Please refer to Figure 7 . Figure 7 This is a schematic diagram illustrating the calculation of several distances and angle differences between a mobile terminal device 110 and several home appliances according to an embodiment of this disclosure. Figure 7 As shown, in this embodiment, if the pointing calculation device 126 determines in step S205 that the azimuth angle ψ and pitch angle θ of the mobile terminal device 110 have changed, then the pointing calculation device 126 then calculates several distances and several angle differences between the mobile terminal device 110 and several home appliances. Specifically, the pointing calculation device 126 calculates the distance D between the mobile terminal device 110 and the home appliance AC. AC The angle difference θ is calculated based on the vector of the straight connection between the mobile terminal device 110 and the home appliance AC, and the pointing direction P110 of the mobile terminal device 110. AC The computing device 126 calculates the distance D between the mobile terminal device 110 and the home appliance TV. TVThe angle difference θ is calculated based on the vector of the straight connection between the mobile terminal device 110 and the home appliance TV, and the pointing direction P110 of the mobile terminal device 110. TV The computing device 126 calculates the distance D between the mobile terminal device 110 and the home appliance CF. CF The angle difference θ is calculated based on the vector of the straight connection between the mobile terminal device 110 and the home device CF, and the pointing direction P110 of the mobile terminal device 110. CF .

[0086] In some embodiments, after the positioning device 124 calculates the position L110 of the mobile terminal device 110 in step S204, step S205 is skipped and step S206 is executed directly. In some embodiments, the angular differences between the mobile terminal device 110 and several home appliances are related to the azimuth angle ψ and the pitch angle θ of the mobile terminal device 110.

[0087] like Figure 7 As shown, in one use case, the distance D AC Greater than distance D TV And distance D CF Greater than distance D AC In one usage scenario, the angle difference θ TV Greater than the angle difference θ AC And the angle difference θ CF Greater than the angle difference θ TV .

[0088] Step S207: Maintain the user interface of the mobile terminal device.

[0089] Please refer to this again. Figure 4B In this embodiment, if the positioning device 124 determines in step S203 that the position L110 of the mobile terminal device 110 has not changed and the pointing computing device 126 determines in step S205 that the azimuth angle ψ and pitch angle θ of the mobile terminal device 110 have not changed, then the control device 128 maintains the user interface UI of the mobile terminal device 110. Specifically, if the cloud server 120 determines that the position L110 and the pointing P110 of the mobile terminal device 110 have not changed (for example, the user US is standing outside the door DR or the user US maintains a fixed posture in the room), then the control user interface UI is maintained at such a position. Figure 4B The screen shown prevents users from controlling multiple home appliances through the user interface (UI).

[0090] Step S208: Sort the home devices based on several distances and angle differences between the mobile terminal device and several home devices.

[0091] Please continue to refer to this. Figure 7 In this embodiment, after the pointing computing device 126 calculates several distances and several angle differences between the mobile terminal device 110 and several home appliances in step S206, the selection device 127 sorts the several home appliances based on the several distances and several angle differences between the mobile terminal device 110 and the several home appliances. Specifically, in step S208, the selection device 127 detects which home appliances are located around the pointing P110 of the mobile terminal device 110, so as to determine which home appliance the user US wants to control.

[0092] In some implementations, the selection device 127 prioritizes sorting several home appliances based on angle differences. For example, the selection device 127 sorts home appliances AC, CF, and TV that are adjacent to the mobile terminal device 110, but due to the angle difference θ TV Greater than the angle difference θ AC And the angle difference θ CF Greater than the angle difference θ TV Therefore, the selection device 127 generates a sequence of "home device AC takes precedence over home device TV, and home device TV takes precedence over home device CF". In other words, the selection device 127 determines that the device that the user US of the handheld mobile terminal device 110 is most likely to want to control is home device AC.

[0093] In some other embodiments, the selection device 127 can simultaneously consider both distance and angle difference to sort several home appliances. Specifically, the selection device 127 can first temporarily sort the several home appliances based on angle difference, and then consider the variable of distance to make a final sort. For example, the temporary sorting order is "home appliance AC takes precedence over home appliance TV, and home appliance TV takes precedence over home appliance CF", but due to distance D AC The distance between the home device AC and the mobile terminal device 110 is greater than a threshold (i.e., the home device AC is too far away from the mobile terminal device 110), so the order is changed to "home device TV takes precedence over home device AC, and home device AC takes precedence over home device CF" in the final sorting.

[0094] Step S209: Display a list associated with the sorting of several home appliances on the user interface of the mobile terminal device.

[0095] Please refer to Figure 8 . Figure 8 This is a schematic diagram on a user interface (UI) showing a list LS associated with the sorting of several home appliances according to one embodiment of the present disclosure. Figure 8As shown, in this embodiment, after the selection device 127 sorts the home appliances based on several distances and angle differences between the mobile terminal device 110 and several home appliances in step S208, the control device 128 displays a list LS associated with the sorting of home appliances on the user interface UI of the mobile terminal device 110. Specifically, the list LS includes home appliances AC, TV, and CF that are adjacent to the mobile terminal device 110, and home appliances AC, TV, and CF are represented by icons ICN1, ICN2, and ICN3, respectively. The control device 128 further controls the user interface UI to display message MSG2. In some embodiments, the content of message MSG2 includes asking the user if they wish to control a specific home appliance. For example, message MSG2 may contain the phrase "Want to control which appliance?". In one usage scenario, such as Figure 7 as well as Figure 8 As shown, if the selection device 127 sorts several home appliances by angle difference in step S208, the list LS will be displayed on the user interface UI in the order of "home appliance AC, home appliance TV, home appliance CF".

[0096] Step S210: Control the home appliances corresponding to the control signals in response to the control signals from the user interface.

[0097] Please refer to Figure 9 . Figure 9 This is a schematic diagram illustrating a user interface (UI) of a home appliance that a user (US) wishes to control, according to one embodiment of the present disclosure. Figure 9 As shown, in this embodiment, after the control device 128 displays a list LS associated with the order of several home appliances on the user interface UI of the mobile terminal device 110 in step S209, the control device 128 controls the home appliance corresponding to the control signal SG2 in response to the control signal SG2 from the user interface UI. Specifically, when the user US provides the control signal SG2 to the user interface UI in the area distributed in the illustrated ICN1, the user US is authorized to control the home appliance AC. Then, the user interface UI displays the control interface associated with the home appliance AC. For example, the control interface of the home appliance AC includes temperature, airflow, cooling mode, heating mode, ventilation mode, and dehumidification mode, but this disclosure is not limited thereto. The user US can again provide the control signal SG2 to the user interface UI to control the home appliance AC corresponding to the control signal SG2. Figure 9 As shown, the current temperature of the home appliance AC is 26 degrees Celsius. In some implementations, the control signal SG2 may be, for example, a touch signal or other suitable type of signal.

[0098] By using the smart home device control system 100 and executing the aforementioned smart home device control method M, the purpose of controlling home devices can be achieved by using the pointing P110 of the mobile terminal device 110 to determine the direction.

[0099] The following will describe in detail other use cases of the smart home device control system 100 disclosed herein.

[0100] Please refer to Figure 10A as well as Figure 10B . Figure 10A This is a schematic diagram illustrating a user US searching for lost item KY according to one embodiment of the present disclosure. Figure 10B This is a schematic diagram illustrating a user (US) searching for lost item (KY) according to another embodiment of this disclosure. Figure 10A as well as Figure 10B As shown, in this embodiment, the smart home device control system 100 of this disclosure has the function of finding lost items. Specifically, as... Figure 10A As shown, the lost item KY is located by access points WF1, WF2, and WF3 in step S201, causing access points WF1, WF2, and WF3 to obtain the location of the lost item KY. Next, the control device 128 determines the location based on the pointing P110 of the mobile terminal device 110 and the distance D between the mobile terminal device 110 and the lost item KY. KY The location of the lost item KY is displayed on the user interface (UI). In some embodiments, the control device 128 controls the user interface (UI) to display an icon ICN4 associated with the lost item KY and a message MSG3. For example, the icon ICN4 is a visual representation of the lost item KY, and the message MSG3 may contain the phrase "located 3.6 meters to the right front".

[0101] Please continue to refer to this. Figure 10B In one usage scenario, the lost item KY is located on home appliance SF. In step S201, the lost item KY and home appliance SF are located by access points WF1, WF2, and WF3, causing access points WF1, WF2, and WF3 to obtain the locations of the lost item KY and home appliance SF. Next, the control device 128 displays the location of the lost item KY on the user interface UI based on whether it is near any home appliance in any room. In some embodiments, the control device 128 controls the user interface UI to display an icon ICN4 associated with the lost item KY and a message MSG4. For example, the message MSG4 may contain the words "Located on the living room sofa".

[0102] Please refer to Figure 11A as well as Figure 11B . Figure 11AThis is a schematic diagram of a mobile terminal device 110, a cloud server 120, and a receiving terminal device 130 according to an embodiment of the present disclosure. Figure 11B This is a schematic diagram illustrating the user interface (UI) display of icons and messages associated with the receiving terminal device 130 according to an embodiment of this disclosure. For example... Figure 11A as well as Figure 11B As shown, in this embodiment, the smart home device control system 100 further includes a receiving terminal device 130 that is communicatively connected to a cloud server 120 via a network ITN. In one usage scenario, a user US wants to use a mobile terminal device 110 to transmit data to the receiving terminal device 130. The positioning device 124 and the pointing calculation device 126 determine which receiving terminal device 130 the mobile terminal device 110 wants to transmit data to based on the location of the receiving terminal device 130 and the pointing P110 of the mobile terminal device 110. Specifically, when the cloud server 120 determines that the pointing P110 of the mobile terminal device 110 passes through the location of the receiving terminal device 130, the mobile terminal device 110 performs an upload operation UL to transmit data to the cloud server 120, and the receiving terminal device 130 performs a download operation DL to download the data.

[0103] Please refer to Figure 12A as well as Figure 12B . Figure 12A This is a schematic diagram of a user US performing a scrolling operation φ1 on a mobile terminal device 110 according to an embodiment of the present disclosure. Figure 12B This is a schematic diagram illustrating the control of home appliance AC based on a rolling operation φ1 according to an embodiment of the present disclosure. Figure 12A as well as Figure 12B As shown, in one usage scenario, user US wants to control home appliance AC by performing a scrolling operation φ1 on mobile terminal device 110. Specifically, user US performs a scrolling operation φ1 on mobile terminal device 110 to provide a control signal SG3. The control signal SG3 is a change in the scroll angle φ. Based on the control signal SG3 of the scrolling operation φ1 from mobile terminal device 110, control device 128 controls the home appliance AC corresponding to the scrolling operation φ1 to perform a feedback operation RSP1. For example, the feedback operation RSP1 could be to increase the temperature. Figure 9 as well as Figure 12B As shown, after the control device 128 performs the feedback operation RSP1, the temperature of the home appliance AC rises from 26 degrees Celsius to 27 degrees Celsius.

[0104] Please refer to Figure 13A as well as Figure 13B . Figure 13A This is a schematic diagram of a user US performing a scrolling operation φ2 on a mobile terminal device 110 according to another embodiment of the present disclosure. Figure 13BThis is a schematic diagram of controlling a home appliance AC based on a rolling operation φ2 according to another embodiment of this disclosure. Figure 13A as well as Figure 13B As shown, in one usage scenario, user US wants to control home appliance AC by performing a scrolling operation φ2 on mobile terminal device 110. Specifically, user US performs a scrolling operation φ2 on mobile terminal device 110 to provide a control signal SG4. The control signal SG4 is a change in the scroll angle φ. The scrolling direction of scrolling operation φ2 is opposite to the scrolling direction of scrolling operation φ1. Based on the control signal SG4 of scrolling operation φ2 from mobile terminal device 110, control device 128 controls the home appliance AC corresponding to scrolling operation φ2 to perform a feedback operation RSP2. For example, feedback operation RSP2 could be lowering the temperature. Figure 9 as well as Figure 13B As shown, after the control device 128 performs the feedback operation RSP2, the temperature of the home appliance AC drops from 26 degrees Celsius to 25 degrees Celsius.

[0105] From the detailed description of the specific embodiments of this disclosure above, it is clear that in the smart home device control system of this disclosure, since the access point uses the distance measurement principle to locate several home devices, and the cloud server determines the home device that the user wants to control based on the direction of the mobile terminal device, the user can operate the home device to be controlled simply by aiming the mobile terminal device at the desired home device, thus eliminating the complicated appliance setting process. In the smart home device control system of this disclosure, since the access point uses the distance measurement principle to locate lost items, and the cloud server determines the position of the lost item relative to the mobile terminal device and the receiving terminal device that the user wants to send a message to based on the direction of the mobile terminal device, the user can accurately find the lost item and the aforementioned receiving terminal device, thereby improving the user's experience and convenience. Therefore, the smart home device control system of this disclosure allows users to control several home devices more intuitively using a mobile terminal, thereby improving the life experience and lowering the barrier to technology use.

[0106] In one embodiment of this disclosure, the smart home device control system 100 is applicable to a server that can be used for artificial intelligence (AI) computing, edge computing, or as a 5G server, cloud server, or vehicle networking server.

[0107] Although this disclosure has been described above with reference to embodiments, it is not intended to limit this disclosure. Any person skilled in the art can make various modifications and improvements without departing from the spirit and scope of this disclosure. Therefore, the scope of protection of this disclosure shall be defined by the claims of this invention.

Claims

1. A smart home device control system, characterized in that, Include: A mobile terminal device includes an inertial measurement unit configured to measure an azimuth angle and a pitch angle of the mobile terminal device; Multiple access points are provided, a network is provided, and the mobile terminal device communicates with these access points through the network, the access points being configured to: Locating multiple home appliances to obtain multiple locations of those appliances; and Measure the distances between each access point and the mobile terminal device; and A cloud server, which communicates with the mobile terminal device via the network, comprises: A positioning device is configured to determine whether a position of the mobile terminal device has changed and to calculate the position of the mobile terminal device. A pointing calculation device is configured to determine whether the azimuth angle and the pitch angle of the mobile terminal device have changed and to calculate the pointing of the mobile terminal device. as well as A control device configured to control at least one of home appliances corresponding to a control signal received from a user interface of the mobile terminal device. The control signal is associated with the azimuth and pitch angles of the mobile terminal device.

2. The smart home device control system as described in claim 1, characterized in that, The mobile terminal device further includes a latitude and longitude calculation device, which is configured to: Measure the distances between the mobile terminal device and the access points respectively.

3. The smart home device control system as described in claim 1, characterized in that, The computing device is further configured to: Calculate the pointing of the mobile terminal device based on its azimuth and pitch angles; and Calculate the multiple distances and angle differences between the mobile terminal device and the home appliances respectively.

4. The smart home device control system as described in claim 3, characterized in that, The cloud server further includes a selection device configured to: The home appliances are sorted based on the distances and angle differences between the mobile terminal device and the home appliances.

5. The smart home device control system as described in claim 4, characterized in that, The control device is further configured to: A list associated with the sorting of these home appliances is displayed on the user interface of the mobile device.

6. The smart home device control system as described in claim 4, characterized in that, The angle differences between the mobile terminal device and the home appliances are related to the azimuth and pitch angles of the mobile terminal device.

7. The smart home device control system as described in claim 1, characterized in that, The control device is further configured to: If the position of the mobile terminal device does not change and the azimuth and pitch angles of the mobile terminal device do not change, the user interface remains unchanged.

8. The smart home device control system as described in claim 1, characterized in that, The access points are further configured to locate a lost item to obtain its location, and the control device is further configured to display the location of the lost item on the user interface based on the orientation of the mobile terminal device and the distance between the mobile terminal device and the lost item.

9. The smart home device control system as described in claim 1, characterized in that, The device further includes a receiving terminal device communicating with the cloud server via the network, wherein the positioning device calculates a location of the receiving terminal device, and when the pointing of the mobile terminal device passes through the location of the receiving terminal device, the mobile terminal device performs an upload operation to transmit data to the cloud server, and the receiving terminal device performs a download operation to download the data.

10. The smart home device control system as described in claim 1, characterized in that, The pointing computing device is further configured to calculate a roll angle of the mobile terminal device, and the control device is further configured to control at least one of the home appliances corresponding to the roll operation to perform a feedback operation based on a roll operation of the mobile terminal device.

Images