Method, system and mobile device for quickly popping up electronic device control window

Abstract

The present application provides a method, system and mobile device for quickly popping up a control window of an electronic device, wherein the mobile device wirelessly communicates with a labeling device and the electronic device to be labeled. The mobile device includes: a processor; a memory; a UWB tag and a direction sensor; and a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by the processor, the mobile device performs the following steps: after the mobile device moves to the position of the electronic device, a first input is received; in response to the first input, the position of the mobile device is recorded; the position of the mobile device is the position of the electronic device; after the mobile device moves to another position and points to the electronic device, the mobile device pops up the first control window of the electronic device. The technical solution provided by the present application enables the mobile device to control electronic devices that do not have a UWB chip installed, thereby improving the user experience, with the help of a labeling device.

Description

Technical Field

[0001] The present application relates to the field of communications, and in particular to a method, system and mobile device for quickly popping up a control window for an electronic device. Background Art

[0002] By controlling the Internet of Things (IoT) devices through applications (APPs) installed on mobile devices, the control functions of different control devices of different IoT devices (for example, remote controls for smart TVs and smart air conditioners, etc.) can be integrated into one mobile device, which is widely used. However, users still need to open the above APP on their mobile devices, identify specific IoT devices from multiple IoT devices, and select specific IoT devices before entering the specific control window and performing specific control. In this way, user operations are still cumbersome and time-consuming; especially for people with weaker operating abilities such as the elderly, it will take even longer.

[0003] Electronic devices are configured with circuits or modules that can achieve precise positioning, so that after the user's mobile device is pointed at the electronic device, the electronic device can be located and the control window of the electronic device can be displayed, which can save user operations and reduce time. However, this will increase the cost of electronic devices too much; especially if all electronic devices such as homes or offices are configured with the above-mentioned circuits or modules, the cost will increase significantly. An average family or an average unit may be able to afford the cost of upgrading one or a few electronic devices, but generally speaking, it may not be able to afford the cost of upgrading all electronic devices. Therefore, for other electronic devices that have not been upgraded, how to enable the user's mobile device to point to the electronic device that has not been upgraded to locate the electronic device is a technical problem that needs to be solved. Summary of the invention

[0004] In order to solve the above technical problems, the present application provides a method, system and mobile device for quickly popping up a control window of an electronic device. When at least one electronic device is configured with the above circuit or module for realizing precise positioning, and other electronic devices do not have the above circuit or module, the user's mobile device can be pointed to the electronic device that is not configured with the above circuit or module, and then the electronic device can be located and the control window of the electronic device can be quickly displayed, which is convenient for user control and saves time.

[0005] In the first aspect, a mobile device is provided. The mobile device communicates wirelessly with a marking device and an electronic device to be marked; the mobile device includes: a processor; a memory; an ultra-wideband UWB tag; a direction sensor; and a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by the processor, the mobile device performs the following steps: after the mobile device moves to the position of the electronic device, a first input is received; in response to the first input, the position of the mobile device is recorded; the position of the mobile device is the position of the electronic device; after the mobile device moves to another position and points to the electronic device, the mobile device pops up a first control window of the electronic device. The mobile device provided in the first aspect marks the position of the electronic device without a UWB chip installed, so that the position of the electronic device without a UWB chip installed can be located, and then when it is determined that the angle at which the mobile device (such as a smart phone) points to the electronic device meets certain conditions, it can be determined that the mobile device points to the electronic device without a UWB chip installed, and then the control window of the electronic device without a UWB chip installed can be popped up; in this way, the user can conveniently control the electronic device without a UWB chip installed.

[0006] Exemplarily, the mobile device may be a portable electronic device such as a smart phone, an air mouse, a PDA, a smart remote control, a tablet computer, etc.; the annotation device may be an electronic device such as a smart TV, a smart wall clock, etc.; the electronic device may be a smart air conditioner, a smart lamp, a smart socket, a smart speaker, a smart printer, etc.

[0007] According to the first aspect, the mobile device further executes: within a preset time after the mobile device pops up the first control window of the electronic device, the first control window is displayed on the mobile device; wherein the first control window is used to receive a first user input for controlling the electronic device; if the first user input is not received within the preset time, the mobile device no longer displays the first control window. In this implementation, even if the user moves or rotates the mobile device, for example, moves the mobile device from a current position to another position, or rotates from a current orientation to another orientation, the control window will continue to be displayed on the mobile device, which can facilitate user use and improve user experience.

[0008] According to the first aspect, or any implementation of the first aspect above, the mobile device communicates wirelessly with the annotation device and the electronic device through the routing device; after the mobile device points to the annotation device, the mobile device pops up a second control window of the annotation device. In this implementation, the annotation device can be controlled by directionality.

[0009] According to the first aspect, or any implementation of the first aspect above, the mobile device further executes: within a preset time after the second control window of the annotation device pops up on the mobile device, the second control window is displayed on the mobile device; wherein the second control window is used to receive a second user input for controlling the annotation device; if the second user input is not received within the preset time, the mobile device no longer displays the second control window. In this implementation, even if the user moves or rotates the mobile device, for example, moves the mobile device from a current position to another position, or rotates from a current orientation to another orientation, the control window will continue to be displayed on the mobile device, which can facilitate user use and improve user experience.

[0010] According to the first aspect, or any implementation manner of the first aspect above, the labeling device includes at least two UWB base stations and a display screen; the at least two UWB base stations include a first UWB base station; the electronic device does not include a UWB base station; before the mobile device moves to the position of the electronic device, the mobile device also executes: obtaining the distance between the first UWB base station and another UWB base station, the distance between the UWB tag and each of the at least two UWB base stations, and the position of the UWB tag; wherein the other UWB base station is a UWB base station other than the first UWB base station among the at least two UWB base stations; based on the distance between the first UWB base station and another UWB base station, the distance between the UWB tag and each of the at least two UWB base stations, and the position of the UWB tag, obtain the pointing direction of the mobile device and the first angle between the vertical line between the mobile device and the display screen.

[0011] According to the first aspect, or any implementation of the first aspect above, before the mobile device moves to the position of the electronic device, the mobile device also performs: when the mobile device moves to the first position, based on the direction sensor, obtaining the first initial azimuth angle and the first initial rolling angle of the mobile device; when the mobile device moves to the second position, based on the direction sensor, obtaining the first real-time azimuth angle and the first real-time rolling angle of the mobile device; based on the first initial azimuth angle, the first initial rolling angle, the first real-time azimuth angle, and the first real-time rolling angle, obtaining the second azimuth angle in the second direction and the second rolling angle in the first direction; the first direction is perpendicular to the second direction.

[0012] According to the first aspect, or any implementation of the first aspect above, the mobile device further executes: based on the second azimuth angle and the second rolling angle, obtaining a second angle between the pointing direction of the mobile device and a vertical line between the mobile device and the display screen.

[0013] According to the first aspect, or any implementation of the first aspect above, after the mobile device moves to another position and points to the electronic device, the mobile device pops up a first control window of the electronic device; including: when the mobile device moves to another position and the deviation between the first angle and the second angle is within a preset first range, the mobile device points to the electronic device, and the mobile device pops up the first control window of the electronic device.

[0014] Exemplarily, if the difference between the second angle θ and the first angle α is within a preset difference range, for example, the difference between the first angle and the second angle θ is 0 degrees, it means that the mobile device is pointing at the electronic device, the mobile device pops up a control window of the electronic device, and the mobile device can control the electronic device.

[0015] If the difference between the second angle θ and the first angle α is not within the preset difference range, it means that the mobile device is not pointing to the electronic device, the mobile device will not pop up the control window of the electronic device, and the mobile device cannot control the electronic device.

[0016] According to the first aspect, or any implementation of the first aspect above, the first angle can be decomposed into a first azimuth angle and a first rolling angle; the first azimuth angle represents the angle between a straight line in the direction in which the mobile device is pointing and a perpendicular line from the mobile device to the marking device in the second direction; the first rolling angle represents the angle between a straight line in the direction in which the mobile device is pointing and a perpendicular line from the mobile device to the marking device in the first direction.

[0017] According to the first aspect, or any implementation of the first aspect above, after the mobile device moves to another position and points to the electronic device, the mobile device pops up a first control window of the electronic device; including: when the mobile device moves to another position, and the deviation between the first azimuth angle and the second azimuth angle is within a preset second range, and the deviation between the first scroll angle and the second scroll angle is within a preset third range, the mobile device points to the electronic device, and the mobile device pops up the first control window of the electronic device.

[0018] Exemplarily, if the difference between the first azimuth angle and the second azimuth angle is within a preset first difference range, for example, the first azimuth angle is equal to the second azimuth angle; and the difference between the first rolling angle and the second rolling angle is also within a preset second difference range, for example, the first rolling angle is equal to the second rolling angle, then it means that the mobile device is pointing at the electronic device, the mobile device will pop up a control window of the electronic device, and the mobile device can control the electronic device.

[0019] If the difference between the first azimuth angle and the second azimuth angle is within a preset first difference range, for example, the first azimuth angle is equal to the second azimuth angle; and the difference between the first rolling angle and the second rolling angle is not within a preset second difference range, it means that the mobile device is not pointing at the electronic device, the mobile device will not pop up the control window of the electronic device, and the mobile device cannot control the electronic device.

[0020] If the difference between the first azimuth angle and the second azimuth angle is not within the preset first difference range, for example, the first azimuth angle is equal to the second azimuth angle; and the difference between the first rolling angle and the second rolling angle is within the preset second difference range, then it means that the mobile device is not pointing at the electronic device, the mobile device will not pop up the control window of the electronic device, and the mobile device cannot control the electronic device.

[0021] If the difference between the first azimuth angle and the second azimuth angle is not within the preset first difference range, for example, the first azimuth angle is equal to the second azimuth angle; and the difference between the first rolling angle and the second rolling angle is also not within the preset second difference range, then it means that the mobile device is not pointing at the electronic device, the mobile device will not pop up the control window of the electronic device, and the mobile device cannot control the electronic device.

[0022] According to the first aspect, or any implementation manner of the first aspect above, the distance between the first UWB base station and each other UWB base station, the distance between the UWB tag and each UWB base station in at least two UWB base stations, and the position of the UWB tag are obtained; including: the mobile device sends a ranging message through the UWB tag; receives a ranging response message from each UWB base station in at least two UWB base stations; in response to the ranging response messages of at least two UWB base stations, obtains the distance between the UWB tag and each UWB base station in at least two UWB base stations; receives a message from the annotation device; obtains the distance between the first UWB base station and each other UWB base station from the message; selects the position of the first UWB base station in at least two UWB base stations as the coordinate origin of the three-dimensional coordinate system; based on the coordinate origin, obtains the position coordinates of another UWB base station in at least two UWB base stations, as well as the position coordinates of the UWB tag.

[0023] According to the first aspect, or any implementation of the first aspect above, the first input is used to select an identifier of an electronic device; the identifier of the electronic device corresponds to a parameter of the electronic device.

[0024] According to the first aspect, or any implementation of the first aspect above, the parameter includes one or more of a media access control MAC address, an Internet Protocol IP address, a serial number SN, or a device identifier of the electronic device.

[0025] According to the first aspect, or any implementation of the first aspect above, the direction sensor is an inertial measurement unit IMU; the IMU is a 9-axis IMU, and the 9-axis IMU includes an acceleration sensor, a gyroscope sensor, and a magnetic sensor; or, the IMU is a 6-axis IMU, and the 6-axis IMU includes an acceleration sensor and a gyroscope sensor.

[0026] According to the first aspect, or any implementation of the first aspect above, the first direction is a horizontal direction, and the second direction is a vertical direction; or, the first direction is a vertical direction, and the second direction is a horizontal direction.

[0027] In a second aspect, a labeling device is provided. The labeling device wirelessly communicates with the mobile device provided in the first aspect and any one of the implementations of the first aspect, and the electronic device to be labeled; the labeling device includes: a processor; a memory; at least two UWB base stations, the at least two UWB base stations including a first UWB base station; and a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by the processor, the labeling device performs the following steps: receiving a ranging message through each of the at least two UWB base stations; in response to the ranging message, sending a ranging response message to the mobile device through each of the at least two UWB base stations; sending a message to the mobile device through any one of the at least two UWB base stations, the message including the distance between the first UWB base station and each of the other UWB base stations. The marking device provided in the second aspect utilizes the ranging information sent by at least two UWB base stations on the marking device to the UWB tag on the mobile device respectively, so as to determine the distance between the UWB tag and at least two UWB base stations respectively, and then determine the first angle, wherein the first angle is the angle between the line between the mobile device and the electronic device and the perpendicular line from the mobile device to the marking device, which enables the mobile device to determine whether it is pointing to the electronic device, wherein the UWB chip is not installed on the electronic device.

[0028] According to the second aspect, the marking device also includes: a display screen; the display screen includes a first edge, a second edge, a third edge and a fourth edge; the first edge is parallel to the second edge, and the third edge is parallel to the fourth edge; the first edge and the second edge are parallel to a first direction, the third edge and the fourth edge are parallel to a second direction, and the first direction is perpendicular to the second direction; at least two UWB base stations are distributed at different positions in front of or behind the display screen; all or part of the vertical projections of the at least two UWB base stations on the display screen are located on the display screen.

[0029] According to the second aspect, at least two UWB base stations include a first UWB base station, a second UWB base station and a third UWB base station; all or part of the vertical projection of the first UWB base station on the display screen is located at the intersection of the third edge and the second edge; all or part of the vertical projection of the second UWB base station on the display screen is located at the intersection of the third edge and the first edge; all or part of the vertical projection of the third UWB base station on the display screen is located at the intersection of the second edge and the fourth edge.

[0030] Exemplarily, the annotation device may be a device with a relatively fixed (or stationary) position, such as a smart TV, a smart screen, a wall clock, a smart air conditioner, etc.

[0031] According to the second aspect, or any implementation of the second aspect, the first direction is a horizontal direction, and the second direction is a vertical direction; or, the first direction is a vertical direction, and the second direction is a horizontal direction.

[0032] In the third aspect, a method for quickly popping up an electronic device control window is provided. The method is applied to a mobile device, which wirelessly communicates with a marking device and an electronic device to be marked; the mobile device includes: a processor; a memory; an ultra-wideband UWB tag; a direction sensor; the method includes: after the mobile device moves to the position of the electronic device, a first input is received; in response to the first input, the position of the mobile device is recorded; the position of the mobile device is the position of the electronic device; after the mobile device moves to another position and points to the electronic device, the mobile device pops up a first control window of the electronic device. The method provided in the third aspect marks the position of the electronic device without a UWB chip installed, so that the position of the electronic device without a UWB chip installed can be located, and then when it is determined that the angle at which the mobile device (such as a smart phone) points to the electronic device meets certain conditions, it can be determined that the mobile device points to the electronic device without a UWB chip installed, and then the control window of the electronic device without a UWB chip installed can be popped up; in this way, the user can conveniently control the electronic device without a UWB chip installed.

[0033] Exemplarily, the mobile device may be a portable electronic device such as a smart phone, an air mouse, a PDA, a smart remote control, a tablet computer, etc.; the annotation device may be an electronic device such as a smart TV, a smart wall clock, etc.; the electronic device may be a smart air conditioner, a smart lamp, a smart socket, a smart speaker, a smart printer, etc.

[0034] According to the third aspect, the method further includes: within a preset time period after the mobile device pops up the first control window of the electronic device, the first control window is displayed on the mobile device; wherein the first control window is used to receive a first user input for controlling the electronic device; if the first user input is not received within the preset time period, the mobile device no longer displays the first control window. In this implementation, even if the user moves or rotates the mobile device, for example, moves the mobile device from a current position to another position, or rotates from a current orientation to another orientation, the control window will continue to be displayed on the mobile device, which can facilitate user use and improve user experience.

[0035] According to the third aspect, or any implementation of the third aspect above, the mobile device wirelessly communicates with the annotation device and the electronic device through a routing device, and the method also includes: after the mobile device points to the annotation device, the mobile device pops up a second control window of the annotation device.

[0036] According to the third aspect, or any implementation of the third aspect above, the method further includes: within a preset time period after the second control window of the annotation device pops up on the mobile device, the second control window is displayed on the mobile device; wherein the second control window is used to receive a second user input for controlling the annotation device; if the second user input is not received within the preset time period, the mobile device no longer displays the second control window. In this implementation, even if the user moves or rotates the mobile device, for example, moves the mobile device from a current position to another position, or rotates from a current orientation to another orientation, the control window will continue to be displayed on the mobile device, which can facilitate user use and improve user experience.

[0037] According to the third aspect, or any implementation manner of the third aspect above, the labeling device includes at least two UWB base stations and a display screen; the at least two UWB base stations include a first UWB base station; the electronic device does not include a UWB base station; before the mobile device moves to the position of the electronic device, the method also includes: obtaining the distance between the first UWB base station and another UWB base station, the distance between the UWB tag and each of the at least two UWB base stations, and the position of the UWB tag; wherein the other UWB base station is a UWB base station other than the first UWB base station among the at least two UWB base stations; based on the distance between the first UWB base station and another UWB base station, the distance between the UWB tag and each of the at least two UWB base stations, and the position of the UWB tag, obtain the pointing direction of the mobile device and the first angle between the vertical line between the mobile device and the display screen.

[0038] According to the third aspect, or any implementation of the third aspect above, before the mobile device moves to the position of the electronic device, the method also includes: when the mobile device moves to the first position, based on the direction sensor, obtaining the first initial azimuth angle and the first initial rolling angle of the mobile device; when the mobile device moves to the second position, based on the direction sensor, obtaining the first real-time azimuth angle and the first real-time rolling angle of the mobile device; based on the first initial azimuth angle, the first initial rolling angle, the first real-time azimuth angle, and the first real-time rolling angle, obtaining the second azimuth angle in the second direction and the second rolling angle in the first direction; the first direction is perpendicular to the second direction.

[0039] According to the third aspect, or any implementation of the third aspect above, the method further includes: based on the second azimuth angle and the second rolling angle, obtaining a second angle between the pointing direction of the mobile device and a vertical line between the mobile device and the display screen.

[0040] According to the third aspect, or any implementation method of the third aspect above, after the mobile device moves to another position and points to the electronic device, the mobile device pops up the first control window of the electronic device; including: when the mobile device moves to another position, and the deviation between the first angle and the second angle is within a preset first range, the mobile device points to the electronic device, and the mobile device pops up the first control window of the electronic device.

[0041] Exemplarily, if the difference between the second angle θ and the first angle α is within a preset difference range, for example, the difference between the first angle and the second angle θ is 0 degrees, it means that the mobile device is pointing at the electronic device, the mobile device pops up a control window of the electronic device, and the mobile device can control the electronic device.

[0042] If the difference between the second angle θ and the first angle α is not within the preset difference range, it means that the mobile device is not pointing to the electronic device, the mobile device will not pop up the control window of the electronic device, and the mobile device cannot control the electronic device.

[0043] According to the third aspect, or any implementation of the third aspect above, the first angle can be decomposed into a first azimuth angle and a first rolling angle; the first azimuth angle represents the angle between a straight line in the direction in which the mobile device is pointing and a perpendicular line from the mobile device to the marking device in the second direction; the first rolling angle represents the angle between a straight line in the direction in which the mobile device is pointing and a perpendicular line from the mobile device to the marking device in the first direction.

[0044] According to the third aspect, or any implementation of the third aspect above, after the mobile device moves to another position and points to the electronic device, the mobile device pops up a first control window of the electronic device; including: when the mobile device moves to another position, and the deviation between the first azimuth angle and the second azimuth angle is within a preset second range, and the deviation between the first scroll angle and the second scroll angle is within a preset third range, the mobile device points to the electronic device, and the mobile device pops up the first control window of the electronic device.

[0045] Exemplarily, if the difference between the first azimuth angle and the second azimuth angle is within a preset first difference range, for example, the first azimuth angle is equal to the second azimuth angle; and the difference between the first rolling angle and the second rolling angle is also within a preset second difference range, for example, the first rolling angle is equal to the second rolling angle, then it means that the mobile device is pointing at the electronic device, the mobile device will pop up a control window of the electronic device, and the mobile device can control the electronic device.

[0046] If the difference between the first azimuth angle and the second azimuth angle is within a preset first difference range, for example, the first azimuth angle is equal to the second azimuth angle; and the difference between the first rolling angle and the second rolling angle is not within a preset second difference range, it means that the mobile device is not pointing at the electronic device, the mobile device will not pop up the control window of the electronic device, and the mobile device cannot control the electronic device.

[0047] If the difference between the first azimuth angle and the second azimuth angle is not within the preset first difference range, for example, the first azimuth angle is equal to the second azimuth angle; and the difference between the first rolling angle and the second rolling angle is within the preset second difference range, then it means that the mobile device is not pointing at the electronic device, the mobile device will not pop up the control window of the electronic device, and the mobile device cannot control the electronic device.

[0048] If the difference between the first azimuth angle and the second azimuth angle is not within the preset first difference range, for example, the first azimuth angle is equal to the second azimuth angle; and the difference between the first rolling angle and the second rolling angle is also not within the preset second difference range, then it means that the mobile device is not pointing at the electronic device, the mobile device will not pop up the control window of the electronic device, and the mobile device cannot control the electronic device.

[0049] According to the third aspect, or any implementation manner of the third aspect above, the distance between the first UWB base station and each other UWB base station, the distance between the UWB tag and each UWB base station in at least two UWB base stations, and the position of the UWB tag are obtained; including: the mobile device sends a ranging message through the UWB tag; receives a ranging response message from each UWB base station in at least two UWB base stations; in response to the ranging response messages of at least two UWB base stations, obtains the distance between the UWB tag and each UWB base station in at least two UWB base stations; receives a message from the annotation device; obtains the distance between the first UWB base station and each other UWB base station from the message; selects the position of the first UWB base station in at least two UWB base stations as the coordinate origin of the three-dimensional coordinate system; based on the coordinate origin, obtains the position coordinates of another UWB base station in at least two UWB base stations, as well as the position coordinates of the UWB tag.

[0050] According to the third aspect, or any implementation of the third aspect, the first input is used to select an identifier of an electronic device; the identifier of the electronic device corresponds to a parameter of the electronic device.

[0051] According to the third aspect, or any implementation of the third aspect above, the parameter includes one or more of a media access control MAC address, an Internet Protocol IP address, a serial number SN, or a device identifier of the electronic device.

[0052] According to the third aspect, or any implementation of the third aspect above, the direction sensor is an inertial measurement unit IMU; the IMU is a 9-axis IMU, and the 9-axis IMU includes an acceleration sensor, a gyroscope sensor, and a magnetic sensor; or, the IMU is a 6-axis IMU, and the 6-axis IMU includes an acceleration sensor and a gyroscope sensor.

[0053] According to the third aspect, or any implementation of the third aspect, the first direction is a horizontal direction, and the second direction is a vertical direction; or, the first direction is a vertical direction, and the second direction is a horizontal direction.

[0054] According to the third aspect, or any implementation of the third aspect, the mobile device includes: a smart phone and an air mouse.

[0055] In a fourth aspect, a method for quickly popping up an electronic device control window is provided. The method is applied to a labeling device, which wirelessly communicates with a mobile device and an electronic device to be labeled; the labeling device includes: a processor; a memory; at least two UWB base stations, the at least two UWB base stations including a first UWB base station; the method includes: receiving a ranging message through each of the at least two UWB base stations; in response to the ranging message, sending a ranging response message to the mobile device through each of the at least two UWB base stations; sending a message to the mobile device through any one of the at least two UWB base stations, the message including the distance between the first UWB base station and each of the other UWB base stations. The method provided in the fourth aspect utilizes the ranging information sent by at least two UWB base stations on the marking device to the UWB tag on the mobile device respectively, so that the distance between the UWB tag and at least two UWB base stations can be determined, and then the first angle can be determined, wherein the first angle is the angle between the line between the mobile device and the electronic device and the vertical line from the mobile device to the marking device, which can enable the mobile device to determine whether it is pointing to the electronic device, wherein the UWB chip is not installed on the electronic device.

[0056] According to the fourth aspect, the marking device also includes: a display screen; the display screen includes a first edge, a second edge, a third edge and a fourth edge; the first edge is parallel to the second edge, and the third edge is parallel to the fourth edge; the first edge and the second edge are parallel to a first direction, the third edge and the fourth edge are parallel to a second direction, and the first direction is perpendicular to the second direction; at least two UWB base stations are distributed at different positions in front of or behind the display screen; all or part of the vertical projections of at least two UWB base stations on the display screen are located on the display screen.

[0057] According to the fourth aspect, or any implementation of the fourth aspect, the first direction is a horizontal direction, and the second direction is a vertical direction; or, the first direction is a vertical direction, and the second direction is a horizontal direction.

[0058] According to the fourth aspect, or any implementation manner of the fourth aspect above, at least two UWB base stations include a first UWB base station, a second UWB base station and a third UWB base station; all or part of the vertical projection of the first UWB base station on the display screen is located at the intersection of the third edge and the second edge; all or part of the vertical projection of the second UWB base station on the display screen is located at the intersection of the third edge and the first edge; all or part of the vertical projection of the third UWB base station on the display screen is located at the intersection of the second edge and the fourth edge.

[0059] Exemplarily, the annotation device may be a device with a relatively fixed (or stationary) position, such as a smart TV, a smart screen, a wall clock, a smart air conditioner, etc.

[0060] In the fifth aspect, a method for quickly popping up an electronic device control window is provided. The method is applied to a system, the system includes a mobile device, a marking device, and an electronic device to be marked; the mobile device communicates wirelessly with the marking device and the electronic device; the mobile device includes: an ultra-wideband UWB tag and a direction sensor; the marking device includes: at least two UWB base stations, at least two UWB base stations include a first UWB base station; the electronic device does not include a UWB base station; the method includes: after the mobile device moves to the position of the electronic device, the mobile device receives a first input; the mobile device responds to the first input and records the position of the mobile device; the position of the mobile device is the position of the electronic device; after the mobile device moves to another position and points to the electronic device, the mobile device pops up the first control window of the electronic device. The method provided in the fifth aspect, by marking the position of the electronic device without the UWB chip installed, the position of the electronic device without the UWB chip installed can be located, and then when it is determined that the angle at which the mobile device (such as a smart phone) points to the electronic device meets certain conditions, it can be determined that the mobile device points to the electronic device without the UWB chip installed, and then the control window of the electronic device without the UWB chip installed can be popped up. In this way, the user can conveniently control the electronic device that is not equipped with a UWB chip.

[0061] Exemplarily, the mobile device may be a portable electronic device such as a smart phone, an air mouse, a PDA, a smart remote control, a tablet computer, etc.; the annotation device may be an electronic device such as a smart TV, a smart wall clock, etc.; the electronic device may be a smart air conditioner, a smart lamp, a smart socket, a smart speaker, a smart printer, etc.

[0062] According to the fifth aspect, the method further includes: within a preset time period after the mobile device pops up the first control window of the electronic device, the first control window is displayed on the mobile device; wherein the first control window is used to receive a first user input for controlling the electronic device; if the first user input is not received within the preset time period, the mobile device no longer displays the first control window. In this implementation, even if the user moves or rotates the mobile device, for example, moves the mobile device from a current position to another position, or rotates from a current orientation to another orientation, the control window will continue to be displayed on the mobile device, which can facilitate user use and improve user experience.

[0063] According to the fifth aspect, or any implementation of the fifth aspect above, the mobile device wirelessly communicates with the annotation device and the electronic device through a routing device, and the method also includes: after the mobile device points to the annotation device, the mobile device pops up a second control window of the annotation device.

[0064] According to the fifth aspect, or any implementation of the fifth aspect above, the method also includes: within a preset time length after the second control window of the annotation device pops up on the mobile device, the second control window is displayed on the mobile device; wherein the second control window is used to receive a second user input for controlling the annotation device; if the second user input is not received within the preset time length, the mobile device no longer displays the second control window.

[0065] According to the fifth aspect, or any implementation manner of the fifth aspect above, before the mobile device moves to the position of the electronic device, the method also includes: the mobile device obtains the distance between the first UWB base station and another UWB base station, the distance between the UWB tag and each of the at least two UWB base stations, and the position of the UWB tag; wherein the other UWB base station is a UWB base station other than the first UWB base station among the at least two UWB base stations; the mobile device obtains the pointing direction of the mobile device and the first angle between the vertical line between the mobile device and the display screen based on the distance between the first UWB base station and another UWB base station, the distance between the UWB tag and each of the at least two UWB base stations, and the position of the UWB tag.

[0066] According to the fifth aspect, or any implementation of the fifth aspect above, before the mobile device moves to the position of the electronic device, the method also includes: when the mobile device moves to the first position, the mobile device obtains a first initial azimuth angle and a first initial rolling angle of the mobile device based on the direction sensor; when the mobile device moves to the second position, the mobile device obtains a first real-time azimuth angle and a first real-time rolling angle of the mobile device based on the direction sensor; the mobile device obtains a second azimuth angle in the second direction and a second rolling angle in the first direction based on the first initial azimuth angle, the first initial rolling angle, the first real-time azimuth angle, and the first real-time rolling angle; the first direction is perpendicular to the second direction.

[0067] According to the fifth aspect, or any implementation of the fifth aspect above, the method also includes: the mobile device obtains a second angle between the pointing direction of the mobile device and a vertical line between the mobile device and the display screen based on the second azimuth angle and the second rolling angle.

[0068] According to the fifth aspect, or any implementation of the fifth aspect above, after the mobile device moves to another position and points to the electronic device, the mobile device pops up the first control window of the electronic device; including: when the mobile device moves to another position, and the deviation between the first angle and the second angle is within a preset first range, the mobile device points to the electronic device, and the mobile device pops up the first control window of the electronic device.

[0069] Exemplarily, if the difference between the second angle θ and the first angle α is within a preset difference range, for example, the difference between the first angle and the second angle θ is 0 degrees, it means that the mobile device is pointing at the electronic device, the mobile device pops up a control window of the electronic device, and the mobile device can control the electronic device.

[0070] If the difference between the second angle θ and the first angle α is not within the preset difference range, it means that the mobile device is not pointing to the electronic device, the mobile device will not pop up the control window of the electronic device, and the mobile device cannot control the electronic device.

[0071] According to the fifth aspect, or any implementation of the fifth aspect above, the first angle can be decomposed into a first azimuth angle and a first rolling angle; the first azimuth angle represents the angle between a straight line in the direction in which the mobile device is pointing and a perpendicular line from the mobile device to the marking device in the second direction; the first rolling angle represents the angle between a straight line in the direction in which the mobile device is pointing and a perpendicular line from the mobile device to the marking device in the first direction.

[0072] According to the fifth aspect, or any implementation method of the fifth aspect above, after the mobile device moves to another position and points to the electronic device, the mobile device pops up the first control window of the electronic device; including: when the mobile device moves to another position, and the deviation between the first azimuth angle and the second azimuth angle is within a preset second range, and the deviation between the first scroll angle and the second scroll angle is within a preset third range, the mobile device points to the electronic device, and the mobile device pops up the first control window of the electronic device.

[0073] Exemplarily, if the difference between the first azimuth angle and the second azimuth angle is within a preset first difference range, for example, the first azimuth angle is equal to the second azimuth angle; and the difference between the first rolling angle and the second rolling angle is also within a preset second difference range, for example, the first rolling angle is equal to the second rolling angle, then it means that the mobile device is pointing at the electronic device, the mobile device will pop up a control window of the electronic device, and the mobile device can control the electronic device.

[0074] If the difference between the first azimuth angle and the second azimuth angle is within a preset first difference range, for example, the first azimuth angle is equal to the second azimuth angle; and the difference between the first rolling angle and the second rolling angle is not within a preset second difference range, it means that the mobile device is not pointing at the electronic device, the mobile device will not pop up the control window of the electronic device, and the mobile device cannot control the electronic device.

[0075] If the difference between the first azimuth angle and the second azimuth angle is not within the preset first difference range, for example, the first azimuth angle is equal to the second azimuth angle; and the difference between the first rolling angle and the second rolling angle is within the preset second difference range, then it means that the mobile device is not pointing at the electronic device, the mobile device will not pop up the control window of the electronic device, and the mobile device cannot control the electronic device.

[0076] If the difference between the first azimuth angle and the second azimuth angle is not within the preset first difference range, for example, the first azimuth angle is equal to the second azimuth angle; and the difference between the first rolling angle and the second rolling angle is also not within the preset second difference range, then it means that the mobile device is not pointing at the electronic device, the mobile device will not pop up the control window of the electronic device, and the mobile device cannot control the electronic device.

[0077] According to the fifth aspect, or any implementation manner of the fifth aspect above, the mobile device obtains the distance between the first UWB base station and each other UWB base station, the distance between the UWB tag and each UWB base station of at least two UWB base stations, and the position of the UWB tag; including: the mobile device sends a ranging message through the UWB tag; the annotation device receives the ranging message through each UWB base station of at least two UWB base stations; in response to the ranging message, the annotation device sends a ranging response message to the mobile device through each UWB base station of at least two UWB base stations; the mobile device receives the ranging response message from each UWB base station of at least two UWB base stations; the mobile device obtains the distance between the UWB tag and each UWB base station of at least two UWB base stations in response to the ranging response message of at least two UWB base stations; the annotation device sends a message to the mobile device through any one UWB base station of at least two UWB base stations, and the message includes the distance between the first UWB base station and each other UWB base station. The mobile device receives a message from the labeling device; the mobile device obtains the distance between the first UWB base station and each other UWB base station from the message; the mobile device selects the position of the first UWB base station among at least two UWB base stations as the coordinate origin of the three-dimensional coordinate system; based on the coordinate origin, the mobile device obtains the position coordinates of another UWB base station among at least two UWB base stations, as well as the position coordinates of the UWB tag.

[0078] According to the fifth aspect, or any implementation of the fifth aspect above, the first input is used to select an identification of an electronic device; the identification of the electronic device corresponds to a parameter of the electronic device.

[0079] According to the fifth aspect, or any implementation of the fifth aspect above, the parameter includes one or more of the media access control MAC address, Internet Protocol IP address, serial number SN or device identification of the electronic device.

[0080] According to the fifth aspect, or any implementation of the fifth aspect above, the direction sensor is an inertial measurement unit IMU; the IMU is a 9-axis IMU, and the 9-axis IMU includes an acceleration sensor, a gyroscope sensor, and a magnetic sensor; or, the IMU is a 6-axis IMU, and the 6-axis IMU includes an acceleration sensor and a gyroscope sensor.

[0081] According to the fifth aspect, or any implementation of the fifth aspect, the first direction is a horizontal direction, and the second direction is a vertical direction; or, the first direction is a vertical direction, and the second direction is a horizontal direction.

[0082] According to the fifth aspect, or any implementation manner of the fifth aspect above, the marking device also includes: a display screen; the display screen includes a first edge, a second edge, a third edge and a fourth edge; the first edge is parallel to the second edge, and the third edge is parallel to the fourth edge; the first edge and the second edge are parallel to a first direction, the third edge and the fourth edge are parallel to a second direction, and the first direction is perpendicular to the second direction; at least two UWB base stations are distributed at different positions in front of or behind the display screen; all or part of the vertical projections of at least two UWB base stations on the display screen are located on the display screen.

[0083] According to the fifth aspect, or any implementation manner of the fifth aspect above, at least two UWB base stations include a first UWB base station, a second UWB base station and a third UWB base station; all or part of the vertical projection of the first UWB base station on the display screen is located at the intersection of the third edge and the second edge; all or part of the vertical projection of the second UWB base station on the display screen is located at the intersection of the third edge and the first edge; all or part of the vertical projection of the third UWB base station on the display screen is located at the intersection of the second edge and the fourth edge.

[0084] In the sixth aspect, a system for quickly popping up an electronic device control window is provided. The system includes a mobile device, a labeling device, and an electronic device to be labeled; the mobile device communicates wirelessly with the labeling device and the electronic device; the mobile device includes: an ultra-wideband UWB tag and a direction sensor; the labeling device includes: at least two UWB base stations, at least two UWB base stations including a first UWB base station; the electronic device does not include a UWB base station. The mobile device is used to execute the steps executed by the mobile device in the fifth aspect above, or any one of the implementations of the fifth aspect above, and the labeling device is used to execute the steps executed by the labeling device in the fifth aspect above, or any one of the implementations of the fifth aspect above.

[0085] Other implementations of the sixth aspect correspond to other implementations of the fifth aspect and will not be described in detail here.

[0086] The technical effects corresponding to the sixth aspect and any one of the implementation methods of the sixth aspect can be found in the technical effects corresponding to the above-mentioned fifth aspect and any one of the implementation methods of the fifth aspect, and will not be repeated here.

[0087] In a seventh aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program (also referred to as an instruction or code), and when the computer program is executed by a mobile device, the mobile device executes the method of the third aspect and any one of the implementations of the third aspect.

[0088] The technical effects corresponding to the seventh aspect and any one of the implementation methods of the seventh aspect can be found in the technical effects corresponding to the above-mentioned third aspect and any one of the implementation methods of the third aspect, and will not be repeated here.

[0089] In an eighth aspect, a computer-readable storage medium is provided. The computer-readable storage medium stores a computer program (also referred to as an instruction or code), and when the computer program is executed by a marking device, the marking device executes the method of the fourth aspect and any one of the implementations of the fourth aspect.

[0090] The technical effects corresponding to the eighth aspect and any one of the implementation methods of the eighth aspect can be referred to the technical effects corresponding to the above-mentioned fourth aspect and any one of the implementation methods of the fourth aspect, and will not be repeated here.

[0091] In a ninth aspect, a chip is provided, which includes a processor and a memory, wherein the processor is configured to read and execute a computer program stored in the memory to execute the method of the third aspect and any one of the implementation modes of the third aspect.

[0092] The technical effects corresponding to the ninth aspect and any one of the implementation methods of the ninth aspect can be found in the technical effects corresponding to the above-mentioned third aspect and any one of the implementation methods of the third aspect, and will not be repeated here.

[0093] In a tenth aspect, a chip is provided, which includes a processor and a memory, wherein the processor is used to read and execute a computer program stored in the memory to execute the method of the fourth aspect and any one of the implementation modes of the fourth aspect.

[0094] The technical effects corresponding to the tenth aspect and any one of the implementation methods of the tenth aspect can be found in the technical effects corresponding to the above-mentioned fourth aspect and any one of the implementation methods of the fourth aspect, and will not be repeated here.

[0095] In an eleventh aspect, a computer program product is provided, which includes a computer program (also referred to as an instruction or code), and when the computer program is executed by a mobile device, the mobile device executes the method of the third aspect and any one of the implementations of the third aspect.

[0096] The technical effects corresponding to the eleventh aspect and any one of the implementation methods of the eleventh aspect can be referred to the technical effects corresponding to the above-mentioned third aspect and any one of the implementation methods of the third aspect, and will not be repeated here.

[0097] In a twelfth aspect, a computer program product is provided, which includes a computer program (also referred to as instructions or codes), which, when executed by a marking device, causes the marking device to execute the method of the fourth aspect and any one of the implementations of the fourth aspect.

[0098] The technical effects corresponding to the twelfth aspect and any one of the implementation methods of the twelfth aspect can be found in the technical effects corresponding to the above-mentioned fourth aspect and any one of the implementation methods of the fourth aspect, and will not be repeated here. BRIEF DESCRIPTION OF THE DRAWINGS

[0099] Figure 1 A schematic diagram of a scenario of a method for quickly popping up a control window of an electronic device provided in an embodiment of the present application;

[0100] Figure 2 A schematic diagram of the hardware structure of a mobile device provided in an embodiment of the present application;

[0101] Figure 3 A schematic diagram of the hardware structure of the marking device provided in the embodiment of the present application;

[0102] Figure 4 A partial flow chart of a method for quickly popping up an electronic device control window provided in an embodiment of the present application;

[0103] Figure 5A In a method for quickly popping up an electronic device control window provided in an embodiment of the present application, Figure 4 A flowchart of a portion of the process shown previously executed;

[0104] Figure 5B A partial flow chart of a method for quickly popping up an electronic device control window provided in an embodiment of the present application;

[0105] Figure 6 In a method for quickly popping up an electronic device control window provided in an embodiment of the present application, Figure 4 The process steps shown are flowcharts related to the internal implementation;

[0106] Figure 7 Schematic diagram of the principle of the provided reference coordinate axis;

[0107] Figure 8 Schematic diagram of the principle of the azimuth provided;

[0108] Fig. 9 Schematic diagram of the principle of the tilt angle provided;

[0109] Fig.10 Schematic diagram of the principle of the roll angle provided;

[0110] Fig.11 A schematic diagram of the x-axis, y-axis and z-axis on a mobile device is provided for an embodiment of the present application;

[0111] Fig.12 A schematic diagram of a mobile device provided in an embodiment of the present application facing a display screen of a labeling device;

[0112] Fig.13 A schematic diagram of a rolling angle σ between a display screen of a mobile device and a marking device in an embodiment of the present application;

[0113] Fig.14 A schematic diagram of an azimuth angle ψ between a display screen of a mobile device and a marking device in an embodiment of the present application;

[0114] Fig.15 A schematic diagram of the direction of the pointing direction of the mobile device and the second angle θ between the mobile device and the display screen of the marking device in the embodiment of the present application;

[0115] Fig.16 is a schematic diagram of the first angle α in the embodiment of the present application;

[0116] Fig.17 A schematic diagram of the relative positions of a mobile device, a marking device, and an electronic device in an embodiment of the present application;

[0117] Fig.18Schematic diagram of the principle of determining the distance between two devices using UWB technology and two-way time-of-flight method;

[0118] Fig.19 A schematic diagram of determining the position coordinates of the electronic device and the first angle α after the mobile device is moved to the position of the electronic device in the present application;

[0119] Fig. 20 This is a schematic diagram of an interface displayed on a mobile device when labeling an electronic device in an embodiment of the present application;

[0120] Fig.21 is a schematic diagram of the calculation of the first angle α in the embodiment of the present application;

[0121] Fig. 22 This is a schematic diagram of an embodiment of the present application in which the second angle θ is the same as the first angle α and the mobile device is pointed at the electronic device;

[0122] Fig.23 This is a schematic diagram of an embodiment of the present application in which the difference between the second angle θ and the first angle α is not within a preset difference range, and the mobile device is not pointing to the electronic device;

[0123] Fig.24 This is a schematic diagram of a mobile device pointing to an electronic device in both vertical and horizontal directions in an embodiment of the present application;

[0124] Fig.25 This is a schematic diagram of an embodiment of the present application in which the mobile device is pointed at the electronic device in a vertical direction but not pointed at the electronic device in a horizontal direction;

[0125] Fig.26 This is a schematic diagram of an embodiment of the present application in which the mobile device is pointed at the electronic device in a vertical direction but not pointed at the electronic device in a horizontal direction;

[0126] Fig. 27 This is a schematic diagram of an embodiment of the present application in which the mobile device is not pointed at the electronic device in both the vertical direction and the horizontal direction;

[0127] Fig.28 This is a schematic diagram of an interface displayed by a mobile device when the mobile device points to an electronic device in an embodiment of the present application; DETAILED DESCRIPTION

[0128] The technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application.

[0129] The terms used in the following embodiments are only for the purpose of describing specific embodiments, and are not intended to be used as limitations on the present application. As used in the specification and appended claims of the present application, the singular expressions "a", "said", "above", "the" and "this" are intended to also include expressions such as "one or more", unless there is a clear contrary indication in the context. It should also be understood that in the embodiments of the present application, "one or more" refers to one, two or more (including two); "and / or" describes the association relationship of associated objects, indicating that three relationships may exist; for example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The character " / " generally indicates that the objects associated before and after are in an "or" relationship.

[0130] References to "one embodiment" or "some embodiments" etc. described in this specification mean that a particular feature, structure or characteristic described in conjunction with the embodiment is included in one or more embodiments of the present application. Thus, the phrases "in one embodiment", "in some embodiments", "in some other embodiments", "in some other embodiments", etc. that appear at different places in this specification do not necessarily refer to the same embodiment, but mean "one or more but not all embodiments", unless otherwise specifically emphasized in other ways. The terms "including", "comprising", "having" and their variations all mean "including but not limited to", unless otherwise specifically emphasized in other ways.

[0131] The multiple involved in the embodiments of the present application means greater than or equal to two. It should be noted that in the description of the embodiments of the present application, the words "first", "second", etc. are only used for the purpose of distinguishing the description, and cannot be understood as indicating or implying relative importance, nor can they be understood as indicating or implying an order.

[0132] There are generally many IoT devices in home scenes, office scenes or other scenes. Among these IoT devices, some are equipped with ultra wideband (UWB) chips (UWB chips can be UWB base stations or UWB tags), and some are not equipped with UWB chips. Users can use mobile devices (such as smart phones, air mice, smart remote controls, etc.) equipped with UWB chips and direction sensors to achieve directional control of IoT devices equipped with UWB chips. For example, when a user points a mobile device to an IoT device (such as a large-screen device) equipped with a UWB chip, the mobile device senses that the mobile device has pointed to the IoT device, and then displays a control interface for controlling the IoT device to the user. The user uses the control interface displayed on the mobile device to control the device, and can achieve the function of controlling the IoT device equipped with the UWB chip. However, this directional control method requires that the UWB chip also be deployed on the target device being pointed to. Because many old electronic devices such as air conditioners, washing machines, sockets, smart lamps, printers, etc. in homes or offices do not have UWB chips installed on them. In this way, when a user points a mobile electronic device at an electronic device that does not have a UWB chip installed, the mobile device cannot sense whether it is pointing at the electronic device. Therefore, the mobile device cannot display the control interface of the electronic device to the user, resulting in the user being unable to control the electronic device that does not have a UWB chip installed through the mobile device, resulting in a poor user experience.

[0133] In view of this, the present application provides a method and system for quickly popping up an electronic device control window, by marking the position of the electronic device without the UWB chip installed, so that the position of the electronic device without the UWB chip installed can be located, and then when it is determined that the angle at which the mobile device (such as a smart phone) points to the electronic device meets certain conditions, it can be determined that the mobile device points to the electronic device without the UWB chip installed, and then the control window of the electronic device without the UWB chip installed is popped up. In this way, the user can conveniently control the electronic device without the UWB chip installed.

[0134] Figure 1 A schematic diagram of a method for quickly popping up an electronic device control window provided by an embodiment of the present application. Figure 1As shown, the mobile device 100 is a portable electronic device equipped with a UWB tag 101 and a direction sensor 102. The annotation device 200 is an IoT device equipped with a UWB base station. The electronic device 300 is an IoT device without a UWB circuit installed. Among them, the UWB base station and the UWB tag can be understood as two devices or modules required to utilize the UWB ranging technology. Through the ranging message sent between the UWB base station and the UWB tag, the distance between the UWB base station and the UWB tag can be determined, thereby determining the distance between the mobile device 100 and the annotation device 200.

[0135] Exemplarily, the mobile device 100 may be a portable electronic device such as a smart phone, an air mouse, a personal digital assistant (PDA), a smart remote control, a tablet computer, etc.; the annotation device 200 may be an electronic device such as a smart TV and a smart wall clock; the electronic device 300 may be a smart air conditioner, a smart lamp, a smart socket, a smart speaker, a smart printer, etc.

[0136] It should be understood that in the embodiment of the present application, the number of UWB base stations on the labeling device 200 is greater than or equal to 2. Figure 1 In the illustrated scenario, the annotation device 200 further includes a display screen 211. The first UWB base station 212, the second UWB base station 213, and the third UWB base station 214 on the annotation device 200 may be distributed at different positions in front of or behind the display screen 211, and all or part of the vertical projections of the first UWB base station 212, the second UWB base station 213, and the third UWB base station 214 on the display screen 211 are located on the display screen 211. Figure 1 The display screen of the labeling device 200 shown is a rectangle for illustration.

[0137] The rectangular display screen 211 includes a first edge, a second edge, a third edge, and a fourth edge. The first edge is parallel to the second edge, the third edge is parallel to the fourth edge, the first edge and the second edge are parallel to the first direction, the third edge and the fourth edge are parallel to the second direction, and the first direction is perpendicular to the second direction. In the following description, the first direction is the horizontal direction, the second direction is the vertical direction, the first edge is the upper edge (or upper boundary) of the display screen, the second edge is the lower edge (or lower boundary) of the display screen, the third edge is the left edge (or left boundary) of the display screen, and the fourth edge is the right edge (or right boundary) of the display screen.

[0138] All or part of the vertical projection of the first UWB base station 212 on the display screen 211 is located at the intersection of the third edge and the second edge; all or part of the vertical projection of the second UWB base station 213 on the display screen 211 is located at the intersection of the fourth edge and the second edge; all or part of the vertical projection of the third UWB base station 214 on the display screen 211 is located at the intersection of the first edge and the third edge.

[0139] It should be noted that the annotation device 200 is deployed with two UWB base stations, and the technical solution of the present application can also be implemented. Just for the sake of convenience, the annotation device 200 is deployed with a first UWB base station 212, a second UWB base station 213, and a third UWB base station 214 as an example. It should be understood that in the embodiment of the present application, the number of UWB base stations on the display screen 211 can be greater than or equal to 2; for example, 4, 5, etc. As long as the left and right boundary positions and the upper and lower boundary positions of the display screen can be determined using multiple UWB base stations on the display screen, the embodiment of the present application does not limit this.

[0140] It should be understood that in the example of the present application, the positions of the first UWB base station 212, the second UWB base station 213 and the third UWB base station 214 on the display screen 211 can be interchanged, and the present application does not limit this.

[0141] It should be understood that in the embodiment of the present application, there is no limitation on the specific shape of the display screen of the marking device 200. For example, the display screen may be a rectangle, or a parallelogram, hexagon, octagon, decagon, circle, etc. The present application does not limit this.

[0142] Optionally, in the embodiment of the present application, the direction sensor 102 on the mobile device 100 may include an inertial measurement unit (IMU). For example, the direction sensor may be a 9-axis IMU composed of an accelerometer sensor, a gyroscope sensor, and a magnetic sensor. Alternatively, the direction sensor may also be a 6-axis IMU composed of an accelerometer sensor, a gyroscope sensor, etc. The present application does not limit this.

[0143] It should be understood that Figure 1 In the illustrated scene, the electronic device 300 is only an illustrative example. In addition to the electronic device 300, the scene may also include other electronic devices ( Figure 1 Other electronic devices may also be annotated by the mobile device 100 and the annotating device 200.

[0144] It should be understood that in the embodiment of the present application, the mobile device 100, the annotation device 200 and the electronic device 300 can establish communication connections between each other through a variety of short-range wireless communication methods such as Wi-Fi connection, Bluetooth, routing devices, etc. In this way, messages can be transmitted between the two devices.

[0145] For example, Figure 2 FIG. 1 shows the hardware structure of the mobile device 100 provided in the embodiment of the present application. Figure 2 As shown, the mobile device 100 includes a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, an antenna, a wireless communication module 150, a sensor module 160, and a UWB tag 170. The sensor module 160 may include a direction sensor. Optionally, the mobile device 100 may also include a display screen. The display screen is used to display the control interface of the electronic device to the user so that the user can control the electronic device.

[0146] Understandably, Figure 2 The illustrated structure does not constitute a specific limitation on the mobile device 100. In other embodiments of the present application, the mobile device 100 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0147] The processor 110 may include one or more processing units. For example, the processor 110 may include an application processor (AP), a modem processor, a graphics processor (GPU), an image signal processor (ISP), a controller, a video codec, a digital signal processor (DSP), a baseband processor, and / or a neural-network processing unit (NPU). Different processing units may be independent components or integrated into one or more processors. In some embodiments, the mobile device 100 may also include one or more processors 110. The controller may generate an operation control signal based on the instruction opcode and the timing signal to complete the control of fetching and executing instructions.

[0148] In some embodiments, the processor 110 may include one or more interfaces. The interface may include an inter-integrated circuit (I2C) interface, an integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a SIM card interface, and / or a USB interface, etc. Among them, the USB interface 130 is an interface that complies with the USB standard specification, and specifically can be a Mini USB interface, a Micro USB interface, a USB Type C interface, etc. The USB interface 130 can be used to connect a charger to charge the mobile device 100, and can also be used to transmit data between the mobile device 100 and peripheral devices.

[0149] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the mobile device 100. In other embodiments of the present application, the mobile device 100 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0150] The wireless communication function of the mobile device 100 can be implemented through an antenna and a wireless communication module 150. The wireless communication module 150 can provide wireless communication solutions including Wi-Fi (including Wi-Fi sensing and Wi-Fi AP), Bluetooth (BT), wireless data transmission modules (for example, 433MHz, 868MHz, 915MHz) and the like applied on the mobile device 100. The wireless communication module 150 can be one or more devices integrating at least one communication processing module. The wireless communication module 150 receives electromagnetic waves via an antenna, filters and frequency modulates electromagnetic wave signals, and sends the processed signals to the processor 110. The wireless communication module 150 can also receive a signal to be sent from the processor 110, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation through the antenna. Optionally, the above-mentioned antenna can be one antenna or multiple antennas. In the case of multiple antennas, only some of the multiple antennas can be used.

[0151] The external memory interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the mobile device 100. The external memory card communicates with the processor 110 through the external memory interface 120 to implement a data storage function, such as storing music, video and other files in the external memory card.

[0152] The internal memory 121 can be used to store one or more computer programs, which include instructions. The processor 110 can execute the above instructions stored in the internal memory 121, so that the mobile device 100 executes the method for quickly popping up the electronic device control window provided in some embodiments of the present application, as well as various applications and data processing. The internal memory 121 may include a code storage area and a data storage area. Among them, the code storage area can store an operating system. The data storage area can store data created during the use of the mobile device 100, etc. In addition, the internal memory 121 may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more disk storage components, flash memory components, universal flash storage (universal flash storage, UFS), etc. In some embodiments, the processor 110 can execute the method for quickly popping up the electronic device control window provided in the embodiment of the present application, as well as other applications and data processing by running the instructions stored in the internal memory 121, and / or the instructions stored in the memory provided in the processor 110.

[0153] For example, Figure 3 2 shows the hardware structure of the labeling device 200 provided by the present application. Figure 3 As shown, the annotation device 200 may include a processor 210, an external memory interface 230, an internal memory 231, a universal serial bus (USB) interface 240, a charging management module 220, a power management module 221, a battery 222, an antenna 1, an antenna 2, a wireless communication module 260, a sensor module 270, and at least two UWB base stations. Optionally, the annotation device 200 may also include a display screen 280. If the annotation device 200 includes a display screen 280, at least two UWB base stations may be deployed on the display screen 280. Exemplarily, when three UWB base stations are deployed on the display screen 280, the positions of the three UWB base stations may be Figure 1 If the marking device 200 does not include the display screen 280, at least two UWB base stations may be set at other locations of the marking device 200, and this application does not limit this.

[0154] Understandably, Figure 3The illustrated structure does not constitute a specific limitation on the marking device 200. In other embodiments of the present application, the marking device 200 may include more or fewer components than shown in the figure, or combine some components, or split some components, or arrange the components differently. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0155] It is understandable that the interface connection relationship between the modules illustrated in the embodiment of the present application is only a schematic illustration and does not constitute a structural limitation on the labeling device 200. In other embodiments of the present application, the labeling device 200 may also adopt different interface connection methods in the above embodiments, or a combination of multiple interface connection methods.

[0156] The mobile device 100 and the labeling device 200 provided in this embodiment cooperate with each other to implement a method for quickly popping up a control window of an electronic device.

[0157] Figure 4 A partial flow chart of a method for quickly popping up an electronic device control window provided in an embodiment of the present application. Figure 4 Before the process shown, the mobile device 100 has established a communication connection with the annotation device 200, established a three-dimensional coordinate system, obtained the position coordinates of the three UWB base stations in the annotation device 200, and obtained the real-time position coordinates of the mobile device 100. In other words, the method for quickly popping up an electronic device control window provided by the embodiment of the present application includes: Figure 4 The process shown and Figure 4 The steps before the process shown. Figure 4 As shown, the method includes: S410, after the mobile device moves to the position of the electronic device to be marked, a first input is received, and the position coordinates of the mobile device at this time are recorded as the position coordinates of the electronic device; S420, after the mobile device points to the electronic device, the mobile device pops up a control window of the electronic device.

[0158] Optionally, a step of calibrating the orientation of the mobile device 100 may be added before S410 ; the step is used to calibrate the orientation of the mobile device 100 , and even to calibrate the direction of a vertical line from the mobile device 100 to the marking device 200 .

[0159] Exemplarily, after the user holds the mobile device 100 and makes the direction of the mobile device 100 "vertical" (the user artificially thinks it is vertical, but it may not be vertical in reality) to the display screen 211 of the labeling device 200, the angle of the mobile device 100 can be adjusted so that the prompt information fed back on the mobile device 100 through the direction sensor of the mobile device 100 informs the user that the mobile device 100 is vertical (verticality is confirmed by measuring the angle) to the display screen 211 of the labeling device 200 at this time. The mobile device 100 can define the straight line direction where the mobile device 100 is pointing at this time as the perpendicular direction from the mobile device 100 to the labeling device 200.

[0160] Exemplarily, after the user holds the mobile device 100 and makes the direction of the mobile device 100 "vertical" (the user artificially thinks it is vertical, but it may not be vertical in reality) to the display screen 211 of the labeling device 200, due to the error of the artificially adjusted verticality, the mobile device 100 may determine the direction of the straight line where the mobile device 100 is pointing at this time as the first direction; thereafter, the mobile device 100 may determine the direction of the straight line where the subsequent mobile device 100 is pointing as the second direction; and according to the deviation angle between the second direction and the first direction, determine the angle between the straight line where the subsequent mobile device 100 is pointing and the perpendicular line from the mobile device 100 to the labeling device 200.

[0161] Before S410, it can be executed Figure 5B The process shown. Figure 5B The process shown includes: calibrating the orientation of the mobile device 100 and determining the direction in which the mobile device 100 is pointing and a second angle θ of a vertical line between the mobile device 100 and the display screen 211 .

[0162] Figure 5B The process shown specifically includes:

[0163] S501b: After the pointing direction of the mobile device is perpendicular to the marking device, first measurement information of the direction sensor is obtained.

[0164] Specifically, the verticality in S501b is what the user thinks is “vertical”; in fact, it may not be vertical.

[0165] In one implementation, the first measurement information includes an angle between a straight line in which the mobile device 100 is pointing and a fixed direction.

[0166] S502b: After the pointing direction of the mobile device is not perpendicular to the marking device, second measurement information of the direction sensor is obtained.

[0167] In one implementation, the second measurement information includes an angle between a straight line where the mobile device 100 is pointing and a fixed direction. The fixed direction of S502b is the same as the fixed direction of S501b.

[0168] S503b: Acquire a second azimuth angle and a second rolling angle according to the second measurement information and the first measurement information.

[0169] S504b: Acquire a second angle θ according to the second azimuth angle and the second rolling angle.

[0170] like Figure 5B In S501b shown, after the display screen 211 of the device 200 is vertically annotated with the pointing direction of the mobile device (or the straight line where the pointing direction of the mobile device is located), the first measurement information (initial azimuth and roll angle) of the direction sensor on the mobile device 100 can be obtained. For example, when the user holds the mobile device 100 so that the pointing direction of the mobile device 100 is perpendicular to the display screen 211 of the annotated device 200, the user can rotate the mobile device 100 so that the upper surface of the mobile device 100 (i.e., the surface that the user faces when using the mobile device 100) is parallel to the upper and lower edges of the display screen 211 and perpendicular to the left and right edges of the display screen. At this time, the angle between the upper surface of the mobile device 100 and the upper and lower edges of the display screen 211 is 0 degrees, the angle between the upper surface of the smart phone and the left and right edges of the display screen is 90 degrees, the mobile device 100 is perpendicular to the display screen 211, and the pointing direction of the mobile device 100 is perpendicular to the display screen 211 of the annotated device 200.

[0171] Optionally, the mobile device 100 may also prompt the user based on the measurement information of the direction sensor. For example, the user is prompted on the display interface of the mobile device 100 whether the direction in which the mobile device 100 is pointed is perpendicular to the display screen 211 of the marking device 200, so that the user makes adjustments according to the prompt, and finally the direction in which the mobile device 100 is pointed is perpendicular to the display screen 211 of the marking device 200.

[0172] It should be understood that in the embodiment of the present application, S501b is an optional step. For example, if the mobile device 100 has been calibrated at the factory, that is, the above-mentioned S501b has been executed, then the user is actually using it, and the mobile device 100 may not execute S501b, and directly execute S502b-S504b. After the user rotates the mobile device 100, the pointing direction of the mobile device 100 will change, and the pointing direction of the mobile device 100 will not be perpendicular to the display screen 211 of the device 200. The process of S502b-S504b can be understood that after the mobile device 100 is rotated, the actual azimuth and roll angle (second measurement information) of the mobile device are obtained in real time according to the direction sensor on the mobile device, and the actual rotation angle of the mobile device is calculated according to the actual azimuth and roll angle, and the initial azimuth and roll angle (first measurement information), and the actual rotation angle of the mobile device is obtained, and the actual rotation angle includes the second azimuth and the second roll angle. The second azimuth angle can be understood as the angle between the pointing direction of the mobile device 100 and the vertical line between the mobile device 100 and the display screen 211 in the second direction. The second rolling angle can be understood as the angle between the pointing direction of the mobile device 100 and the vertical line between the mobile device 100 and the display screen 211 in the first direction. The first direction and the second direction are perpendicular. For example, the first direction is the horizontal direction of the display screen 211, and the second direction is the vertical direction of the display screen 211.

[0173] In S504b , the mobile device may determine a second angle θ according to the second azimuth angle and the second rolling angle, where the second angle θ is the angle between the pointing direction of the mobile device 100 and a vertical line between the mobile device 100 and the display screen 211 .

[0174] In the embodiment of the present application, the second angle θ can be decomposed into a second azimuth angle and a second rolling angle. The second azimuth angle can be understood as the component of the second angle θ in the second direction, and the second rolling angle can be understood as the component of the second angle θ in the first direction.

[0175] It should be understood that Figure 5B The process shown is executed in real time and continuously. Figure 5B In the process shown, S501b may be executed only once, while S502b-S504b are executed in real time and continuously.

[0176] Before S410, you can also execute Figure 5A The process shown. Figure 5A The process shown is used to obtain the distance between the UWB tag on the mobile device 100 and each UWB base station on the annotation device 200, as well as the distance between every two UWB base stations on the annotation device, and based on these distances, determine the position coordinates of the UWB tag, the first UWB base station, the second UWB base station, and the third UWB base station in the three-dimensional coordinate system.

[0177] Figure 5A The process shown includes:

[0178] S501a. The mobile device sends ranging messages to the first UWB base station, the second UWB base station and the third UWB base station on the marked device respectively through the UWB tag.

[0179] S502a: The annotation device receives the ranging message through the first UWB base station, the second UWB base station, and the third UWB base station respectively.

[0180] S503a, after the first UWB base station, the second UWB base station and the third UWB base station on the marking device respectively receive the ranging message, the marking device sends a ranging response message to the UWB tag on the mobile device through the first UWB base station, the second UWB base station and the third UWB base station respectively.

[0181] S504a, the mobile device determines the distance S3 between the first UWB base station and the UWB tag, the distance S4 between the second UWB base station and the UWB tag, and the distance S5 between the third UWB base station and the UWB tag according to the multiple ranging response messages and the multiple ranging messages.

[0182] S505a: The marking device determines a distance S1 between the first UWB base station and the second UWB base station, and a distance S2 between the first UWB base station and the third UWB base station.

[0183] S506a: The marking device sends the distance S1 between the first UWB base station and the second UWB base station, and the distance S2 between the first UWB base station and the third UWB base station to the mobile device.

[0184] S507a, the mobile device uses the position of the first UWB base station as the coordinate origin, and determines the position coordinates of the second UWB base station, the third UWB base station, and the UWB tag in the three-dimensional coordinate system according to S1, S2, S3, S4, and S5.

[0185] Alternatively, in S507a, the mobile device may also use the position of the second UWB base station or the third UWB base station as the coordinate origin; using the position of the first UWB base station as the coordinate origin is only an illustrative example. That is, the mobile device may use the position of any one of the at least two UWB base stations on the marking device as the coordinate origin. This application does not limit this.

[0186] It should be understood that in the embodiments of the present application, Figure 5A The process shown can be executed only once, for example, when the mobile device and the annotation device are connected after being powered on for the first time, and it does not need to be executed again subsequently.

[0187] It should also be understood that in an embodiment of the present application, S503a and S506a can also be combined in one message and sent to a mobile device. That is, the labeling device can send a ranging response message to the mobile device, and the ranging response message includes: the distance S1 between the first UWB base station and the second UWB base station, and the distance S2 between the first UWB base station and the third UWB base station, so that S506a does not need to be executed, thereby saving signaling overhead. If S503a and S506a are not combined, that is, if both S503a and S506a are to be executed, the mobile device can receive the ranging response message more quickly, shortening the time taken by the mobile device to determine the distance S3, the distance S4, and the distance S5, thereby ensuring the timeliness of communication.

[0188] It should also be understood that in the embodiments of the present application, Figure 5A The process shown and Figure 5B There is no restriction on the order of execution of the processes shown. Figure 5A The process shown and Figure 5B The processes shown can be executed simultaneously or sequentially, and the present application embodiment does not limit this.

[0189] It should also be understood that in the embodiments of the present application, Figure 5A The process shown and Figure 5B The processes shown are all in Figure 4 Execute before the steps shown.

[0190] Figure 6 In a method for quickly popping up an electronic device control window provided in an embodiment of the present application, Figure 4 The process steps shown are flowcharts related to internal implementation. Figure 6 The process shown is a specific process for a mobile device to determine whether to pop up an electronic device control window. Figure 6 The process shown in Figure 5A The process shown and Figure 5B Executed after the process shown. Figure 6 The process shown includes:

[0191] S601: After the mobile device moves to the position of the electronic device to be marked, a first input is received, and the position coordinates of the mobile device at this time are recorded as the position coordinates of the electronic device.

[0192] S602, the mobile device determines a first angle α according to S1, S2, S3, S4, S5 and the position coordinates of the first UWB base station, the second UWB base station, the third UWB base station, the UWB tag and the position coordinates of the electronic device. The first angle α represents the angle between a straight line in the direction in which the mobile device is pointing when the mobile device is pointing at the electronic device and a vertical line between the display screen of the mobile device and the marking device.

[0193] S603: When the difference between the first angle α and the second angle θ is within a preset range, the mobile device pops up a control window of the electronic device.

[0194] It should be understood that Figure 6 In the process shown, S601 is executed according to the user's first input, and is not real-time and always executed. However, S602 and S603 are real-time and always executed. S602 and S603 can be understood as the specific process of S420. The first input can be a user input, such as voice input, touch input (including but not limited to long press, click, slide after selection, etc.), etc.

[0195] Specifically, in S603, there are two possible implementation methods:

[0196] The first implementation method: the mobile device 100 determines the difference between the first angle α and the second angle θ. If the difference is within a preset difference range, for example, the difference between the first angle α and the second angle θ is 0 degrees, it means that the mobile device 100 is pointing to the electronic device 300, the mobile device 100 can control the electronic device 300, and the mobile device 100 pops up a control window of the electronic device 300. If the difference is not within the preset difference range, it means that the mobile device 100 is not pointing to the electronic device 300, and the mobile device 100 cannot control the electronic device 300.

[0197] Second implementation: The mobile device can also determine the actual azimuth (i.e., the second azimuth) and rolling angle (i.e., the second rolling angle) of the mobile device based on the second angle θ. Furthermore, the mobile device determines the azimuth component (i.e., the first azimuth) and rolling component (i.e., the first rolling angle) of the first angle α based on the first angle α; the rolling component indicates the angle between the straight line in the direction in which the mobile device points when the mobile device points to the electronic device and the perpendicular line between the mobile device and the display screen in the first direction; the azimuth component indicates the angle between the straight line in the direction in which the mobile device points when the mobile device points to the electronic device and the perpendicular line between the mobile device and the display screen in the second direction.

[0198] If the difference between the azimuth angle and the azimuth angle component is within a preset range, for example, the value of the azimuth angle and the value of the azimuth angle component are equal, and the difference between the value of the roll angle and the value of the roll angle component is also within the preset range, for example, the value of the roll angle and the value of the roll angle component are equal, then it means that the mobile device is pointing at the electronic device, the mobile device pops up a control window of the electronic device, and the mobile device can control the electronic device.

[0199] If the difference between the azimuth angle and the azimuth angle component is within a preset range, for example, the azimuth angle value and the azimuth angle component value are equal, but the difference between the roll angle and the roll angle component is not within the preset range, it means that the mobile device is not pointing at the electronic device, the mobile device will not pop up the control window of the electronic device, and the mobile device cannot control the electronic device.

[0200] If the difference between the azimuth angle and the azimuth angle component is not within the preset range, and the difference between the roll angle and the roll angle component is within the preset range, it means that the mobile device is not pointing to the electronic device, and the mobile device cannot control the electronic device.

[0201] If the difference between the azimuth angle and the azimuth angle component is not within the preset range, and the difference between the roll angle and the roll angle component is also not within the preset range, it means that the mobile device is not pointing at the electronic device, the mobile device will not pop up the control window of the electronic device, and the mobile device cannot control the electronic device.

[0202] Optional, in Figure 5A The process shown or Figure 5B Before the process shown, the method 400 may also include: the mobile device sends instruction information to the annotation device, the instruction information is used to instruct the use of the mobile device to control the electronic device. In other words, before the mobile device executes S410, the mobile device can communicate with the annotation device, and the information or message transmitted during the communication process is used to notify the annotation device that the mobile device needs to control the electronic device.

[0203] In the embodiment of the present application, the vertical line between the mobile device and the plane of the display screen (or the mobile device to the display screen) can be understood as: from any point on the mobile device (for example, any point on the UWB tag on the mobile device, or the center position point on the top of the mobile device, etc.) as an endpoint of the line segment (denoted as point Q), a vertical line is drawn to the plane of the display screen, and the intersection of the vertical line and the display screen is the other endpoint of the line segment (denoted as point P). The line segment QP is the vertical line between the mobile device and the plane of the display screen (or the mobile device to the display screen). In other words, the mobile device can be regarded as a point, and a vertical line is drawn from the point to the plane of the display screen. The vertical line is the vertical line between the mobile device and the plane of the display screen, and the vertical line between the mobile device and the plane of the display screen is perpendicular to the display screen.

[0204] The straight line in the direction in which the mobile device is pointing can be understood as: starting from any point on the mobile device (for example, any point on the UWB tag on the mobile device, or the center point on the top of the mobile device, etc.) as the starting point of the straight line, the direction of the straight line is the direction in which the mobile device is actually pointing. In addition, the starting point of the straight line in the direction in which the mobile device is pointing is the same as the starting point (point Q) of the vertical line between the mobile device and the display screen. The mobile device can rotate around the point (point Q) in the first direction or the second direction. When the mobile device rotates left or right in the first direction around point Q, the rolling angle between the mobile device and the display screen will change. When the mobile device rotates up or down in the second direction around point Q, the azimuth angle between the mobile device and the display screen will change. In other words, the mobile device can be regarded as a point, with the point (point Q) as the starting point of the straight line in the direction in which the mobile device is pointing, and the direction of the straight line (or it can be understood as a ray starting from point Q) is the direction in which the mobile device is actually pointing.

[0205] In the following description, the annotation device 200 is a smart TV (also referred to as a large-screen device) and the mobile device 100 is a smart phone. For example, the steps of the method for quickly popping up an electronic device control window provided by this application are described. In addition, in the following description, unless otherwise specified, electronic devices refer to electronic devices that do not have a UWB base station or UWB tag installed. It should be understood that this should not impose any restrictions on the mobile devices and annotation devices in the embodiments of the present application. For example, the mobile device can also be an air mouse, a remote control, or other wearable devices.

[0206] First, let's briefly introduce the concept of heading angle. Heading angle includes azimuth angle, tilt angle and roll angle.

[0207] Figure 7 Schematic diagram of the principle of the reference coordinate axis provided. Figure 7 Taking a mobile phone as an example, the meaning of the azimuth angle (expressed by γ), the tilt angle (expressed by η), and the roll angle (expressed by θ) are explained. Figure 7 As shown, the y-axis and the x-axis are directions parallel to the plane of the mobile phone display screen, and the z-axis is a direction perpendicular to the mobile phone display screen, wherein the y-axis is a direction along the length of the mobile phone, and the x-axis is a direction along the width of the mobile phone.

[0208] Azimuth γ, which indicates the direction of the mobile phone around Figure 7 The angle of rotation (rotation) about the z-axis shown in . Figure 8 Schematic diagram of the principle of the azimuth provided. Figure 8 Around the phone Figure 7 Schematic diagram of 90 degree rotation (rotation) of the z-axis shown in; Figure 8 The diagram a in the figure shows a schematic diagram of the mobile phone before it rotates around the z-axis; Figure 8The diagram b in the figure shows the schematic diagram of the mobile phone after it rotates around the z axis. Figure 8 The direction indicated by the arrow on the z-axis in Figure a is the direction of rotation of the mobile phone), and the position of the y-axis becomes Figure 8 The position of the y' axis shown in b in the figure, the position of the x axis becomes Figure 8 The value of γ is equivalent to the angle between the y axis and the front and back sides of the mobile phone when it rotates (turns) around the z axis. Figure 8 The phone is rotated around the z-axis by 0 degrees to indicate the north direction (North), rotated by 90 degrees to indicate the east direction (East), rotated by 180 degrees to indicate the south direction (South), and rotated by 270 degrees to indicate the west direction (West).

[0209] Tilt angle η, the phone moves around Figure 7 The angle when the x-axis is rotated or turned as shown in the figure. The value of the tilt angle will change when the phone is rotated or turned around the x-axis. Fig. 9 Schematic diagram of the principle of the provided tilt angle. Fig. 9 Around the phone Figure 7 Schematic diagram of x-axis rotation (rotation) shown in . Fig. 9 As shown, Fig. 9 a in the figure is a schematic diagram of the mobile phone before it rotates around the x-axis; Fig. 9 b in the figure is a schematic diagram of the mobile phone after it rotates around the x-axis. Fig. 9 The direction indicated by the arrow on the x-axis in a is the direction of rotation of the mobile phone), and the position of the z-axis becomes Fig. 9 The position of the z' axis shown in b in the figure, the position of the y axis becomes Fig. 9 The value of η is equivalent to the angle between the z axis and the front and back sides of the mobile phone when it rotates (rotates) around the x axis ( Fig. 9 The value range of the tilt angle η is between [-180°, 180°]. For example, if you put the phone on a table and the table is completely horizontal, the value of η is 0 degrees. Start lifting the phone from the top until the phone rotates 180 degrees around the x-axis, at which point the phone display is facing down and horizontally touching the table. During this rotation process, the value of η will vary from 0 to -180 degrees, that is, when the phone is lifted, the value of η will gradually decrease until it is equal to -180 degrees. If you start lifting the phone from the bottom until the phone rotates 180 degrees around the x-axis, the value of η will vary from 0 to 180 degrees.

[0210] Roll angle θ, the mobile phone rotates Figure 7 The angle of rotation or rotation of the y-axis shown in . The value range of θ is [-90°, 90°]. Fig.10 Schematic diagram of the principle of the roll angle provided. Fig.10 Around the phone Figure 7 Schematic diagram of the y-axis rotation (rotation) shown in . Fig.10 a in the figure is a schematic diagram of the mobile phone before it rotates around the y-axis; Fig.10 b in the figure is a schematic diagram of the mobile phone after it rotates around the y axis. Fig.10 As shown, after the mobile phone rotates around the y axis ( Fig.10 The direction indicated by the arrow on the y-axis in a is the direction of rotation of the mobile phone), and the position of the z-axis becomes Fig.10 The position of the z' axis shown in b in the figure, the position of the x axis becomes Fig.10 The value of θ is equivalent to the angle between the z axis and the front and back sides of the mobile phone when it rotates (turns) around the y axis. Fig.10 θ is the angle between the z-axis and the z'-axis shown in b in the figure). Assume that the mobile phone is placed horizontally on the table with the display facing upward. If the table is flat, the value of θ should be 0 degrees. Gradually lift the phone from the left side, and the value of θ will gradually decrease until the phone is placed perpendicular to the table. At this time, the value of θ is -90 degrees. Gradually lift the phone from the right side, and the value of θ will gradually increase until the phone is placed perpendicular to the table. At this time, the value of θ is 90 degrees. If the phone continues to roll to the right or left when it is perpendicular to the table, the value of θ will continue to change between -90 degrees and 90 degrees.

[0211] Heading angle of the smartphone: The orientation sensor on the smartphone takes values ​​of the x-axis and y-axis. The x-axis represents the degree to which the smartphone is tilted up and down, and the y-axis represents the degree to which the smartphone is tilted left and right. Fig.11 Schematic diagram of the x-axis, y-axis and z-axis on the smart phone 100. The z-axis is perpendicular to the surface of the smart phone 100, the x-axis and the y-axis are located in the same plane, and the z-axis is perpendicular to the plane where the x-axis and the y-axis are located.

[0212] The direction sensor 102 on the smart phone 100 can detect changes in the azimuth, tilt angle, and roll angle of the smart phone 100. In the embodiment of the present application, the values ​​detected by the direction sensor 102 on the smart phone 100 are the values ​​of the azimuth and roll angles. The roll angle represents the angle between the straight line in the direction in which the smart phone is pointing and the perpendicular line between the smart phone and the display screen in the first direction (horizontal direction), and the azimuth represents the angle between the straight line in the direction in which the smart phone is pointing and the perpendicular line between the smart phone and the display screen in the second direction (vertical direction).

[0213] Among them, the vertical line between the plane of the smartphone and the display screen (or from the smartphone to the display screen) can be understood as: from any point on the smartphone 100 (for example, any point on the UWB tag on the smartphone, or the center point on the top of the smartphone, etc.) as an end point of the line segment (recorded as point Q), a vertical line is drawn to the plane of the display screen 211 of the labeling device 200, and the intersection of the vertical line and the display screen 211 is the other end point of the line segment (recorded as point P). The line segment QP is the vertical line between the smartphone and the plane of the display screen (or from the smartphone to the display screen). In other words, the smartphone can be regarded as a point, and a vertical line is drawn from the point to the plane of the display screen. The vertical line is the vertical line between the smartphone and the plane of the display screen, and the vertical line between the smartphone and the plane of the display screen is perpendicular to the display screen.

[0214] The straight line where the smartphone is pointing (in the embodiment of the present application, the straight line where the smartphone is pointing can also be referred to as the smartphone pointing direction) can be understood as: starting from any point on the smartphone (for example, any point on the UWB tag on the smartphone, or the center point on the top of the smartphone, etc.), the direction of the straight line is the direction in which the smartphone is actually pointing, and the starting point of the straight line where the smartphone is pointing is the same as the starting point of the vertical line between the smartphone and the display screen (point Q). In other words, the smartphone can be regarded as a point, with the point (point Q) as the starting point of the straight line where the smartphone is pointing, and the direction of the straight line (or the ray starting from point Q) is the direction in which the smartphone is actually pointing.

[0215] The following describes a process for calibrating the smartphone 100. Figure 5B Steps S501b to S504b are shown.

[0216] Since the display screen 211 of the smart TV 200 is fixed, the calibration is performed when the smart phone 100 is facing the display screen 211 .

[0217] Specifically, when the user holds the upper surface of the smartphone (i.e., the surface facing the user when using the smartphone) parallel to the upper and lower edges of the display screen and perpendicular to the left and right edges of the display screen, the angle between the upper surface of the smartphone and the upper and lower edges of the display screen is 0 degrees, the angle between the upper surface of the smartphone and the left and right edges of the display screen is 90 degrees, and the smartphone is perpendicular to the display screen. Fig.12 FIG. 2 is a schematic diagram of a smart phone 100 facing a display screen 211. Fig.12As shown, the straight line where the smartphone 100 is pointing coincides with the perpendicular line between the smartphone 100 and the display screen 211 (or from the smartphone to the display screen), and the line segment QP is the perpendicular line between the smartphone 100 and the display screen 211 (or from the smartphone to the display screen). In the embodiment of the present application, when the user holds the smartphone 100 and manually makes it perpendicular to the display screen 211 of the smart TV 200, the smartphone 100 can also prompt the user according to the measurement information of the direction sensor, for example, prompting the user on the display interface of the smartphone 100 whether the straight line where the smartphone 100 is pointing is perpendicular to the display screen 211 of the smart TV 200, so that the user can make adjustments according to the prompt, and finally make the smartphone 100 pointing perpendicular to the display screen 211 of the smart TV 200.

[0218] If the smartphone is tilted toward the upper edge of the display, the angle of tilt is denoted as ψ;

[0219] If the smartphone is tilted toward the lower edge of the display, the tilt angle is denoted as λ;

[0220] If the smartphone is tilted toward the left edge of the display, the tilt angle is denoted as σ;

[0221] If the smartphone is tilted toward the right edge of the display, the angle of tilt is ω.

[0222] Optionally, in the embodiment of the present application, when the smartphone is tilted toward the lower edge of the display screen, the tilt angle is recorded as a negative value; when the smartphone is tilted toward the upper edge of the display screen, the tilt angle is recorded as a positive value. When the smartphone is tilted toward the left edge of the display screen, the tilt angle is recorded as a positive value; when the smartphone is tilted toward the right edge of the display screen, the tilt angle is recorded as a negative value. The upper edge, lower edge, left edge, and right edge are the four edges of the display screen (also referred to as four boundaries).

[0223] In the embodiment of the present application, ψ and λ can be understood as the actual azimuth angle between the smartphone and the display screen. Optionally, the value of λ is defined as a negative value, and the value of ψ is defined as a positive value. For example, the value of λ can be -40°, and the value of ψ can be +60°.

[0224] σ and ω can be understood as the actual rolling angle between the smartphone and the display. Optionally, the value of σ is defined as a positive value and the value of ω is defined as a negative value. For example, the value of σ is +30° and the value of ω is -45°.

[0225] In the embodiment of the present application, the actual angle of rotation of the smartphone includes: the actual azimuth angle (ψ or λ) between the smartphone and the display screen and the actual roll angle (σ or ω) between the smartphone and the display screen.

[0226] In the embodiment of the present application, the actual rolling angle between the smartphone and the display screen represents: the angle between a straight line in the pointing direction of the smartphone and a vertical line between the smartphone and the display screen in a first direction (horizontal direction).

[0227] The actual azimuth angle between the smartphone and the display screen represents: the angle between a straight line in the direction in which the smartphone is pointing and a perpendicular line between the smartphone and the display screen in a second direction (vertical direction).

[0228] For example, Fig.13 A schematic diagram of the actual rolling angle σ between the smartphone and the display provided in this application, Fig.13 In FIG. 1 , the line segment QP is a perpendicular line between the plane of the smartphone 100 and the display screen 211 (or between the smartphone 100 and the display screen 211). Fig.12 When the smartphone 100 is in the position shown, when the smartphone 100 rotates leftward or rightward around the point Q in the horizontal direction, the rolling angle σ between the smartphone 100 and the display screen 211 will change.

[0229] Fig.14 Schematic diagram of the actual azimuth angle ψ between the smartphone and the display screen provided in this application. The line segment QP is the perpendicular line between the smartphone 100 and the display screen 211 (or the smartphone 100 to the display screen 211). Fig.14 When the smartphone 100 is in the position shown, when the smartphone 100 rotates upward or downward in the vertical direction around the point Q, the azimuth angle between the smartphone 100 and the display screen 211 will change.

[0230] Optionally, in the examples of the present application, the straight line in the direction in which the smartphone is pointing can also be called the extension line of the central axis of the smartphone, and the straight line in the direction in which the smartphone is pointing means that, based on the outer edge contour shape of the smartphone, there is a straight line that makes the smartphone symmetrical about the central axis. That is, the vertical distances from a point on the central axis to the upper and lower surfaces of the smartphone are equal, and the vertical distances from a point on the central axis to the left and right surfaces of the smartphone are equal. For example, the smartphone is a mobile phone with a touch screen, and the outer edge contour shape of the mobile phone is generally symmetrical. For example, the outer edge contour shape of the mobile phone is a rectangular parallelepiped. Among them, the starting point of the central axis extension line and the starting point (point Q) of the vertical line from the smartphone to the display are the same.

[0231] After completing the angle calibration of the smartphone, in S501b, after the straight line where the smartphone is pointing is perpendicular to the display screen of the annotation device, the smartphone can obtain an initial direction measurement message (i.e., first measurement information) on the direction sensor. The initial measurement information includes the initial azimuth and initial roll angle of the smartphone when the straight line where the smartphone is pointing is perpendicular to the display screen of the annotation device.

[0232] The smartphone may determine an initial azimuth angle and an initial roll angle based on the initial direction measurement message.

[0233] In S502b, after the smartphone is rotated, that is, after the smartphone is pointed in a direction that is not perpendicular to the display screen 211, the smartphone may obtain the current direction measurement message (second measurement information) on the direction sensor.

[0234] The smartphone determines the current azimuth angle and the current roll angle based on the current direction measurement message.

[0235] In S503b, the smart phone determines a second azimuth angle ψ or λ based on the current azimuth angle and the initial azimuth angle, and determines a second roll angle σ or ω based on the current roll angle and the initial roll angle.

[0236] For example, the initial azimuth angle may be subtracted from the current azimuth angle to obtain the second azimuth angle ψ or λ.

[0237] The second azimuth angle ψ or λ represents the actual azimuth angle between the smart phone and the display screen. The second roll angle σ or ω represents the actual roll angle between the smart phone and the display screen.

[0238] The second azimuth angle ψ or λ is the angle between the pointing direction of the smart phone 100 and the perpendicular line between the smart phone 100 and the display screen 211 of the smart TV 200 in the second direction (vertical direction).

[0239] The second rolling angle σ or ω is the angle between the pointing direction of the smart phone 100 and the vertical line between the smart phone 100 and the display screen 211 of the smart TV 200 in the first direction (horizontal direction).

[0240] In S504b, the smartphone 100 may also determine a second angle θ based on a second azimuth angle ψ or λ and a second roll angle σ or ω. The second angle θ is the angle between the pointing direction of the smartphone and the vertical line between the smartphone and the display screen (i.e., the actual angle of rotation of the smartphone). θ may be understood as an angle synthesized from the actual azimuth angle ψ or λ between the smartphone and the display screen and the actual roll angle σ or ω between the smartphone and the display screen. In other words, the second angle θ may be decomposed into a second azimuth angle and a second roll angle. The second azimuth angle may be understood as the component of the second angle θ in the second direction, and the second roll angle may be understood as the component of the second angle θ in the first direction.

[0241] For example, Fig.15 It is a schematic diagram showing a second angle θ between a straight line in the pointing direction of the smart phone 100 and a perpendicular line between the smart phone 100 and the display screen 211 .

[0242] It should be understood that in the embodiment of the present application, the direction sensor on the smart phone will detect the direction information in real time, that is, the smart phone can calculate the second angle θ in real time.

[0243] Optionally, in the embodiment of the present application, the second angle θ may be defined as positive or negative. For example, in the embodiment of the present application, when the straight line in the direction in which the smartphone is pointing is located to the left of the vertical line between the smartphone and the display screen, the second angle θ is a positive value. For example, Fig.15 The illustrated case is that the straight line in which the smartphone 100 is pointing is located on the left side of the vertical line between the smartphone 100 and the display screen 211 .

[0244] When the straight line in which the smartphone is pointing is located on the right side of the vertical line between the smartphone and the display screen, the second angle θ is a negative value.

[0245] Optionally, the mobile device may also use the second angle θ to determine the actual azimuth angle ψ or λ between the smartphone and the display screen, and the actual roll angle σ or ω between the smartphone and the display screen, that is, decompose the second angle θ into the actual azimuth angle ψ or λ between the smartphone and the display screen, and the actual roll angle σ or ω between the smartphone and the display screen. The azimuth angle ψ or λ can be regarded as the component of the second angle θ in the second direction, that is, the angle between the straight line in the direction in which the smartphone is pointing and the perpendicular line between the smartphone and the display screen in the second direction (vertical direction). The roll angle σ or ω can be regarded as the component of the second angle θ in the first direction, that is, the angle between the straight line in the direction in which the smartphone is pointing and the perpendicular line between the smartphone and the display screen in the first direction (horizontal direction).

[0246] Through the above method, the actual azimuth and roll angle of the smartphone are determined.

[0247] After the angle calibration of the smart phone 100 is completed, the value of the first angle (denoted by α) can be calculated. Fig.16 The diagram shows a schematic diagram of the first angle α. The first angle α represents the angle between the line connecting the electronic device 300 and the smart phone 100 and the perpendicular line between the smart phone 100 and the plane of the display screen 211. The line connecting the electronic device 300 and the smart phone 100 can be understood as: starting from any point on the smart phone 100 (for example, any point on the UWB tag on the smart phone 100, or the center point of the top of the smart phone 100, etc.) as the starting point of the straight line, and the end point of the straight line is any point on the electronic device 300. In addition, the starting point of the line connecting the electronic device 300 and the smart phone 100 is the same as the starting point (point Q) of the perpendicular line between the smart phone 100 and the display screen 211. In other words, the electronic device 300 and the smart phone 100 can be regarded as a point, and the line connecting these two points is the line connecting the electronic device 300 and the smart phone 100. In other words, the first angle α represents the angle between the straight line in the direction in which the smart phone 100 points when the smart phone 100 points to the electronic device 300 and the perpendicular line between the smart phone 100 and the display screen 211.

[0248] The process of calculating the first angle α is described below.

[0249] In the embodiment of the present application, when the angle of the smart phone 100 is calibrated, the position coordinates of the smart phone 100 can be determined. Fig.17 Schematic diagram of the relative position relationship of smartphones, smart TVs and electronic devices. Fig.17 A schematic diagram of the X-axis, Y-axis and Z-axis on the display screen 211 of the smart TV is shown, wherein the Z-axis is perpendicular to the surface of the display screen 211, the X-axis and the Y-axis are located in the same plane, and the Z-axis is perpendicular to the plane where the X-axis and the Y-axis are located.

[0250] like Fig.17As shown, point A1 represents the position of the first UWB base station 212 on the display screen 211 (point A1 can be any point on the first UWB base station 212), point A2 represents the position of the second UWB base station 213 on the display screen 211 (point A2 can be any point on the second UWB base station 213), point A3 represents the position of the third UWB base station 214 on the display screen 211 (point A3 can be any point on the third UWB base station 214), point Q represents the relative position of the smart phone 100 in the coordinate system, point Q can be any point on the smart phone 100 (for example, any point on the UWB tag 101 set on the smart phone 100). Point M represents the relative position of the electronic device 300 in the coordinate system, point M can be any point on the electronic device 300. Point P represents the intersection of the vertical line between the smart phone 100 and the display screen 211 and the display screen 211, and line segment QP represents the vertical line between the smart phone 100 and the plane of the display screen 211, and line segment QP is perpendicular to the display screen 211. In the embodiment of the present application, the location coordinates of the first UWB base station 212 are set to (0, 0, 0).

[0251] exist Fig.17 In the example shown, |A1Q| represents the distance between the first UWB base station 212 and the UWB tag 101, which is Fig.17 S3 in |A2Q| represents the distance between the second UWB base station 213 and the UWB tag 101, which is Fig.17 S4 in |A3Q| represents the distance between the third UWB base station 214 and the UWB tag 211, which is Fig.17 S5 in |A1A2| represents the distance between the first UWB base station 212 and the second UWB base station 213, which is Fig.17 S1 in the figure; |A1A3| represents the distance between the first UWB base station 212 and the third UWB base station 214, which is Fig.17 In S2. Fig.17 In the coordinate system shown, the position coordinates of point A1 are (0, 0, 0).

[0252] In the embodiment of the present application, the distance between two UWB base stations and the distance between a UWB base station and a UWB tag can be determined according to the distance measurement principle of UWB. First, the distance measurement principle of UWB is briefly introduced.

[0253] UWB distance measurement mainly uses the time of flight (TOF) distance measurement method to measure the distance between two devices. This distance measurement method uses the flight time of the signal between two asynchronous transceivers to measure the distance between nodes. Fig.18 Schematic diagram of determining the distance between two devices using the two-way time-of-flight method. Fig.18As shown, device A first sends a data packet to device B and records the time T of sending the data packet. a1 The time length for the data packet to be transmitted between device A and device B is T prop After receiving the data packet, device B records the time when the data packet is received. b1 , then device B waits for T reply1 After the time length, at T b2 At time t, another data packet is sent to device A. The time length for the data packet to be transmitted between device A and device B is T. prop After receiving the data packet, device A records the time when the data packet is received T a2 Then device A waits for T reply2 After the time length, at T a3 Another data packet is sent to device B at time t. The time length of the transmission of this data packet between device A and device B is T prop After receiving the data packet, device b records the time when the data packet is received T b3 Among them, the time T a1 and time T a2 The time length between round1 , time T b2 and time T b3 The time length between round2 , then the flight time of the data packet in the air can be determined according to the following formula (1):

[0254]

[0255] According to formula (1), the flight time of the data packet in the air can be calculated use Multiplying by the speed of light gives the distance between device A and device B.

[0256] Figure 5A The flow shown is a process of determining S1 to S5 using the distance measurement principle of UWB.

[0257] When measuring the distance using the UWB ranging principle, the smart TV 200 controls the first UWB base station 212 on the display screen 211 to send a first ranging message to the second UWB base station 213. The second UWB base station 213 can send a first response message in response to the first ranging message to the first UWB base station 212 according to the first ranging message. The smart TV 200 can determine the distance S1 between the first UWB base station 212 and the second UWB base station 213 according to the time difference between the first ranging message and the first response message, and further determine the position coordinates of the second UWB base station 213 (i.e., at Fig.17 The position coordinates of point A2 in the coordinate system shown in the figure). Fig.17 In the coordinate system shown, the position coordinates of point A1 are (0, 0, 0).

[0258] The smart TV 200 controls the first UWB base station 212 on the display screen 211 to send a second ranging message to the third UWB base station 214. The third UWB base station 214 can send a second response message in response to the second ranging message to the first UWB base station 212 according to the second ranging message. The smart TV 200 can determine the distance S2 between the first UWB base station 212 and the third UWB base station 214 according to the time difference between the second ranging message and the second response message, and further determine the position coordinates of the third UWB base station 214 (i.e., at Fig.17 The position coordinates of point A3 in the coordinate system shown).

[0259] In S490 , the smart TV 200 sends the distance S2 between the first UWB base station 212 and the third UWB base station 214 , the position coordinates of the third UWB base station 214 , the distance S1 between the first UWB base station 212 and the second UWB base station 213 , and the position coordinates of the second UWB base station 213 to the smart phone 100 .

[0260] In S440 to S470, the smartphone 100 controls the UWB tag 101 on the smartphone to send a third ranging message to the first UWB base station 212. The first UWB base station 212 can send a third response message in response to the third ranging message to the UWB tag 101 according to the third ranging message. The smartphone 100 can determine the distance S3 between the first UWB base station 212 and the UWB tag 101 according to the time difference between the third ranging message and the third response message, and further determine the position coordinates of the UWB tag 101 (i.e., at Fig.17 The position coordinates of point Q in the coordinate system shown).

[0261] The smart phone 100 controls the UWB tag 101 on the smart phone to send a fourth ranging message to the second UWB base station 213. The second UWB base station 213 can send a fourth response message in response to the fourth ranging message to the UWB tag 101 based on the fourth ranging message. The smart phone 100 can determine the distance S4 between the second UWB base station 213 and the UWB tag 101 based on the time difference between the fourth ranging message and the fourth response message.

[0262] The smart phone 100 controls the UWB tag 101 on the smart phone to send a fifth ranging message to the third UWB base station 214. The third UWB base station 214 can send a fifth response message in response to the fifth ranging message to the UWB tag 101 based on the fifth ranging message. The smart phone 100 can determine the distance S5 between the third UWB base station 214 and the UWB tag 101 based on the time difference between the fifth ranging message and the fifth response message.

[0263] After determining the relative position coordinates of S1 to S5 and the first UWB base station 212, the second UWB base station 213, the third UWB base station 214, and the UWB tag 101, the smart phone 100 can further determine the relative position relationship between the first UWB base station 212, the second UWB base station 213, the third UWB base station 214, and the UWB tag 101.

[0264] The above process corresponds to Figure 5A S501a to S507a shown.

[0265] The following introduces Figure 6 The process shown.

[0266] In S601, after the angle of the smartphone 100 is calibrated, the position of the electronic device 300 can be marked. Specifically, the user can place the smartphone 100 at the position of the electronic device 300. For example, the smartphone 100 is placed near the electronic device 300, or the smartphone 100 is attached to the electronic device 300. At this time, the position of the smartphone 100 can replace the position of the electronic device 300, and the position coordinates of the electronic device 300 can be determined using the UWB tag 211 on the smartphone 100, for example, Fig.19 A schematic diagram of determining the position of the electronic device 300 and the first angle α when attaching the smart phone 100 to the electronic device 300. In other words, Fig.17 In the example shown, the user moves the position of the smartphone 100 from point Q to point M, and the smartphone 100 is located at point M. After the smartphone 100 is located at point M, the UWB tag 101 on the smartphone sends a ranging message to the first UWB base station 212. The first UWB base station 212 can send a response message in response to the ranging message to the UWB tag 101 based on the ranging message. The smartphone 100 can determine the distance |MA1| between the first UWB base station 212 and the UWB tag 101 based on the time difference between the ranging message and the response message, that is, determine the distance |MA1| between point M and point A1. After determining the distance |MA1| between point M and point A1, since the position coordinates of point A1 are (0, 0, 0), the distance at point A1 can be determined based on the value of |MA1|. Fig.17The position coordinates of point M (ie, the electronic device 300) in the coordinate system shown.

[0267] In the embodiment of the present application, when the smart phone 100 is placed near the electronic device 300, the smart phone can receive the first input of the user. For example, Fig. 20 The diagram shown is a schematic diagram of the operation process of the user's first input. Fig. 20 As shown in a in FIG. 1 , the user can click the “Smart Life” icon on the smart phone 100, and the interface displayed is as follows Fig. 20 As shown in b in FIG. 1 , icons of various electronic devices in the home are displayed on the smart phone 100. The user can click on the icon of the electronic device 300 to Fig. 20 In the example, the electronic device 300 is a smart light. When the user clicks (i.e., the first input) the icon of "smart light", the UWB tag 101 on the smart phone is triggered to send a distance measurement message to the first UWB base station 212. Further, the smart phone 100 can determine the distance |MA1| between the first UWB base station 212 and the UWB tag 101, thereby determining the distance between the electronic device 300 and the first UWB base station 212. Fig.16 The position coordinates of point M in the coordinate system shown in FIG. 1 determine the corresponding relationship between the smart light and the position of the smart light. In other words, Fig.17 In the example shown, after the position of the smart phone 100 is moved from point Q to point M, the smart phone 100 can determine that the electronic device corresponding to the position of point M is a smart light. Furthermore, the smart phone 100 can also obtain relevant information of the smart light. For example, when the user clicks on the "smart light" icon on the smart phone 100, the smart phone 100 can obtain relevant information of the smart light (or can also be called annotation information), which may include: one or more of the media access control (MAC) address, internet protocol (IP) address, serial number (SN) or device identification (ID) of the smart light. The MAC address, IP address, SN or device ID of the smart light uniquely corresponds to the smart light. When the user uses the smart phone 100 to annotate the electronic device 300, the smart phone 100 can determine the correspondence between the relevant information of the electronic device 300 and the position of the electronic device 300, that is, determine Fig.17 The corresponding relationship between the position coordinates of the point M and the relevant information of the smart light. The smart phone 100 can store the corresponding relationship.

[0268] The process of the smart phone determining the first angle α in S602 will be described below.

[0269] like Fig.21 As shown, Fig.21 The meaning of each point in Fig.17 The various points in the table have the same meaning. Fig.21 In the embodiment of the present application, after determining the distance |MQ| between point M and point Q, when determining the first angle α, it is also necessary to determine the length of line segment QP and the length of line segment MP. Fig.21 In the example shown, point P represents the intersection of the perpendicular line between the smartphone 100 and the display screen 211 and the display screen 211, line segment QP represents the perpendicular line between the smartphone 100 and the plane of the display screen 211, line segment QP is perpendicular to the display screen 211, and h represents the length of line segment QP. The perpendicular line from point P to the left edge of the display screen 211 intersects with the left edge of the display screen 211 at W, and line segment PW is perpendicular to line segment A3 A1. The perpendicular line from point P to the right edge of the display screen 211 intersects with the right edge of the display screen 211 at N, and line segment PN is perpendicular to line segment A2 E. Draw a perpendicular line from point P to the upper edge of the display screen 211 (i.e., line segment A3 E), and the perpendicular line from point P to the upper edge of the display screen 211 intersects with the upper edge of the display screen 211 at point D, and line segment PD is perpendicular to line segment A3 E. Triangle A3PQ is a right triangle, and ∠A3PQ is a right angle. Connect points W and D to form line segment WD. Triangle WPD is a right triangle, and ∠WPD is a right angle. Triangle WPQ is a right triangle, and ∠WPQ is a right angle. Triangle NPQ is a right triangle, and ∠NPQ is a right angle. Connect A1 P as an auxiliary line, and connect A2 P as an auxiliary line. Triangle A1PU is a right triangle, and ∠A1UP is a right angle. Triangle A1PQ is a right triangle, and ∠A1PQ is a right angle. Triangle A2PU is a right triangle, and ∠A2UP is a right angle. Triangle A2 PQ is a right triangle, and ∠A2PQ is a right angle. Quadrilateral WPUA1 is a rectangle, and quadrilateral NPUA2 is a rectangle. Quadrilateral WPDA3 is a rectangle. The length of line segment WD is equal to the length of line segment A3P.

[0270] exist Fig.21 In the example shown, the formula is as follows:

[0271] x 2 +y 2 =|A1Q| 2 -h 2 (2)

[0272] x 2 +(|A1A3|-y) 2 =|A3Q| 2 -h 2 (3)

[0273] y 2 +(|A1A2|-x) 2 =|A2Q| 2 -h2 (4)

[0274] Subtracting formula (4) from formula (2) yields formula (5):

[0275] x 2 -(|A1A2|-x) 2 =|A1Q| 2 -|A2Q| 2 (5)

[0276] Simplifying formula (5), we get formula (6):

[0277]

[0278] Subtracting formula (3) from formula (2) yields formula (7):

[0279] y 2 -(|A1A3|-y) 2 =|A1Q| 2 -|A3Q| 2 (7)

[0280] Simplifying formula (7), we get formula (8):

[0281]

[0282] This gives us the values ​​of x and y.

[0283] After calculating the values ​​of x and y, since the position coordinates of point A1 are (0, 0, 0), the position coordinates of point U can be determined based on the value of x. After the position coordinates of point U are determined, since triangle A1 UP is a right triangle and ∠A1 UP is a right angle, the position coordinates of point P can be determined based on the position coordinates of point U and the value of y.

[0284] After the position coordinates of point P are determined, the distance |PQ| between point P and point Q can be determined, and the length value of |PQ| is equal to h.

[0285] After the position coordinates of point P are determined, since the position coordinates of point M have been determined, the distance |PM| between point P and point M can be determined.

[0286] When the three side lengths of triangle PMQ are known, the value of the first angle α can be determined.

[0287] In the triangle PMQ, as shown in formula (9),

[0288]

[0289] Then the value of the first angle α can be determined, as shown in formula (10):

[0290]

[0291] After calculating the value of the first angle α, the electronic device 300 is marked, that is, the relative position of the electronic device 300 relative to the smart TV 200 is determined. The first angle α is equivalent to the angle obtained by rotating the line segment QP counterclockwise to the position of the line segment QM.

[0292] Optionally, in the embodiment of the present application, the positive and negative values ​​of the first angle α may be defined. For example, in the embodiment of the present application:

[0293] When the electronic device 300 is located at the left side of the smart phone 100, the first angle α is a positive value;

[0294] When the electronic device 300 is located at the right side of the smart phone 100, the first angle α is a negative value;

[0295] For example, Fig.21 In the example shown, the electronic device 300 (point M) is located to the left of the smart phone 100 (point Q), and the value of the first angle α can be a positive value, for example, α can be +75° (also called +75 degrees).

[0296] After completing the marking of the electronic device 300, the user takes the smart phone 100 away from the position of the electronic device 300. When the user places the smart phone 100 at the position where the angle of the smart phone 100 is calibrated, for example, Fig.12 When the user needs to use the smart phone 100 to control the electronic device 300, the user can rotate the smart phone 100. When the user rotates the smart phone 100, the direction sensor on the smart phone 100 can measure the actual azimuth angle ψ or λ between the smart phone and the display screen, and the actual roll angle σ or ω between the smart phone and the display screen in real time. Further, the second angle θ can also be determined based on the azimuth angle ψ or λ and the roll angle σ or ω.

[0297] In other words, after the electronic device 300 is labeled and the smart phone 100 is removed from the position of the electronic device 300, when the user rotates the smart phone 100 and the direction sensor on the smart phone detects that the second angle θ has changed, the smart phone 100 can determine that it is necessary to compare the second angle θ with the first angle α to determine whether the smart phone 100 is pointing to and controlling the electronic device 300.

[0298] The following describes the process of determining whether the smart phone pops up a control window of the electronic device according to the second angle θ and the first angle α in S603. There are two possible implementation methods:

[0299] A first implementation method: The smart phone determines whether the difference between the second angle θ and the first angle α is within a preset range.

[0300] If the difference between the second angle θ and the first angle α is within a preset difference range, for example, the difference between the first angle and the second angle θ is 0 degrees, it means that the smartphone is pointing at the electronic device, and a control window of the electronic device pops up on the smartphone, and the smartphone can control the electronic device. Fig. 22 The diagram shown is a schematic diagram showing that the second angle θ is the same as the first angle α. Fig. 22 In the illustrated scenario, the smartphone 100 is pointed at and can control the electronic device 300 .

[0301] If the difference between the second angle θ and the first angle α is not within the preset difference range, for example, the difference between the first angle and the second angle θ is +7°, and the preset difference range is [-5°, +5°], it means that the smartphone is not pointing at the electronic device, the smartphone will not pop up the control window of the electronic device, and the smartphone cannot control the electronic device. Fig.23 The diagram shown is a schematic diagram showing that the difference between the second angle θ and the first angle α is not within a preset difference range, and the smart phone 100 is not pointing to the electronic device 300 .

[0302] Second implementation method: After the smartphone 100 determines the value of the first angle α, further, according to the value of the first angle α, the azimuth component (expressed by β) and the rolling angle component (expressed by δ) of the first angle α are determined. The azimuth component of the first angle α may also be referred to as the first azimuth angle, and the rolling angle component of the first angle α may also be referred to as the first rolling angle. The rolling angle component δ represents the angle between the straight line in the direction in which the smartphone 100 is pointing and the perpendicular line between the smartphone 100 and the display screen 211 in the first direction (horizontal direction) when the smartphone 100 is pointing at the electronic device 300. The azimuth component β represents the angle between the straight line in the direction in which the smartphone 100 is pointing and the perpendicular line between the smartphone 100 and the display screen 211 in the second direction (vertical direction) when the smartphone 100 is pointing at the electronic device 300.

[0303] When the user uses the smart phone 100 to control the electronic device 300, the direction sensor on the smart phone 100 will detect the azimuth angle ψ or λ, and the value of the roll angle σ or ω in real time. The smart phone 100 can determine whether the smart phone is pointing to the electronic device based on the actual azimuth angle ψ or λ, the value of the roll angle σ or ω, and the value of the azimuth angle component β and the roll angle component δ. In the second implementation, there are four specific cases:

[0304] The first case: if the difference between the azimuth angle ψ or λ and the azimuth angle component β is within a preset first difference range, for example, the preset first difference range is [-3°, +3°], the azimuth angle ψ or λ and the azimuth angle component β are equal; and the difference between the roll angle σ or ω and the roll angle component δ is also within a preset second difference range, for example, the preset second difference range is [-2°, +2°], the roll angle δ and the roll angle component σ or ω are equal, then it means that the smart phone 100 is pointing at the electronic device 300, the smart phone 100 will pop up a control window of the electronic device 300, and the smart phone 100 can control the electronic device 300. In other words, when the smart phone 100 points to the position of the electronic device 300, it means that the smart phone 100 is pointing at the electronic device 300, and the smart phone 100 can control the electronic device 300. The smart phone 100 will obtain the relevant information corresponding to the location of the electronic device 300 stored previously, such as: one or more of the MAC address, IP address, SN or device ID of the electronic device 300, and the MAC address, IP address, SN or device ID of the electronic device 300 is uniquely corresponding to the electronic device 300. Based on this information, the user can use the smart phone 100 to send corresponding control information to the electronic device 300 to control the electronic device 300. For example, Fig.24 The diagram shown is a schematic diagram showing that the smart phone 100 is pointing toward the electronic device 300 in both the vertical direction and the horizontal direction.

[0305] The second case: if the difference between the azimuth angle ψ or λ and the azimuth angle component β is within the preset first difference range; and the difference between the roll angle σ or ω and the roll angle component δ is not within the preset second difference range, for example, the difference between the roll angle σ or ω and the roll angle component δ is +5°, and the preset second difference range is [-2°, +2°], it is determined that the smartphone is pointing to the electronic device 300 in the vertical direction, but not in the horizontal direction, the smartphone 100 will not pop up the control window of the electronic device 300, and the smartphone 100 cannot control the electronic device 300. For example, Fig.25 The diagram shown is a schematic diagram in which the smart phone 100 is pointed toward the electronic device 300 in the vertical direction, but not pointed toward the electronic device 300 in the horizontal direction.

[0306] The third case: if the difference between the azimuth angle ψ or λ and the azimuth angle component β is not within the preset first difference range, for example, the difference between the azimuth angle ψ or λ and the azimuth angle component β is -4°, and the preset first difference range is [-3°, +3°], then the difference between the azimuth angle ψ or λ and the azimuth angle component β is not within the preset first difference range; and the difference between the roll angle σ or ω and the roll angle component δ is within the preset second difference range, then it is determined that the smartphone is not pointing at the electronic device 300 in the vertical direction, but is pointing at the electronic device 300 in the horizontal direction, the smartphone 100 will not pop up the control window of the electronic device 300, and the smartphone 100 cannot control the electronic device 300. For example, Fig.26 The diagram shown is a schematic diagram in which the smart phone 100 is not pointed at the electronic device 300 in the vertical direction, but is pointed at the electronic device 300 in the horizontal direction.

[0307] The fourth case: if the difference between the value of the azimuth angle ψ or λ and the value of the azimuth angle component β is not within the preset first difference range, for example, the difference between the value of the azimuth angle ψ or λ and the value of the azimuth angle component β is -4°, and the preset first difference range is [-3°, +3°], then the difference between the value of the azimuth angle ψ or λ and the value of the azimuth angle component β is not within the preset first difference range; and, the difference between the value of the roll angle σ or ω and the value of the roll angle component δ is not within the preset second difference range, for example, the difference between the value of the roll angle σ or ω and the value of the roll angle component δ is +5°, and the preset second difference range is [-2°, +2°]. It is determined that the smartphone is not pointing at the electronic device 300 in both the vertical and horizontal directions, the smartphone 100 will not pop up the control window of the electronic device 300, and the smartphone 100 cannot control the electronic device 300. For example, Fig. 27 The diagram shown is a schematic diagram in which the smart phone 100 is not pointing toward the electronic device 300 in either the vertical direction or the horizontal direction.

[0308] Optionally, in the embodiment of the present application, when the smart phone 100 determines that it is pointing at the electronic device 300, the smart phone 100 will pop up a control window of the electronic device 300. Fig.28 What is shown is a control interface (also referred to as a control window) displayed by the smart phone 100 to the user when the smart phone 100 points to the electronic device 300 . The control interface is used to receive input or operation of the user to control the electronic device 300 . Fig.28In the example shown, assuming that the electronic device 300 is a smart light, the control interface displayed to the user by the smart phone 100 may include: a switch control icon, a brightness control icon, a timer off icon, etc. of the smart light. The user uses the corresponding control icon to control the smart light according to his / her needs on the smart phone 100.

[0309] Optionally, in the embodiment of the present application, when the smart phone 100 points to the electronic device 300 and controls the electronic device 300, the electronic device 300 may also send the capability information it supports to the smart phone 100, and the corresponding control icon will be presented on the smart phone 100. For example, when the electronic device 300 is a smart refrigerator, the capability information of the smart refrigerator may include: temperature control of different storage rooms of the smart refrigerator, switch of the smart refrigerator, etc. The user uses the corresponding control icon to control the electronic device 300 on the smart phone 100 according to his needs.

[0310] Optionally, in an embodiment of the present application, within a preset time period after the control window of the electronic device 300 pops up on the smartphone 100, the control window will continue to be displayed on the smartphone 100, even if the user moves or rotates the smartphone 100, for example, moving the smartphone 100 from a current position to another position, or rotating it from a current orientation to another orientation, the control window will continue to be displayed on the smartphone 100, which can facilitate user use and improve user experience.

[0311] If the user does not perform any operation or input on the control window within the preset time (for example, 5S), that is, the smartphone does not receive any input or operation from the user within the preset time, the smartphone 100 will no longer display the control window.

[0312] If the smartphone receives the user's input or operation within the preset time period, the smartphone 100 will close the control window after the user's input or operation is completed.

[0313] It should be understood that in the embodiment of the present application, there is no restriction on the length of the preset time period. The length of the preset time period can be set by the user or preset by the smartphone when it leaves the factory. The embodiment of the present application does not impose any restriction on this.

[0314] The above takes electronic device 300 as an example to illustrate the process of marking electronic device 300 and the process of using smart phone 100 to point at electronic device 300 and control electronic device 300. In an embodiment of the present application, for other electronic devices that are not equipped with UWB tags or UWB base stations, the process of marking their locations and the process of using smart phone 100 to point at the electronic device and control the electronic device are similar to the above for electronic device 300. For related descriptions, please refer to the above description of electronic device 300. For the sake of brevity, they will not be repeated here.

[0315] The method for quickly popping up an electronic device control window provided in the present application marks the relative positions of electronic devices in the home that do not have a UWB chip installed, so that the positions of electronic devices that do not have a UWB chip installed can be located, and then when it is determined that the angle at which a mobile device (such as a smart phone) points to the electronic device meets certain conditions, it can be determined that the mobile device is pointing to the electronic device that does not have a UWB chip installed, thereby achieving the use of the mobile device to control the electronic device that does not have a UWB chip installed, thereby improving the user experience.

[0316] It should be understood that the above is only to help those skilled in the art better understand the embodiments of the present application, rather than to limit the scope of the embodiments of the present application. According to the above examples given, those skilled in the art can obviously make various equivalent modifications or changes. For example, some steps in each embodiment of the above method may be unnecessary, or some new steps may be added. Or a combination of any two or any multiple embodiments of the above. Such modifications, changes or combined solutions also fall within the scope of the embodiments of the present application.

[0317] It should also be understood that the above description of the embodiments of the present application focuses on emphasizing the differences between the various embodiments. The same or similar points that are not mentioned can be referenced to each other. For the sake of brevity, they will not be repeated here.

[0318] It should also be understood that the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present application.

[0319] It should also be understood that the division of the methods, situations, categories and embodiments in the embodiments of the present application is only for the convenience of description and should not constitute a special limitation. The features of various methods, categories, situations and embodiments can be combined without contradiction.

[0320] It should also be understood that in the various embodiments of the present application, unless otherwise specified or there is a logical conflict, the terms and / or descriptions between different embodiments are consistent and can be referenced to each other, and the technical features in different embodiments can be combined to form new embodiments according to their internal logical relationships.

[0321] The above combination Figure 1-Figure 28 An embodiment of a method for quickly popping up a control window of an electronic device provided by an embodiment of the present application is described. The following describes a mobile device, a labeling device, and a system for labeling electronic devices provided by an embodiment of the present application.

[0322] In this embodiment, the mobile device and the marking device can be divided into functional modules according to the above method. For example, each function can be divided into functional modules, or two or more functions can be integrated into one processing module. The above integrated modules can be implemented in the form of hardware. It should be noted that the division of modules in this embodiment is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0323] It should be noted that the relevant contents of each step involved in the above method embodiment can all be referred to the functional description of the corresponding functional module, which will not be repeated here.

[0324] Mobile devices include but are not limited to terminal devices used by users, such as smart phones, air mice, PDAs, smart remote controls, various tablet computers, etc., or portable mobile devices with other operating systems.

[0325] The annotation device and mobile device provided in this embodiment are used to execute the above-mentioned method of quickly popping up the electronic device control window, so the same effect as the above-mentioned implementation method can be achieved. In the case of an integrated unit, the annotation device and the mobile device may include a processing module, a storage module and a communication module. Among them, the processing module can be used to control and manage the actions of the annotation device or mobile device. For example, it can be used to support the annotation device or mobile device to execute the steps performed by the processing unit. The storage module can be used to support the execution of stored program codes and data, etc. The communication module can be used for the communication between the annotation device or mobile device and other devices (for example, with an electronic device).

[0326] For example, the hardware structure diagram of the mobile device provided in this application can be as follows: Figure 2 As shown, the hardware structure diagram of the annotation device can be as follows Figure 3 shown.

[0327] An embodiment of the present application also provides a system for labeling electronic devices, the system comprising: the electronic device, the mobile device and the labeling device in the above method example.

[0328] The embodiment of the present application also provides a computer-readable storage medium for storing computer program code, the computer program including instructions for executing the method for quickly popping up an electronic device control window provided in the embodiment of the present application. The readable medium may be a read-only memory (ROM) or a random access memory (RAM), which is not limited in the embodiment of the present application.

[0329] The present application also provides a computer program product, which includes instructions. When the instructions are executed, the marking device and the mobile device respectively perform corresponding operations corresponding to the above method.

[0330] The embodiment of the present application also provides a chip located in a communication device, the chip comprising: a processing unit and a communication unit, the processing unit, for example, may be a processor, the communication unit, for example, may be an input / output interface, a pin or a circuit, etc. The processing unit may execute computer instructions to enable the communication device to execute the method for quickly popping up an electronic device control window provided in the embodiment of the present application.

[0331] Optionally, the computer instructions are stored in a storage unit.

[0332] Optionally, the storage unit is a storage unit in the chip, such as a register, a cache, etc. The storage unit can also be a storage unit located outside the chip in the terminal, such as a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM), etc. Among them, the processor mentioned in any of the above can be a CPU, a microprocessor, an ASIC, or one or more integrated circuits for controlling the execution of the program of the method for quickly popping up the electronic device control window. The processing unit and the storage unit can be decoupled and respectively set on different physical devices, and connected by wired or wireless means to realize the respective functions of the processing unit and the storage unit, so as to support the system chip to realize the various functions in the above embodiments. Alternatively, the processing unit and the memory can also be coupled on the same device.

[0333] In addition, an embodiment of the present application further provides a device. The device may be a chip, a component or a module. The device may include a connected processor and a memory; wherein the memory is used to store computer-executable instructions, and when the device is running, the processor may execute the computer-executable instructions stored in the memory, so that the chip executes the method for quickly popping up an electronic device control window in the above embodiment.

[0334] Among them, the mobile device, the marking device, the system for marking electronic devices, the computer storage medium, the computer program product or the chip provided in this embodiment are all used to execute the corresponding methods provided above. Therefore, the beneficial effects that can be achieved can refer to the beneficial effects in the corresponding methods provided above, and will not be repeated here.

[0335] Through the description of the above implementation mode, those skilled in the art can understand that for the convenience and simplicity of description, only the division of the above functional modules is used as an example. In practical applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device can be divided into different functional modules to complete all or part of the functions described above.

[0336] It should be understood that the devices and methods disclosed in the several embodiments provided in the present application can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of modules or units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device. In addition, some features can be ignored or not performed. In addition, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0337] The units described as separate components may or may not be physically separated. The components shown as units may be one physical unit or multiple physical units. That is, they may be located in one place or distributed in multiple different places. Some or all of the units may be selected to achieve the purpose of this solution according to actual needs.

[0338] In addition, each functional unit in each embodiment of the present application may be integrated into a processing unit; or may exist physically separately; or some units may be integrated into one unit, and some units may exist physically separately. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software functional units.

[0339] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a readable storage medium. Based on this understanding, all or part of the technical solution of the embodiment of the present application can be embodied in the form of a software product. The software product is stored in a storage medium. The software product includes a number of instructions to enable a device (which can be a single-chip microcomputer, chip, etc.) or a processor (processor) to perform all or part of the steps of the various embodiments of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM), random access memory (RAM), disk or optical disk and other media that can store program code.

[0340] It should be noted that all or part of the above-mentioned embodiments provided in the present application (for example, part or all of any feature) can be arbitrarily combined or used in combination with each other.

[0341] The above contents are only specific implementation methods of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A mobile device, characterized in that: The mobile device communicates wirelessly with the marking device and the electronic device to be marked; the mobile device comprises: processor; Memory; an ultra-wideband UWB tag; Direction sensor; and a computer program, wherein the computer program is stored on the memory and when the computer program is executed by the processor, causes the mobile device to perform the following steps: After the mobile device moves to the location of the electronic device, a first input is received; In response to the first input, recording the location of the mobile device; the location of the mobile device is the location of the electronic device; After the mobile device moves to another position and points at the electronic device, the mobile device determines whether to pop up the first control window of the electronic device based on a first angle and a second angle, wherein the first angle represents the angle between a straight line in the direction in which the mobile device is pointing when the mobile device is pointing at the electronic device and a perpendicular line between the mobile device and the display screen of the marking device, and the second angle represents the angle between the direction in which the mobile device is pointing and a perpendicular line between the mobile device and the display screen.

2. The mobile device according to claim 1, characterized in that The mobile device also performs: Within a preset time period after the mobile device pops up the first control window of the electronic device, the first control window is displayed on the mobile device; wherein the first control window is used to receive a first user input for controlling the electronic device; If the first user input is not received within the preset time period, the mobile device no longer displays the first control window.

3. The mobile device according to claim 1 or 2, characterized in that: The mobile device communicates wirelessly with the marking device and the electronic device through a routing device; After the mobile device points to the marking device, the mobile device pops up a second control window of the marking device.

4. The mobile device according to claim 3, characterized in that The mobile device also performs: Within a preset time period after the second control window of the annotation device pops up on the mobile device, the second control window is displayed on the mobile device; wherein the second control window is used to receive a second user input for controlling the annotation device; If the second user input is not received within the preset time period, the mobile device no longer displays the second control window.

5. The mobile device according to claim 1, 2 or 4, characterized in that: The marking device includes at least two UWB base stations and a display screen; the at least two UWB base stations include a first UWB base station; the electronic device does not include a UWB base station; before the mobile device moves to the location of the electronic device, the mobile device further performs: Acquire the distance between the first UWB base station and each other UWB base station, the distance between the UWB tag and each of the at least two UWB base stations, and the position of the UWB tag; wherein each other UWB base station is each UWB base station among the at least two UWB base stations except the first UWB base station; Based on the distance between the first UWB base station and each of the other UWB base stations, the distance between the UWB tag and each of the at least two UWB base stations, and the position of the UWB tag, the first angle between the pointing direction of the mobile device and the vertical line between the mobile device and the display screen is obtained.

6. The mobile device according to claim 5, characterized in that Before the mobile device moves to the location of the electronic device, the mobile device further executes: When the mobile device moves to a first position, acquiring a first initial azimuth angle and a first initial rolling angle of the mobile device based on the direction sensor; When the mobile device moves to a second position, based on the direction sensor, a first real-time azimuth angle and a first real-time rolling angle of the mobile device are acquired; Based on the first initial azimuth, the first initial rolling angle, the first real-time azimuth, and the first real-time rolling angle, a second azimuth in the second direction and a second rolling angle in the first direction are acquired; the first direction is perpendicular to the second direction.

7. The mobile device according to claim 6, characterized in that The mobile device also performs: Based on the second azimuth angle and the second rolling angle, the second angle between the pointing direction of the mobile device and the vertical line between the mobile device and the display screen is obtained.

8. The mobile device according to claim 7, characterized in that After the mobile device moves to another position and points to the electronic device, the mobile device determines whether to pop up a first control window of the electronic device according to the first angle and the second angle; comprising: When the mobile device moves to another position and the deviation between the first angle and the second angle is within a preset first range, the mobile device points to the electronic device and the mobile device pops up a first control window of the electronic device.

9. The mobile device according to claim 6, characterized in that: The first angle is decomposed into a first azimuth angle and a first rolling angle; the first azimuth angle represents the angle between a straight line in the direction in which the mobile device is pointing and a perpendicular line from the mobile device to the marking device in the second direction; the first rolling angle represents the angle between a straight line in the direction in which the mobile device is pointing and a perpendicular line from the mobile device to the marking device in the first direction.

10. The mobile device according to claim 9, characterized in that After the mobile device moves to another position and points to the electronic device, the mobile device determines whether to pop up a first control window of the electronic device according to the first angle and the second angle; comprising: When the mobile device moves to another position, and the deviation between the first azimuth angle and the second azimuth angle is within a preset second range, and the deviation between the first rolling angle and the second rolling angle is within a preset third range, the mobile device points to the electronic device, and the mobile device pops up a first control window of the electronic device.

11. The mobile device according to any one of claims 6 to 10, characterized in that: The acquiring of the distance between the first UWB base station and each of the other UWB base stations, the distance between the UWB tag and each of the at least two UWB base stations, and the position of the UWB tag includes: The mobile device sends a ranging message via the UWB tag; receiving a ranging response message from each of the at least two UWB base stations; In response to the ranging response messages of the at least two UWB base stations, acquiring the distance between the UWB tag and each of the at least two UWB base stations; receiving a message from the annotation device; Acquire the distance between the first UWB base station and each of the other UWB base stations from the message; Selecting the position of a first UWB base station among the at least two UWB base stations as the coordinate origin of the three-dimensional coordinate system; Based on the coordinate origin, the position coordinates of another UWB base station among the at least two UWB base stations and the position coordinates of the UWB tag are acquired.

12. The mobile device according to claim 1, 2, 4, 6, 7, 8, 9 or 10, characterized in that: The first input is used to select an identification of an electronic device; the identification of the electronic device corresponds to a parameter of the electronic device.

13. The mobile device according to claim 12, characterized in that The parameters include one or more of a media access control MAC address, an Internet Protocol IP address, a serial number SN or a device identification of the electronic device.

14. The mobile device according to claim 1, 2, 4, 6, 7, 8, 9, 10 or 13, characterized in that: The direction sensor is an inertial measurement unit IMU; the IMU is a 9-axis IMU, and the 9-axis IMU includes an acceleration sensor, a gyroscope sensor, and a magnetic sensor; or the IMU is a 6-axis IMU, and the 6-axis IMU includes an acceleration sensor and a gyroscope sensor.

15. The mobile device according to any one of claims 6 to 10, characterized in that: The first direction is a horizontal direction, and the second direction is a vertical direction; or, the first direction is a vertical direction, and the second direction is a horizontal direction.

16. The mobile device according to claim 1, 2, 4, 6, 7, 8, 9, 10 or 13, characterized in that: The mobile device includes: a smart phone and an air mouse.

17. A marking device, which wirelessly communicates with the mobile device according to any one of claims 1 to 16 and the electronic device to be marked; characterized in that: The marking device comprises: processor; Memory; At least two UWB base stations, the at least two UWB base stations including a first UWB base station; and a computer program, wherein the computer program is stored in the memory, and when the computer program is executed by the processor, the marking device performs the following steps: receiving a ranging message through each of the at least two UWB base stations; In response to the ranging message, sending a ranging response message to the mobile device through each of the at least two UWB base stations; A message is sent to the mobile device through any one of the at least two UWB base stations, the message including the distance between the first UWB base station and each of the other UWB base stations, the ranging response message and the distance between the first UWB base station and each of the other UWB base stations are used by the mobile device to determine a first angle, the first angle representing the angle between a straight line in the direction in which the mobile device is pointing when the mobile device is pointing to the electronic device and a vertical line between the mobile device and the display screen of the marking device.

18. The marking device according to claim 17, characterized in that: The marking device also includes: A display screen; the display screen comprises a first edge, a second edge, a third edge and a fourth edge; the first edge is parallel to the second edge, and the third edge is parallel to the fourth edge; the first edge and the second edge are parallel to a first direction, the third edge and the fourth edge are parallel to a second direction, and the first direction is perpendicular to the second direction; the at least two UWB base stations are distributed at different positions in front of or behind the display screen; all or part of the vertical projections of the at least two UWB base stations on the display screen are located on the display screen.

19. The marking device according to claim 18, characterized in that: The at least two UWB base stations include a first UWB base station, a second UWB base station and a third UWB base station; all or part of the vertical projection of the first UWB base station on the display screen is located at the intersection of the third edge and the second edge; all or part of the vertical projection of the second UWB base station on the display screen is located at the intersection of the third edge and the first edge; all or part of the vertical projection of the third UWB base station on the display screen is located at the intersection of the second edge and the fourth edge.

20. A method for quickly popping up an electronic device control window, the method being applied to a mobile device, the mobile device wirelessly communicating with a marking device and an electronic device to be marked; The mobile device comprises: a processor; a memory; an ultra-wideband UWB tag; a direction sensor; and the method comprises: After the mobile device moves to the location of the electronic device, a first input is received; In response to the first input, recording the location of the mobile device; the location of the mobile device is the location of the electronic device; After the mobile device moves to another position and points at the electronic device, the mobile device determines whether to pop up the first control window of the electronic device based on a first angle and a second angle, wherein the first angle represents the angle between a straight line in the direction in which the mobile device is pointing when the mobile device is pointing at the electronic device and a perpendicular line between the mobile device and the display screen of the marking device, and the second angle represents the angle between the direction in which the mobile device is pointing and a perpendicular line between the mobile device and the display screen.

21. The method according to claim 20, characterized in that The method further comprises: Within a preset time period after the mobile device pops up the first control window of the electronic device, the first control window is displayed on the mobile device; wherein the first control window is used to receive a first user input for controlling the electronic device; If the first user input is not received within the preset time period, the mobile device no longer displays the first control window.

22. The method according to claim 20 or 21, characterized in that The mobile device communicates wirelessly with the annotation device and the electronic device through a routing device, and the method further includes: After the mobile device points to the marking device, the mobile device pops up a second control window of the marking device.

23. The method according to claim 22, characterized in that The method further comprises: Within a preset time period after the second control window of the annotation device pops up on the mobile device, the second control window is displayed on the mobile device; wherein the second control window is used to receive a second user input for controlling the annotation device; If the second user input is not received within the preset time period, the mobile device no longer displays the second control window.

24. The method according to claim 20, 21 or 23, characterized in that The marking device includes at least two UWB base stations and a display screen; the at least two UWB base stations include a first UWB base station; the electronic device does not include a UWB base station; before the mobile device moves to the location of the electronic device, the method further includes: Acquire the distance between the first UWB base station and each other UWB base station, the distance between the UWB tag and each of the at least two UWB base stations, and the position of the UWB tag; wherein the other UWB base station is each of the at least two UWB base stations except the first UWB base station; Based on the distance between the first UWB base station and each of the other UWB base stations, the distance between the UWB tag and each of the at least two UWB base stations, and the position of the UWB tag, the first angle between the pointing direction of the mobile device and the vertical line between the mobile device and the display screen is obtained.

25. The method according to claim 24, characterized in that Before the mobile device moves to the location of the electronic device, the method further includes: When the mobile device moves to a first position, acquiring a first initial azimuth angle and a first initial rolling angle of the mobile device based on the direction sensor; When the mobile device moves to a second position, based on the direction sensor, a first real-time azimuth angle and a first real-time rolling angle of the mobile device are acquired; Based on the first initial azimuth, the first initial rolling angle, the first real-time azimuth, and the first real-time rolling angle, a second azimuth in the second direction and a second rolling angle in the first direction are acquired; the first direction is perpendicular to the second direction.

26. The method according to claim 25, characterized in that The method further comprises: Based on the second azimuth angle and the second rolling angle, the second angle between the pointing direction of the mobile device and the vertical line between the mobile device and the display screen is obtained.

27. The method according to claim 26, characterized in that After the mobile device moves to another position and points to the electronic device, the mobile device determines whether to pop up a first control window of the electronic device according to the first angle and the second angle; comprising: When the mobile device moves to another position and the deviation between the first angle and the second angle is within a preset first range, the mobile device points to the electronic device and the mobile device pops up a first control window of the electronic device.

28. The method according to claim 25, characterized in that The first angle is decomposed into a first azimuth angle and a first rolling angle; the first azimuth angle represents the angle between a straight line in the direction in which the mobile device is pointing and a perpendicular line from the mobile device to the marking device in the second direction; the first rolling angle represents the angle between a straight line in the direction in which the mobile device is pointing and a perpendicular line from the mobile device to the marking device in the first direction.

29. The method according to claim 28, characterized in that After the mobile device moves to another position and points to the electronic device, the mobile device determines whether to pop up a first control window of the electronic device according to the first angle and the second angle; comprising: When the mobile device moves to another position, and the deviation between the first azimuth angle and the second azimuth angle is within a preset second range, and the deviation between the first rolling angle and the second rolling angle is within a preset third range, the mobile device points to the electronic device, and the mobile device pops up a first control window of the electronic device.

30. The method according to any one of claims 25 to 29, characterized in that The acquiring of the distance between the first UWB base station and each of the other UWB base stations, the distance between the UWB tag and each of the at least two UWB base stations, and the position of the UWB tag includes: The mobile device sends a ranging message via the UWB tag; receiving a ranging response message from each of the at least two UWB base stations; In response to the ranging response messages of the at least two UWB base stations, acquiring the distance between the UWB tag and each of the at least two UWB base stations; receiving a message from the annotation device; Acquire the distance between the first UWB base station and each of the other UWB base stations from the message; Selecting the position of a first UWB base station among the at least two UWB base stations as the coordinate origin of the three-dimensional coordinate system; Based on the coordinate origin, the position coordinates of another UWB base station among the at least two UWB base stations and the position coordinates of the UWB tag are acquired.

31. The method of claim 20, 21, 23, 25, 26, 27, 28 or 29, wherein: The first input is used to select an identification of an electronic device; the identification of the electronic device corresponds to a parameter of the electronic device.

32. The method according to claim 31, characterized in that The parameters include one or more of a media access control MAC address, an Internet Protocol IP address, a serial number SN or a device identification of the electronic device.

33. The method of claim 20, 21, 23, 25, 26, 27, 28, 29 or 32, wherein: The direction sensor is an inertial measurement unit IMU; the IMU is a 9-axis IMU, and the 9-axis IMU includes an acceleration sensor, a gyroscope sensor, and a magnetic sensor; or the IMU is a 6-axis IMU, and the 6-axis IMU includes an acceleration sensor and a gyroscope sensor.

34. The method according to any one of claims 25 to 29, characterized in that The first direction is a horizontal direction, and the second direction is a vertical direction; or, the first direction is a vertical direction, and the second direction is a horizontal direction.

35. The method of claim 20, 21, 23, 25, 26, 27, 28, 29 or 32, wherein: The mobile device includes: a smart phone and an air mouse.

36. A method for quickly popping up an electronic device control window, the method being applied to a marking device, the marking device wirelessly communicating with the mobile device according to any one of claims 1 to 16 and the electronic device to be marked; The marking device comprises: processor; Memory; At least two UWB base stations, wherein the at least two UWB base stations include a first UWB base station; wherein the method comprises: receiving a ranging message through each of the at least two UWB base stations; In response to the ranging message, sending a ranging response message to the mobile device through each of the at least two UWB base stations; A message is sent to the mobile device through any one of the at least two UWB base stations, the message including the distance between the first UWB base station and each of the other UWB base stations, the ranging response message and the distance between the first UWB base station and each of the other UWB base stations are used by the mobile device to determine a first angle, the first angle representing the angle between a straight line in the direction in which the mobile device is pointing when the mobile device is pointing to the electronic device and a vertical line between the mobile device and the display screen of the marking device.

37. The method according to claim 36, characterized in that The marking device also includes: a display screen; the display screen includes a first edge, a second edge, a third edge and a fourth edge; the first edge is parallel to the second edge, and the third edge is parallel to the fourth edge; the first edge and the second edge are parallel to a first direction, the third edge and the fourth edge are parallel to a second direction, and the first direction is perpendicular to the second direction; the at least two UWB base stations are distributed at different positions in front of or behind the display screen; all or part of the vertical projections of the at least two UWB base stations on the display screen are located on the display screen.

38. The method according to claim 37, characterized in that The at least two UWB base stations include a first UWB base station, a second UWB base station and a third UWB base station; all or part of the vertical projection of the first UWB base station on the display screen is located at the intersection of the third edge and the second edge; all or part of the vertical projection of the second UWB base station on the display screen is located at the intersection of the third edge and the first edge; all or part of the vertical projection of the third UWB base station on the display screen is located at the intersection of the second edge and the fourth edge.

39. A method for quickly popping up an electronic device control window, the method being applied to a system, the system comprising a mobile device, a marking device, and an electronic device to be marked; the mobile device communicates wirelessly with the marking device and the electronic device; the mobile device comprises: an ultra-wideband UWB tag and orientation sensor; The labeling device includes at least two UWB base stations; The at least two UWB base stations include a first UWB base station; The electronic device does not include a UWB base station; the method comprises: The mobile device sends a ranging message via the UWB tag; The marking device receives the ranging message through each of the at least two UWB base stations; In response to the ranging message, the annotation device sends a ranging response message to the mobile device through each of the at least two UWB base stations; The mobile device receives a ranging response message from each of the at least two UWB base stations through the UWB tag; In response to the ranging response messages of the at least two UWB base stations, the mobile device acquires the distance between the UWB tag and each of the at least two UWB base stations; The marking device sends a message to the mobile device through any one of the at least two UWB base stations, wherein the message includes a distance between the first UWB base station and each of the other UWB base stations; The mobile device receives the message from the marking device; The mobile device obtains the distance between the first UWB base station and each other UWB base station from the message; The mobile device selects the position of the first UWB base station among the at least two UWB base stations as the coordinate origin of the three-dimensional coordinate system; The mobile device acquires the position coordinates of another UWB base station among the at least two UWB base stations and the position coordinates of the UWB tag based on the coordinate origin; After the mobile device moves to the location of the electronic device, the mobile device receives a first input; The mobile device records the location of the mobile device in response to the first input; the location of the mobile device is the location of the electronic device; The mobile device determines a first angle according to the distance between the UWB tag and each of the at least two UWB base stations, the distance between the first UWB base station and each of the other UWB base stations, the position coordinates of another UWB base station among the at least two UWB base stations, the position coordinates of the UWB tag, and the position of the electronic device, wherein the first angle represents the angle between a straight line in a pointing direction of the mobile device when the mobile device points to the electronic device and a vertical line between the mobile device and a display screen of the marking device; After the mobile device moves to another position and points to the electronic device, the mobile device determines whether to pop up the first control window of the electronic device based on the first angle and the second angle, where the second angle is the angle between the pointing direction of the mobile device and the vertical line between the mobile device and the display screen.

40. The method according to claim 39, characterized in that Before the mobile device moves to the location of the electronic device, the method further includes: When the mobile device moves to a first position, the mobile device acquires a first initial azimuth angle and a first initial rolling angle of the mobile device based on the direction sensor; When the mobile device moves to the second position, the mobile device acquires a first real-time azimuth angle and a first real-time rolling angle of the mobile device based on the direction sensor; The mobile device acquires a second azimuth in a second direction and a second rolling angle in a first direction based on the first initial azimuth, the first initial rolling angle, the first real-time azimuth, and the first real-time rolling angle; the first direction is perpendicular to the second direction.

41. A system for quickly popping up an electronic device control window, the system comprising a mobile device, a marking device, and an electronic device to be marked; The mobile device communicates wirelessly with the marking device and the electronic device; The mobile device comprises: an ultra-wideband UWB tag and a direction sensor; the labeling device comprises: at least two UWB base stations, the at least two UWB base stations comprising a first UWB base station; the electronic device does not include a UWB base station; The mobile device is used to execute the steps performed by the mobile device in the method as claimed in claim 39 or 40; the labeling device is used to execute the steps performed by the labeling device in the method as claimed in claim 39 or 40.

42. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed on a mobile device, the mobile device executes the method according to any one of claims 20 to 35.

43. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program runs on a labeling device, the labeling device executes the method according to any one of claims 36 to 38.

44. A computer program product, characterized in that When the computer program product is executed on a mobile device, the mobile device is enabled to execute the method according to any one of claims 20 to 35.

45. A computer program product, characterized in that When the computer program product runs on a marking device, the marking device is caused to perform the method according to any one of claims 36 to 38.

Images