Remote control method, remote control device, and controlled device

Abstract

The embodiment of the present application provides a remote control method, a remote control device and a controlled device. In the technical scheme provided by the embodiment of the present application, the controlled device comprises a screen, and the method comprises the following steps: receiving first information of the remote control device; determining a target position of the screen which is faced by the remote control device according to the first information; and performing a first operation on the screen according to the target position. The embodiment of the present application can enable a user to realize remote operation on the controlled device without needing to mechanically and frequently operate a five-way navigation key, thereby improving the experience of the user in the use process.

Description

[Technical Field]

[0001] This invention relates to the field of remote control technology, and more particularly to a remote control method, a remote control device, and a controlled device. [Background Technology]

[0002] Currently, there are two main methods for remotely controlling the controlled device: one is remote operation with a remote control, and the other is using a mobile phone as the remote control for the controlled device.

[0003] Current remote controls, whether infrared or Bluetooth, all include a five-way navigation key. While easy for users to use, the user experience is inferior compared to similar interactive products like smartphones and tablets. With the trend towards larger screens, the amount of content displayed on the screen is increasing, and users often need to perform repetitive, mechanical operations to quickly locate the desired content using the basic five-way navigation key. Using a smartphone as a remote control for a large screen requires the phone to have infrared functionality and corresponding software. Furthermore, using a smartphone as a remote control is limited by distance and infrared emission angle, failing to fundamentally solve the problem of switching back and forth using the five-way navigation key for interaction, resulting in a poor user experience.

[0004] Therefore, the current remote control method for the controlled device requires users to mechanically and frequently operate the five-way navigation key to remotely operate the large screen, resulting in a poor user experience. [Summary of the Invention]

[0005] In view of this, embodiments of the present invention provide a remote control method, a remote control device, and a controlled device, enabling users to remotely operate the controlled device without having to perform frequent mechanical operations on the five-way navigation key, thereby improving the user experience.

[0006] In a first aspect, embodiments of the present invention provide a remote control method applied to a controlled device, the controlled device including a screen, the method comprising:

[0007] Receive the first information from the remote control device;

[0008] The target position of the remote control device facing the screen is determined based on the first information;

[0009] Perform a first operation on the screen according to the target position.

[0010] In one possible implementation, the first information includes the position coordinates of the target position of the remote control device facing the screen;

[0011] Determining the target position of the remote control device facing the screen based on the first information includes:

[0012] Based on the location coordinates, determine the target position on the screen corresponding to the location coordinates.

[0013] In one possible implementation, the location coordinates of the target position are obtained based on an image containing the outline of the screen captured by the binocular camera of the remote control device.

[0014] In one possible implementation, the first information includes an image containing the outline of the screen captured by the binocular camera of the remote control device;

[0015] Determining the target position of the remote control device facing the screen based on the first information includes: determining the target position on the screen corresponding to the position coordinates based on the image.

[0016] In one possible implementation, determining the target position on the screen corresponding to the position coordinates based on the image includes:

[0017] The position coordinates of the target position of the remote control device facing the screen are obtained from the image;

[0018] Based on the location coordinates, determine the target position on the screen corresponding to the location coordinates.

[0019] In one possible implementation, the first operation includes: displaying a pointer at the target location; or highlighting or shadowing the control corresponding to the target location.

[0020] On the other hand, embodiments of the present invention provide a remote control method applied to a remote control device, the method comprising:

[0021] Obtain first information, which is used to determine the target position of the remote control device facing the screen of the controlled device;

[0022] The first information is sent to the controlled device so that the controlled device determines the target location based on the first information and performs a first operation on the screen based on the target location.

[0023] In one possible implementation, the remote control device includes a binocular camera, and the first information includes the position coordinates of the target position of the remote control device facing the screen;

[0024] Obtain first information, including:

[0025] When the remote control device is pointed at the screen, the binocular camera captures an image containing the outline of the screen.

[0026] The position coordinates of the target position of the remote control device facing the screen are obtained from the image.

[0027] In one possible implementation, the remote control device includes a binocular camera, and the first information includes an image captured by the binocular camera that includes the outline of the screen;

[0028] Obtain first information, including:

[0029] When the remote control device is pointed at the screen, the image is captured by the binocular camera.

[0030] In one possible implementation, the first operation includes: displaying a pointer at the target location; or highlighting or shadowing the control corresponding to the target location.

[0031] On the other hand, embodiments of the present invention provide a remote control device applied to a controlled device, the controlled device including a screen, the device comprising:

[0032] The receiving module is used to receive the first information from the remote control device;

[0033] The determining module is used to determine the target position of the remote control device facing the screen based on the first information;

[0034] An execution module is used to perform a first operation on the screen according to the target position.

[0035] In one possible implementation, the first information includes the position coordinates of the target position of the remote control device facing the screen;

[0036] The determining module is specifically used to determine the target position on the screen corresponding to the position coordinates based on the position coordinates.

[0037] In one possible implementation, the location coordinates of the target position are obtained based on an image containing the outline of the screen captured by the binocular camera of the remote control device.

[0038] In one possible implementation, the first information includes an image containing the outline of the screen captured by the binocular camera of the remote control device;

[0039] The determining module is specifically used to determine the target position on the screen corresponding to the position coordinates based on the image.

[0040] In one possible implementation, the determining module is specifically configured to: obtain the position coordinates of the target position of the remote control device facing the screen based on the image, and determine the target position on the screen corresponding to the position coordinates based on the position coordinates.

[0041] In one possible implementation, the first operation includes: displaying a pointer at the target location; or highlighting or shadowing the control corresponding to the target location.

[0042] On the other hand, embodiments of the present invention provide a remote control device for use in remote control equipment, the device comprising:

[0043] The acquisition module is used to acquire first information, which is used to determine the target position of the remote control device facing the screen of the controlled device.

[0044] The sending module is configured to send the first information to the controlled device, so that the controlled device determines the target location based on the first information and performs a first operation on the screen based on the target location.

[0045] In one possible implementation, the remote control device includes a binocular camera, and the first information includes the position coordinates of the target position of the remote control device facing the screen;

[0046] The acquisition module is specifically used to capture an image containing the outline of the screen through the binocular camera when the remote control device is pointed at the screen, and to obtain the position coordinates of the target position of the remote control device facing the screen based on the image.

[0047] In one possible implementation, the remote control device includes a binocular camera, and the first information includes an image captured by the binocular camera that includes the outline of the screen;

[0048] The acquisition module is specifically used to capture the image using the binocular camera when the remote control device is pointed at the screen.

[0049] In one possible implementation, the first operation includes: displaying a pointer at the target location; or highlighting or shadowing the control corresponding to the target location.

[0050] On the other hand, embodiments of the present invention provide a controlled device, including a screen, a processor, and a memory, wherein the memory is used to store a computer program, the computer program including program instructions, and when the processor executes the program instructions, the controlled device performs the following steps:

[0051] Receive the first information from the remote control device;

[0052] The target position of the remote control device facing the screen is determined based on the first information;

[0053] Perform a first operation on the screen according to the target position.

[0054] In one possible implementation, when the processor executes the program instructions, the controlled device performs the following steps:

[0055] The first information includes the position coordinates of the target position of the remote control device facing the screen;

[0056] Determining the target position of the remote control device facing the screen based on the first information includes:

[0057] Based on the location coordinates, determine the target position on the screen corresponding to the location coordinates.

[0058] In one possible implementation, the location coordinates of the target position are obtained based on an image containing the outline of the screen captured by the binocular camera of the remote control device.

[0059] In one possible implementation, when the processor executes the program instructions, the controlled device performs the following steps:

[0060] The first information includes an image containing the outline of the screen captured by the binocular camera of the remote control device;

[0061] Determining the target position of the remote control device facing the screen based on the first information includes: determining the target position on the screen corresponding to the position coordinates based on the image.

[0062] In one possible implementation, when the processor executes the program instructions, the controlled device performs the following steps:

[0063] Determining the target position on the screen corresponding to the position coordinates based on the image includes:

[0064] The position coordinates of the target position of the remote control device facing the screen are obtained from the image;

[0065] Based on the location coordinates, determine the target position on the screen corresponding to the location coordinates.

[0066] In one possible implementation, the first operation includes: displaying a pointer at the target location; or highlighting or shadowing the control corresponding to the target location.

[0067] On the other hand, embodiments of the present invention provide a remote control device, including a processor and a memory, wherein the memory is used to store a computer program, the computer program including program instructions, and when the processor executes the program instructions, the remote control device performs the following steps:

[0068] Obtain first information, which is used to determine the target position of the remote control device facing the screen of the controlled device;

[0069] The first information is sent to the controlled device so that the controlled device determines the target location based on the first information and performs a first operation on the screen based on the target location.

[0070] In one possible implementation, the remote control device includes a binocular camera, and the first information includes the position coordinates of the target location facing the screen; when the processor executes the program instructions, the remote control device performs the following steps:

[0071] Obtain first information, including:

[0072] When the remote control device is pointed at the screen, the binocular camera captures an image containing the outline of the screen.

[0073] The position coordinates of the target position of the remote control device facing the screen are obtained from the image.

[0074] In one possible implementation, the remote control device includes a binocular camera, and the first information includes an image captured by the binocular camera that includes the outline of the screen; when the processor executes the program instructions, the remote control device performs the following steps:

[0075] Obtain first information, including:

[0076] When the remote control device is pointed at the screen, the image is captured by the binocular camera.

[0077] In one possible implementation, the first operation includes: displaying a pointer at the target location; or highlighting or shadowing the control corresponding to the target location.

[0078] On the other hand, embodiments of the present invention provide a computer-readable storage medium storing a computer program, the computer program including program instructions that, when the program requests to be run by a computer, cause the computer to perform the method described above.

[0079] In the remote control method, remote control device, and controlled device provided in this invention embodiment, the controlled device includes a screen, and the method includes: receiving first information from the remote control device; determining a target position of the remote control device facing the screen based on the first information; and performing a first operation on the screen based on the target position. This invention embodiment enables users to remotely operate the controlled device without requiring frequent mechanical operation of the five-way navigation key, thus improving the user experience. [Attached Image Description]

[0080] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0081] Figure 1 This is an architecture diagram of a remote control system provided in an embodiment of the present invention;

[0082] Figure 2 An architecture diagram of a remote control system provided in another embodiment of the present invention;

[0083] Figure 3 This is a diagram illustrating how a remote-controlled device can be pointed at a screen.

[0084] Figure 4 A schematic diagram showing the outline of the screen itself and its outline information;

[0085] Figure 5 This is a schematic diagram illustrating the workflow of the algorithm module;

[0086] Figure 6 A flowchart illustrating a remote control method according to an embodiment of the present invention;

[0087] Figure 7 for Figure 6 The flowchart shows the specific process of executing the first operation corresponding to the first information on the screen based on the first information.

[0088] Figure 8 A flowchart of a remote control method provided in another embodiment of the present invention;

[0089] Figure 9 for Figure 8 The flowchart shows the process of obtaining first information, which is used to determine the target position of the remote control device facing the screen of the controlled device.

[0090] Figure 10 This is a schematic diagram of the structure of an electronic device provided in an embodiment of the present invention.

Detailed Implementation Methods

[0091] To better understand the technical solution of the present invention, the embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0092] It should be understood that the described embodiments are merely some, not all, of the embodiments of the present invention. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0093] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to limit the invention. The singular forms “a,” “the,” and “the” as used in the embodiments of this invention and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise.

[0094] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, or B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0095] Because the controlled device possesses an open operating system and chip, and has an open application platform, it can achieve two-way human-computer interaction, allowing people to freely choose to watch any program they like at their leisure time. Furthermore, with the development of artificial intelligence technology, the controlled device can recommend programs that match the user's viewing habits and preferences, greatly enhancing entertainment value and gradually making it the most important interactive center for living room entertainment. However, the interaction of the controlled device is achieved through remote control.

[0096] Currently, there are two main methods for remotely controlling the controlled device: one is remote operation with a remote control, and the other is using a mobile phone as the remote control for the controlled device.

[0097] Current remote controls, whether infrared or Bluetooth, all include a five-way navigation key. While easy for users to use, the user experience is inferior compared to similar interactive products like smartphones and tablets. With the trend towards larger screens, the amount of content displayed on the screen is increasing, and users often need to perform repetitive, mechanical operations to quickly locate the desired content using the basic five-way navigation key. Using a smartphone as a remote control for a large screen requires the phone to have infrared functionality and corresponding software. Furthermore, using a smartphone as a remote control is limited by distance and infrared emission angle, failing to fundamentally solve the problem of switching back and forth using the five-way navigation key for interaction, resulting in a poor user experience.

[0098] Therefore, the current remote control method for the controlled device requires users to mechanically and frequently operate the five-way navigation key to remotely operate the large screen, resulting in a poor user experience.

[0099] To address the aforementioned technical problems, embodiments of the present invention provide a remote control method, a remote control device, and a controlled device.

[0100] See Figure 1 , Figure 1 This is an architectural diagram of a remote control system provided in an embodiment of the present invention. The remote control system of this embodiment includes two electronic devices, wherein the two electronic devices may include a controlled device and a remote control device, and the communication method between the controlled device and the remote control device includes wireless transmission. Figure 1 As shown, the remote control system includes a remote control device 100 and a controlled device 200. The remote control device 100 and the controlled device 200 can transmit data wirelessly, for example via Bluetooth, infrared, mobile network, or WLAN. The remote control device 100 includes a binocular camera and a transmitting module. The controlled device 200 includes a receiving module, an algorithm module, and a screen.

[0101] Figure 1In this process, the binocular camera of the remote control device 100 is used to capture an image of the screen of the controlled device 200, the image including the outline of the screen, and the image is sent to the sending module; the sending module is used to send first information to the controlled device 200 via wireless transmission. The first information includes an image including the outline of the screen captured by the binocular camera of the remote control device 100. The receiving module of the controlled device 200 is used to receive the first information and send it to the algorithm module. The algorithm module determines the position coordinates of the target position of the remote control device 100 facing the screen based on the first information. Specifically, the algorithm module obtains the position coordinates of the target position of the remote control device 100 facing the screen based on the image and sends the position coordinates to the screen. The screen is used to determine the target position on the screen corresponding to the position coordinates based on the position coordinates, and performs a first operation on the screen according to the target position.

[0102] See Figure 2 , Figure 2 This is an architectural diagram of a remote control system according to another embodiment of the present invention. The remote control system of this embodiment includes two electronic devices, wherein the two electronic devices may include a controlled device and a remote control device, and the communication method between the controlled device and the remote control device includes wireless transmission. Figure 1 As shown, the remote control system includes a remote control device 100 and a controlled device 200. The remote control device 100 and the controlled device 200 can transmit data wirelessly, for example via Bluetooth, infrared, mobile network, or WLAN. The remote control device 100 includes a binocular camera, an algorithm module, and a transmitting module. The controlled device 200 includes a receiving module and a screen.

[0103] Figure 2 In this process, the binocular camera of the remote control device 100 captures an image of the screen of the controlled device 200, the image containing the outline of the screen, and sends the image containing the screen outline to the algorithm module. The algorithm module obtains the position coordinates of the target position of the remote control device 100 facing the screen based on the image, and sends first information to the sending module. The first information includes the position coordinates of the target position of the remote control device 100 facing the screen. The sending module transmits the first information to the controlled device 200 wirelessly. The receiving module of the controlled device 200 receives the first information and sends it to the screen. The screen determines the target position of the remote control device facing the screen based on the first information; and performs a first operation on the screen based on the target position. Specifically, the screen determines the target position on the screen corresponding to the position coordinates based on the position coordinates, and performs the first operation on the screen based on the target position.

[0104] exist Figure 1 and Figure 2 In the middle, the remote control device 100 includes a binocular camera. Specifically, such as... Figure 3As shown, the binocular camera is located at the front of the remote control device 100 and is capable of recognizing the outline of the screen. The binocular camera captures two images of the screen, each containing the screen's outline, as shown below. Figure 4 As shown, the image on the left, which includes the outline of the screen, was taken by the left eye camera, and the image on the right, which also includes the outline of the screen, was taken by the right eye camera.

[0105] For example, binocular cameras include binocular depth cameras.

[0106] It should be noted that, as Figure 3 As shown, when the remote control device 100 is facing the screen, the line connecting the center point of the remote control device 100 and the target position on the screen is parallel to the two longer sides of the remote control device 100.

[0107] Among them, such as Figure 5 As shown, the algorithm module is specifically used for: calibrating the binocular camera in offline mode; removing distortion caused by the binocular camera by correcting two images on the screen; calculating the matching points between the two corrected images through binocular matching to obtain a disparity map; and calculating the position coordinates of the remote control device 100 facing the screen using 3D coordinates based on the disparity map.

[0108] The purpose of calibrating the binocular cameras offline is to align the two cameras by obtaining their intrinsic and extrinsic parameters. First, the left camera is calibrated, obtaining its intrinsic and extrinsic parameters. Second, the right camera is calibrated, obtaining its intrinsic and extrinsic parameters. Finally, the binocular cameras are calibrated, obtaining the translation and rotation relationship between the left and right cameras. The intrinsic parameters include focal length, image center, and distortion coefficients; the extrinsic parameters include rotation and translation matrices.

[0109] By correcting the two images on the screen, the difference between them in the X-direction can be obtained, which can improve the accuracy of parallax calculation. Specifically, the correction of the two images on the screen includes distortion correction and conversion to a standard form.

[0110] Binocular matching is the core component of binocular depth estimation, and its main purpose is to calculate the relative matching relationship of pixels between two images. Specifically, binocular matching includes five steps: matching error calculation, error ensemble, disparity map calculation, disparity map optimization, and disparity map correction.

[0111] Specifically, the three-dimensional information of the screen in the image is reconstructed based on the matching information in the disparity map and the principle of triangulation, and the position coordinates of the remote control device 100 facing the screen are obtained by 3D coordinate calculation.

[0112] The first operation includes: displaying a pointer at the target location; or highlighting or shadowing the control corresponding to the target location.

[0113] It should be noted that when the user moves the remote control device 100, the image containing the screen outline captured by the remote control device 100 will change, and the position coordinates of the remote control device 100 facing the screen need to be recalculated. The controlled device 200 refreshes the pointer position according to the new position coordinates, and the pointer moves according to the direction of the remote control device; or, the controlled device 200 refreshes the highlighted or shaded controls according to the new position coordinates.

[0114] Based on the above Figure 1 and Figure 2 The provided architecture diagram shows that an embodiment of the present invention provides a remote control method applied to a controlled device 200, wherein the control device 200 includes a screen. Figure 6 This is a flowchart illustrating a remote control method according to an embodiment of the present invention. Figure 6 As shown, the method includes:

[0115] Step 102: Receive the first information from the remote control device.

[0116] The first information includes the position coordinates of the target position of the remote control device facing the screen when the remote control device is pointed at the screen; or, the first information includes an image containing the outline of the screen captured by the binocular camera of the remote control device.

[0117] In embodiments of the present invention, such as Figure 1 As shown, the first information includes an image containing the outline of the screen captured by the binocular camera of the remote control device. The receiving module of the controlled device 200 receives the first information from the remote control device 100 and then sends the first information to the algorithm module.

[0118] In embodiments of the present invention, such as Figure 2 As shown, the first information includes the position coordinates of the target position of the remote control device 100 facing the screen. The receiving module of the controlled device 200 receives the first information sent by the remote control device 100 and then sends the first information to the screen.

[0119] The remote control device includes a binocular camera, and the target location's coordinates are obtained from an image containing the screen's outline captured by the binocular camera. Specifically, the binocular camera is located at the front of the remote control device and is capable of recognizing the screen's outline, capturing two images of the screen's outline.

[0120] For example, binocular cameras include binocular depth cameras.

[0121] Step 104: Determine the target position of the remote control device facing the screen based on the first information.

[0122] As an optional solution, the first information includes the position coordinates of the target position facing the screen of the remote control device; step 104 includes: determining the target position on the screen corresponding to the position coordinates based on the position coordinates.

[0123] like Figure 2 As shown, the receiving module of the controlled device 200 sends the first information to the screen, and the screen determines the target position corresponding to the position coordinates based on the position coordinates in the first information.

[0124] As an alternative, the first information includes an image containing the outline of the screen captured by the binocular camera of the remote control device; step 104 includes: determining the target position on the screen corresponding to the position coordinates based on the image.

[0125] Specifically, such as Figure 7 As shown, step 104 includes:

[0126] Step 1042: Obtain the position coordinates of the target position of the remote control device facing the screen based on the image.

[0127] like Figure 1 As shown, the algorithm module of the controlled device 200 obtains the position coordinates of the target position of the remote control device 100 facing the screen based on the image, and then sends the position coordinates to the screen.

[0128] Specifically, the algorithm module calibrates the binocular camera in offline mode; it corrects the two images on the screen to remove the distortion caused by the binocular camera; it calculates the matching points between the two corrected images through binocular matching to obtain a disparity map; and it calculates the position coordinates of the remote control device 100 facing the screen using 3D coordinates based on the disparity map.

[0129] Step 1044: Determine the target position on the screen corresponding to the position coordinates based on the position coordinates.

[0130] like Figure 1 As shown, the algorithm module of the controlled device 200 sends the position coordinates to the screen, and the screen determines the target position on the screen corresponding to the position coordinates based on the position coordinates in the first information.

[0131] Step 106: Perform the first operation on the screen according to the target position.

[0132] like Figure 1 and 2 As shown, the screen of the controlled device 200 performs a first operation at the target position corresponding to the position coordinates on the screen.

[0133] The first operation includes: displaying a pointer at the target location; or highlighting or shadowing the control corresponding to the target location.

[0134] It should be noted that when the user moves the remote control device, the image captured by the remote control device changes, and the position coordinates need to be recalculated. The controlled device refreshes the pointer position according to the new position coordinates, and the pointer moves according to the direction of the remote control device; or, the controlled device refreshes the highlighted or shaded controls according to the new position coordinates.

[0135] In the remote control method provided by this invention, first information from the remote control device is received; a target position of the remote control device facing the screen is determined based on the first information; and a first operation is performed on the screen based on the target position. This invention enables users to remotely operate the controlled device without requiring frequent mechanical operation of the five-way navigation key, thus improving the user experience.

[0136] Based on the above Figure 1 and Figure 2 The provided architecture diagram illustrates a remote control method applied to a remote control device 100. Figure 8 This is a flowchart illustrating a remote control method according to another embodiment of the present invention. Figure 8 As shown, the method includes:

[0137] Step 202: Obtain first information, which is used to determine the target position of the remote control device facing the screen of the controlled device.

[0138] As an optional solution, the remote control device includes a binocular camera, and the first information includes the position coordinates of the target location where the remote control device is facing the screen; such as Figure 9 As shown, step 202 includes:

[0139] Step 2022: When the remote control device is pointed at the screen, an image containing the outline of the screen is captured by the binocular camera.

[0140] The binocular camera is located at the front of the remote control device and can recognize the outline of the screen. It captures two images of the screen containing the outline of the screen.

[0141] For example, binocular cameras include binocular depth cameras.

[0142] In embodiments of the present invention, such as Figure 2 As shown, the remote control device 100 captures an image containing the outline of the screen using a binocular camera and sends the image to the algorithm module.

[0143] Step 2024: Obtain the position coordinates of the target position of the remote control device facing the screen based on the image.

[0144] In embodiments of the present invention, such as Figure 2As shown, the remote control device 100 captures an image containing the outline of the screen using a binocular camera and sends the image to the algorithm module. The algorithm module determines the position coordinates of the target position of the remote control device facing the screen based on the image and sends the position coordinates to the sending module. Specifically, the algorithm module calibrates the binocular camera offline; it corrects the two images of the screen to remove distortion caused by the binocular camera; it calculates the matching points between the two corrected images through binocular matching to obtain a disparity map; and based on the disparity map, it calculates the position coordinates of the remote control device 100 facing the screen using 3D coordinates. The sending module generates first information based on the position coordinates.

[0145] As an alternative, the remote control device includes a binocular camera, the first information including two images containing the outline of the screen captured by the binocular camera, and step 202 including: capturing an image by the binocular camera when the remote control device is pointed at the screen.

[0146] In embodiments of the present invention, such as Figure 1 As shown, the remote control device 100 captures an image containing the outline of the screen using a binocular camera and sends the image to the sending module, which generates first information based on the image.

[0147] Step 204: Send the first information to the controlled device so that the controlled device can determine the target location based on the first information and perform the first operation on the screen based on the target location.

[0148] like Figure 1 As shown, the first information includes an image containing the outline of the screen captured by the binocular camera of the remote control device 100. The transmitting module of the remote control device 100 sends the first information to the receiving module of the controlled device 200. The receiving module receives the first information and sends it to the algorithm module. The algorithm module determines the position coordinates of the target position of the remote control device 100 facing the screen based on the first information and sends the position coordinates to the screen. Specifically, the algorithm module calibrates the binocular camera in offline mode; corrects the two images on the screen to remove distortion caused by the binocular camera; calculates the matching points between the two corrected images through binocular matching to obtain a disparity map; and calculates the position coordinates of the remote control device 100 facing the screen using 3D coordinates based on the disparity map. The screen determines the target position on the screen corresponding to the position coordinates based on the position coordinates and performs a first operation on the screen based on the target position.

[0149] like Figure 2As shown, the first information includes the position coordinates of the target position of the remote control device 100 facing the screen. The transmitting module of the remote control device 100 sends the first information to the receiving module of the controlled device 200. The receiving module receives the first information and sends it to the screen. Based on the position coordinates, the screen determines the target position on the screen corresponding to the position coordinates, and performs a first operation on the screen according to the target position.

[0150] The first operation includes: displaying a pointer at the target location; or highlighting or shadowing the control corresponding to the target location.

[0151] It should be noted that when the user moves the remote control device, the image captured by the remote control device on the screen changes, and the position coordinates need to be recalculated. The controlled device refreshes the pointer position according to the new position coordinates, and the pointer moves according to the direction of the remote control device; or, the controlled device refreshes the highlighted or shaded controls according to the new position coordinates.

[0152] In the remote control method provided by this invention, the remote control device sends first information to the controlled device, enabling the controlled device to determine the target position of the remote control device facing the screen based on the first information, and then perform a first operation on the screen according to the target position. This invention allows users to remotely operate the controlled device without requiring frequent mechanical operation of the five-way navigation key, thus improving the user experience.

[0153] The above text combined Figures 1 to 9 The remote control method provided by the embodiments of the present invention is described in detail below. Figure 10 The following describes in detail the device embodiments of the present invention. It should be understood that the electronic devices in the embodiments of the present invention can execute the various methods of the foregoing embodiments of the present invention, that is, the specific working processes of the various products below can be referred to the corresponding processes in the foregoing method embodiments.

[0154] This invention provides an electronic device, which can be a terminal device or a circuit device built into the terminal device. This electronic device can be used to perform the functions / steps described in the method embodiments above.

[0155] Figure 10This is a schematic diagram of the structure of an electronic device 300 provided in an embodiment of the present invention. The electronic device 300 may include a processor 310, an external memory interface 320, an internal memory 321, a universal serial bus (USB) interface 330, a charging management module 340, a power management module 341, a battery 342, antenna 1, antenna 2, a mobile communication module 350, a wireless communication module 360, an audio module 370, a speaker 370A, a receiver 370B, a microphone 370C, a headphone jack 370D, a sensor module 380, buttons 390, a motor 391, an indicator 392, a camera 393, a display screen 394, and a subscriber identification module (SIM) card interface 395, etc. The sensor module 380 may include a pressure sensor 380A, a gyroscope sensor 380B, a barometric pressure sensor 380C, a magnetic sensor 380D, an accelerometer sensor 380E, a distance sensor 380F, a proximity light sensor 380G, a fingerprint sensor 380H, a temperature sensor 380J, a touch sensor 380K, an ambient light sensor 380L, a bone conduction sensor 380M, etc.

[0156] It is understood that the structures illustrated in the embodiments of the present invention do not constitute a specific limitation on the electronic device 300. In other embodiments of the present invention, the electronic device 300 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0157] Processor 310 may include one or more processing units, such as application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU). These different processing units may be independent devices or integrated into one or more processors.

[0158] The controller can generate operation control signals based on the instruction opcode and timing signals to complete the control of instruction fetching and execution.

[0159] The processor 310 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 310 is a cache memory. This memory can store instructions or data that the processor 310 has just used or that are used repeatedly. If the processor 310 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 310, and thus improves the efficiency of the system.

[0160] In some embodiments, the processor 310 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0161] The I2C interface is a bidirectional synchronous serial bus, including a serial data line (SDA) and a serial clock line (SCL). In some embodiments, the processor 310 may include multiple I2C buses. The processor 310 can couple to the touch sensor 380K, charger, flash, camera 393, etc., through different I2C bus interfaces. For example, the processor 310 can couple to the touch sensor 380K through the I2C interface, enabling the processor 310 and the touch sensor 380K to communicate through the I2C bus interface, thereby realizing the touch function of the electronic device 300.

[0162] The I2S interface can be used for audio communication. In some embodiments, the processor 310 may include multiple I2S buses. The processor 310 can be coupled to the audio module 370 via the I2S bus to enable communication between the processor 310 and the audio module 370. In some embodiments, the audio module 370 can transmit audio signals to the wireless communication module 360 ​​via the I2S interface to enable the function of answering phone calls through a Bluetooth headset.

[0163] The PCM interface can also be used for audio communication, sampling, quantizing, and encoding analog signals. In some embodiments, the audio module 370 and the wireless communication module 360 ​​can be coupled via the PCM bus interface. In some embodiments, the audio module 370 can also transmit audio signals to the wireless communication module 360 ​​via the PCM interface, enabling the function of answering phone calls through a Bluetooth headset. Both the I2S interface and the PCM interface can be used for audio communication.

[0164] The UART interface is a universal serial data bus used for asynchronous communication. This bus can be a bidirectional communication bus. It converts the data to be transmitted between serial and parallel communication. In some embodiments, the UART interface is typically used to connect the processor 310 and the wireless communication module 360. For example, the processor 310 communicates with the Bluetooth module in the wireless communication module 360 ​​via the UART interface to implement Bluetooth functionality. In some embodiments, the audio module 370 can transmit audio signals to the wireless communication module 360 ​​via the UART interface to enable music playback through Bluetooth headphones.

[0165] The MIPI interface can be used to connect the processor 310 to peripheral devices such as the display screen 394 and the camera 393. The MIPI interface includes a camera serial interface (CSI) and a display serial interface (DSI). In some embodiments, the processor 310 and the camera 393 communicate via the CSI interface to enable the electronic device 300 to capture images. The processor 310 and the display screen 394 communicate via the DSI interface to enable the electronic device 300 to display images.

[0166] The GPIO interface can be configured via software. It can be configured as a control signal or a data signal. In some embodiments, the GPIO interface can be used to connect the processor 310 to a camera 393, a display screen 394, a wireless communication module 360, an audio module 370, a sensor module 380, etc. The GPIO interface can also be configured as an I2C interface, an I2S interface, a UART interface, a MIPI interface, etc.

[0167] USB port 330 is a USB standard compliant interface, which can be a Mini USB port, Micro USB port, USB Type-C port, etc. USB port 330 can be used to connect a charger to charge electronic device 300, and can also be used for data transfer between electronic device 300 and peripheral devices. It can also be used to connect headphones for audio playback. This interface can also be used to connect other electronic devices, such as AR devices.

[0168] It is understood that the interface connection relationships between the modules illustrated in the embodiments of the present invention are merely illustrative and do not constitute a structural limitation on the electronic device 300. In other embodiments of the present invention, the electronic device 300 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0169] The charging management module 340 receives charging input from a charger. The charger can be a wireless charger or a wired charger. In some wired charging embodiments, the charging management module 340 receives charging input from the wired charger via a USB interface 330. In some wireless charging embodiments, the charging management module 340 receives wireless charging input via the wireless charging coil of the electronic device 300. While charging the battery 342, the charging management module 340 can also supply power to the electronic device via the power management module 341.

[0170] The power management module 341 connects the battery 342, the charging management module 340, and the processor 310. The power management module 341 receives input from the battery 342 and / or the charging management module 340, providing power to the processor 310, internal memory 321, display screen 394, camera 393, and wireless communication module 360. The power management module 341 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 341 may be located within the processor 310. In other embodiments, the power management module 341 and the charging management module 340 may be housed in the same device.

[0171] The wireless communication function of electronic device 300 can be realized through antenna 1, antenna 2, mobile communication module 350, wireless communication module 360, modem processor and baseband processor, etc.

[0172] Antenna 1 and antenna 2 are used to transmit and receive electromagnetic wave signals. Each antenna in electronic device 300 can be used to cover one or more communication frequency bands. Different antennas can also be multiplexed to improve antenna utilization. For example, antenna 1 can be multiplexed as a diversity antenna for a wireless local area network. In some other embodiments, the antennas can be used in conjunction with a tuning switch.

[0173] The mobile communication module 350 can provide solutions for wireless communication, including 2G / 3G / 4G / 5G, applied to the electronic device 300. The mobile communication module 350 may include at least one filter, switch, power amplifier, low noise amplifier (LNA), etc. The mobile communication module 350 can receive electromagnetic waves via antenna 1, and perform filtering, amplification, and other processing on the received electromagnetic waves before transmitting them to a modem processor for demodulation. The mobile communication module 350 can also amplify the signal modulated by the modem processor and convert it into electromagnetic waves for radiation via antenna 1. In some embodiments, at least some functional modules of the mobile communication module 350 may be housed in the processor 310. In some embodiments, at least some functional modules of the mobile communication module 350 and at least some modules of the processor 310 may be housed in the same device.

[0174] The modem processor may include a modulator and a demodulator. The modulator modulates the low-frequency baseband signal to be transmitted into a mid-to-high frequency signal. The demodulator demodulates the received electromagnetic wave signal into a low-frequency baseband signal. The demodulator then transmits the demodulated low-frequency baseband signal to the baseband processor for processing. After processing by the baseband processor, the low-frequency baseband signal is transmitted to the application processor. The application processor outputs sound signals through an audio device (not limited to speaker 370A, receiver 370B, etc.) or displays images or videos through a display screen 394. In some embodiments, the modem processor may be a separate device. In other embodiments, the modem processor may be independent of the processor 310 and may be housed in the same device as the mobile communication module 350 or other functional modules.

[0175] The wireless communication module 360 ​​can provide solutions for wireless communication applications on electronic devices 300, including wireless local area networks (WLAN) (such as wireless fidelity (Wi-Fi) networks), Bluetooth (BT), global navigation satellite system (GNSS), frequency modulation (FM), near field communication (NFC), and infrared (IR) technologies.

[0176] The wireless communication module 360 ​​may be one or more devices integrating at least one communication processing module. The wireless communication module 360 ​​receives electromagnetic waves via antenna 2, modulates and filters the electromagnetic wave signals, and sends the processed signal to processor 310.

[0177] The wireless communication module 360 ​​can also receive the signal to be transmitted from the processor 310, frequency modulate it, amplify it, and convert it into electromagnetic waves for radiation via the antenna 2.

[0178] In some embodiments, antenna 1 of electronic device 300 is coupled to mobile communication module 350, and antenna 2 is coupled to wireless communication module 360, enabling electronic device 300 to communicate with networks and other devices via wireless communication technology. The wireless communication technology may include Global System for Mobile Communications (GSM), General Packet Radio Service (GPRS), Code Division Multiple Access (CDMA), Wideband Code Division Multiple Access (WCDMA), Time Division Code Division Multiple Access (TD-SCDMA), Long Term Evolution (LTE), BT, GNSS, WLAN, NFC, FM, and / or IR technologies, etc. The GNSS may include the Global Positioning System (GPS), the Global Navigation Satellite System (GLONASS), the BeiDou Navigation Satellite System (BDS), the Quasi-Zenith Satellite System (QZSS), and / or satellite-based augmentation systems (SBAS).

[0179] Electronic device 300 implements display functions through a GPU, a display screen 394, and an application processor. The GPU is a microprocessor for image processing, connecting the display screen 394 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 310 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0180] Display screen 394 is used to display images, videos, etc. Display screen 394 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 300 may include one or N displays 394, where N is a positive integer greater than 1.

[0181] Electronic device 300 can achieve shooting function through ISP, camera 393, video codec, GPU, display 394 and application processor.

[0182] The ISP (Image Signal Processor) is used to process data fed back from the camera 393. For example, when taking a picture, the shutter is opened, and light is transmitted through the lens to the camera's photosensitive element. The light signal is converted into an electrical signal, and the camera's photosensitive element transmits the electrical signal to the ISP for processing, transforming it into an image visible to the naked eye. The ISP can also perform algorithmic optimization of image noise, brightness, and skin tone. The ISP can also optimize parameters such as exposure and color temperature of the shooting scene. In some embodiments, the ISP can be set in the camera 393.

[0183] Camera 393 is used to capture still images or videos. An object is projected onto a photosensitive element by generating an optical image through the lens. The photosensitive element can be a charge-coupled device (CCD) or a complementary metal-oxide-semiconductor (CMOS) phototransistor. The photosensitive element converts the light signal into an electrical signal, which is then passed to an ISP for conversion into a digital image signal. The ISP outputs the digital image signal to a DSP for processing. The DSP converts the digital image signal into image signals in standard RGB, YUV, or other formats. In some embodiments, the electronic device 300 may include one or N cameras 393, where N is a positive integer greater than 1.

[0184] A digital signal processor (DSP) is used to process digital signals. Besides digital image signals, it can also process other digital signals. For example, when the electronic device 300 is selecting a frequency, the DSP is used to perform Fourier transforms on the frequency energy.

[0185] Video codecs are used to compress or decompress digital video. Electronic device 300 may support one or more video codecs. Thus, electronic device 300 can play or record video in various encoding formats, such as Moving Picture Experts Group (MPEG) 1, MPEG2, MPEG3, MPEG4, etc.

[0186] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs can enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0187] The external storage interface 320 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 300. The external memory card communicates with the processor 310 through the external storage interface 320 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0188] Internal memory 321 can be used to store computer executable program code, which includes instructions. Internal memory 321 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application program required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 300 (such as audio data, phonebook, etc.). Furthermore, internal memory 321 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc. Processor 310 executes various functional applications and data processing of electronic device 300 by running instructions stored in internal memory 321 and / or instructions stored in memory located in the processor.

[0189] Electronic device 300 can implement audio functions such as music playback and recording through audio module 370, speaker 370A, receiver 370B, microphone 370C, headphone jack 370D, and application processor.

[0190] The audio module 370 is used to convert digital audio information into analog audio signal output, and also to convert analog audio input into digital audio signal. The audio module 370 can also be used for encoding and decoding audio signals. In some embodiments, the audio module 370 may be located in the processor 310, or some functional modules of the audio module 370 may be located in the processor 310.

[0191] The speaker 370A, also known as a "loudspeaker," is used to convert audio electrical signals into sound signals. Electronic device 300 can listen to music or make hands-free calls through the speaker 370A.

[0192] The receiver 370B, also known as the "earpiece," is used to convert audio electrical signals into sound signals. When the electronic device 300 answers a telephone call or voice message, the receiver 370B can be brought close to the listener's ear to hear the voice.

[0193] Microphone 370C, also known as a "microphone" or "voice transducer," is used to convert sound signals into electrical signals. When making a phone call or sending a voice message, the user can speak by bringing their mouth close to microphone 370C, inputting the sound signal into microphone 370C. Electronic device 300 may have at least one microphone 370C. In some embodiments, electronic device 300 may have two microphones 370C, which, in addition to collecting sound signals, can also perform noise reduction. In other embodiments, electronic device 300 may have three, four, or more microphones 370C, which can collect sound signals, reduce noise, identify the sound source, and perform directional recording, etc.

[0194] The 370D headphone jack is used to connect wired headphones. The 370D headphone jack can be a USB 330 interface or a 3.5mm Open Mobile Terminal Platform (OMTP) standard interface, a CTIA (Cellular Telecommunications Industry Association of the USA) standard interface.

[0195] The pressure sensor 380A is used to sense pressure signals and can convert the pressure signals into electrical signals. In some embodiments, the pressure sensor 380A may be disposed on the display screen 394.

[0196] There are many types of pressure sensors 380A, such as resistive pressure sensors, inductive pressure sensors, and capacitive pressure sensors. A capacitive pressure sensor may include at least two parallel plates with conductive material. When force is applied to the pressure sensor 380A, the capacitance between the electrodes changes. The electronic device 300 determines the pressure intensity based on the change in capacitance. When a touch operation is applied to the display screen 394, the electronic device 300 detects the intensity of the touch operation based on the pressure sensor 380A. The electronic device 300 can also calculate the touch position based on the detection signal from the pressure sensor 380A. In some embodiments, touch operations applied to the same touch position but with different intensities can correspond to different operation commands. For example, when a touch operation with an intensity less than a first pressure threshold is applied to the SMS application icon, a command to view an SMS message is executed. When a touch operation with an intensity greater than or equal to the first pressure threshold is applied to the SMS application icon, a command to create a new SMS message is executed.

[0197] The gyroscope sensor 380B can be used to determine the motion attitude of the electronic device 300. In some embodiments, the gyroscope sensor 380B can determine the angular velocity of the electronic device 300 around three axes (i.e., the x, y, and z axes). The gyroscope sensor 380B can be used for image stabilization. For example, when the shutter is pressed, the gyroscope sensor 380B detects the angle of the electronic device 300's shake, calculates the distance that the lens module needs to compensate based on the angle, and allows the lens to counteract the shake of the electronic device 300 through reverse movement, thus achieving image stabilization. The gyroscope sensor 380B can also be used in navigation and motion-sensing game scenarios.

[0198] The barometric pressure sensor 380C is used to measure air pressure. In some embodiments, the electronic device 300 calculates altitude using the air pressure value measured by the barometric pressure sensor 380C to assist in positioning and navigation.

[0199] The magnetic sensor 380D includes a Hall sensor. The electronic device 300 can use the magnetic sensor 380D to detect the opening and closing of the flip cover. In some embodiments, when the electronic device 300 is a flip phone, the electronic device 300 can detect the opening and closing of the flip cover using the magnetic sensor 380D. Then, based on the detected opening and closing state of the cover or the flip cover, features such as automatic flip unlocking can be set.

[0200] The accelerometer 380E can detect the magnitude of acceleration of an electronic device 300 in various directions (typically three axes). When the electronic device 300 is stationary, it can detect the magnitude and direction of gravity. It can also be used to identify the posture of the electronic device, and can be applied to applications such as screen orientation switching and pedometers.

[0201] A distance sensor 380F is used to measure distance. Electronic device 300 can measure distance via infrared or laser. In some embodiments, during a shooting scene, electronic device 300 can utilize the distance sensor 380F to measure distance for rapid focusing.

[0202] The proximity sensor 380G may include, for example, a light-emitting diode (LED) and a light detector, such as a photodiode. The LED may be an infrared LED. The electronic device 300 emits infrared light outward through the LED. The electronic device 300 uses the photodiode to detect infrared reflected light from nearby objects. When sufficient reflected light is detected, it can be determined that an object is near the electronic device 300. When insufficient reflected light is detected, the electronic device 300 can determine that no object is near the electronic device 300. The electronic device 300 can use the proximity sensor 380G to detect when a user holds the electronic device 300 close to their ear for a call, so as to automatically turn off the screen to save power. The proximity sensor 380G can also be used in holster mode and pocket mode for automatic unlocking and locking of the screen.

[0203] The ambient light sensor 380L is used to sense the brightness of ambient light. The electronic device 300 can adaptively adjust the brightness of its display screen 394 based on the sensed ambient light level. The ambient light sensor 380L can also be used to automatically adjust the white balance when taking photos. The ambient light sensor 380L can also work in conjunction with the proximity sensor 380G to detect whether the electronic device 300 is in a pocket, preventing accidental touches.

[0204] The fingerprint sensor 380H is used to collect fingerprints. The electronic device 300 can utilize the characteristics of the collected fingerprints to achieve fingerprint unlocking, accessing application locks, taking photos with fingerprints, answering calls with fingerprints, etc.

[0205] Temperature sensor 380J is used to detect temperature. In some embodiments, electronic device 300 uses the temperature detected by temperature sensor 380J to execute a temperature handling strategy. For example, when the temperature reported by temperature sensor 380J exceeds a threshold, electronic device 300 performs thermal protection by reducing the performance of a processor located near temperature sensor 380J to reduce power consumption. In other embodiments, when the temperature is below another threshold, electronic device 300 heats battery 342 to prevent abnormal shutdown of electronic device 300 due to low temperature. In still other embodiments, when the temperature is below yet another threshold, electronic device 300 boosts the output voltage of battery 342 to prevent abnormal shutdown due to low temperature.

[0206] Touch sensor 380K, also known as a "touch device," can be located on display screen 394. The touch sensor 380K and display screen 394 together form a touchscreen, also known as a "touchscreen." Touch sensor 380K detects touch operations applied to or near it. The touch sensor can transmit the detected touch operation to the application processor to determine the type of touch event. Visual output related to the touch operation can be provided through display screen 394. In other embodiments, touch sensor 380K may also be located on the surface of electronic device 300, in a different position than display screen 394.

[0207] The bone conduction sensor 380M can acquire vibration signals. In some embodiments, the bone conduction sensor 380M can acquire vibration signals from the vibrating bone segments of the human vocal cords. The bone conduction sensor 380M can also contact the human pulse to receive blood pressure signals. In some embodiments, the bone conduction sensor 380M can also be incorporated into headphones to form bone conduction headphones. The audio module 370 can parse the voice signals from the vibrating bone segments of the vocal cords acquired by the bone conduction sensor 380M to realize voice functionality. The application processor can parse heart rate information from the blood pressure signals acquired by the bone conduction sensor 380M to realize heart rate detection functionality.

[0208] Buttons 390 include a power button, volume buttons, etc. Buttons 390 can be mechanical buttons or touch-sensitive buttons. Electronic device 300 can receive button input and generate key signal inputs related to user settings and function control of electronic device 300.

[0209] Motor 391 can generate vibration alerts. Motor 391 can be used for incoming call vibration alerts or for touch vibration feedback. For example, different vibration feedback effects can be corresponding to touch operations applied to different applications (such as taking photos, playing audio, etc.). Motor 391 can also correspond to different vibration feedback effects for touch operations applied to different areas of the display screen 394. Different application scenarios (such as time reminders, receiving messages, alarm clocks, games, etc.) can also correspond to different vibration feedback effects. The touch vibration feedback effect can also be customized.

[0210] Indicator 392 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0211] The SIM card interface 395 is used to connect a SIM card. The SIM card can be inserted into or removed from the SIM card interface 395 to make contact with and separate from the electronic device 300. The electronic device 300 can support one or N SIM card interfaces, where N is a positive integer greater than 1. The SIM card interface 395 can support Nano SIM cards, Micro SIM cards, SIM cards, etc. Multiple cards can be inserted into the same SIM card interface 395 simultaneously. The multiple cards can be of the same or different types. The SIM card interface 395 is also compatible with different types of SIM cards. The SIM card interface 395 is also compatible with external memory cards. The electronic device 300 interacts with the network through the SIM card to realize functions such as calls and data communication. In some embodiments, the electronic device 300 uses an eSIM, i.e., an embedded SIM card. The eSIM card can be embedded in the electronic device 300 and cannot be separated from the electronic device 300.

[0212] This invention provides a computer-readable storage medium storing instructions that, when executed on a terminal device, cause the terminal device to perform the functions / steps described in the above method embodiments.

[0213] This invention also provides a computer program product containing instructions that, when run on a computer or any at least one processor, cause the computer to perform the functions / steps described in the above method embodiments.

[0214] In this embodiment of the invention, "at least one" refers to one or more, and "more than one" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent the existence of A alone, the simultaneous existence of A and B, or the existence of B alone. A and B can be singular or plural. The character " / " generally indicates that the preceding and following related objects are in an "or" relationship. "At least one of the following" and similar expressions refer to any combination of these items, including any combination of single or plural items. For example, at least one of a, b, and c can represent: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or multiple.

[0215] Those skilled in the art will recognize that the units and algorithm steps described in the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of electronic hardware and software. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementations should not be considered beyond the scope of this invention.

[0216] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0217] In several embodiments provided by this invention, any function, if implemented as a software functional unit and sold or used as an independent product, can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this invention, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause an electronic device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0218] The above description is merely a specific embodiment of the present invention. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this invention should be included within the protection scope of this invention. The protection scope of this invention should be determined by the scope of the claims.

Claims

1. A remote control method, characterized in that, Applied to a controlled device, the controlled device including a screen, the method includes: The system receives first information from a remote control device. This first information includes the position coordinates of the target position of the remote control device facing the screen. The position coordinates are coordinates obtained by the remote control device based on a first contour image and a second contour image of the screen. The first contour image and the second contour image are contour images of the screen captured by a binocular camera on the remote control device. The coordinates obtained by the remote control device based on the first contour image and the second contour image include: calibrating the binocular camera offline, correcting the distortion caused by the binocular camera on the first contour image and the second contour image, calculating the matching points between the corrected first contour image and the second contour image using binocular matching to obtain a disparity map, and calculating the position coordinates based on the disparity map using 3D coordinates. The target position of the remote control device facing the screen is determined based on the first information; Perform a first operation on the screen according to the target position.

2. The method according to claim 1, characterized in that, Determining the target position of the remote control device facing the screen based on the first information includes: Based on the location coordinates, determine the target position on the screen corresponding to the location coordinates.

3. The method according to claim 1 or 2, characterized in that, The first operation includes: displaying a pointer at the target location; or highlighting or shadowing the control corresponding to the target location.

4. A remote control method, characterized in that, Applied to remote control devices, the method includes: The binocular camera on the remote control device is calibrated in an offline state; When the remote control device is pointed at the screen of the controlled device, the binocular camera captures a first outline image and a second outline image containing the screen. The distortion caused by the binocular camera is removed by correcting the first contour image and the second contour image; A disparity map is obtained by calculating the matching points between the corrected first contour image and the second contour image through binocular matching. The position coordinates of the target position of the remote control device facing the screen are calculated using 3D coordinates based on the parallax map. The first information, which includes the location coordinates, is sent to the controlled device so that the controlled device determines the target location based on the first information and performs a first operation on the screen based on the target location.

5. The method according to claim 4, characterized in that, The first operation includes: displaying a pointer at the target location; or highlighting or shadowing the control corresponding to the target location.

6. A remote control device, characterized in that, Applied to a controlled device, the controlled device including a screen, the device includes: A receiving module is configured to receive first information from a remote control device. The first information includes the position coordinates of a target position of the remote control device facing the screen. These position coordinates are coordinates obtained by the remote control device based on a first contour image and a second contour image of the screen. The first and second contour images are contour images of the screen captured by a binocular camera on the remote control device. The coordinates obtained by the remote control device based on the first and second contour images of the screen include: calibrating the binocular camera in an offline state, correcting the distortion caused by the binocular camera by adjusting the first and second contour images, calculating the matching points between the corrected first and second contour images using binocular matching to obtain a disparity map, and calculating the position coordinates based on the disparity map using 3D coordinates. A determining module is configured to determine the target position of the remote control device facing the screen based on the first information; An execution module is used to perform a first operation on the screen according to the target position.

7. The apparatus according to claim 6, characterized in that, The determining module is specifically used to determine the target position on the screen corresponding to the position coordinates based on the position coordinates.

8. The apparatus according to claim 6 or 7, characterized in that, The first operation includes: displaying a pointer at the target location; or highlighting or shadowing the control corresponding to the target location.

9. A remote control device, characterized in that, Applied to remote control devices, the device includes: An acquisition module is used to acquire first information, which is used to determine the target position of the remote control device facing the screen of the controlled device. The first information includes the position coordinates of the target position. The position coordinates are obtained by the remote control device calibrating its binocular camera in an offline state, capturing a first contour image and a second contour image of the screen through the binocular camera when the remote control device is pointing at the screen, correcting the distortion caused by the binocular camera by removing the distortion of the first contour image and the second contour image, calculating the matching points between the corrected first contour image and the second contour image through binocular matching to obtain a disparity map, and calculating the disparity map using 3D coordinates. The sending module is configured to send the first information to the controlled device, so that the controlled device determines the target location based on the first information and performs a first operation on the screen based on the target location.

10. The apparatus according to claim 9, characterized in that, The first operation includes: displaying a pointer at the target location; or highlighting or shadowing the control corresponding to the target location.

11. A controlled device, characterized in that, The device includes a screen, a processor, and a memory, wherein the memory stores a computer program, the computer program including program instructions, which, when executed by the processor, cause the controlled device to perform the following steps: The system receives first information from a remote control device. This first information includes the position coordinates of the target position of the remote control device facing the screen. The position coordinates are coordinates obtained by the remote control device based on a first contour image and a second contour image of the screen. The first contour image and the second contour image are contour images of the screen captured by a binocular camera on the remote control device. The coordinates obtained by the remote control device based on the first contour image and the second contour image include: calibrating the binocular camera offline, correcting the distortion caused by the binocular camera on the first contour image and the second contour image, calculating the matching points between the corrected first contour image and the second contour image using binocular matching to obtain a disparity map, and calculating the position coordinates based on the disparity map using 3D coordinates. The target position of the remote control device facing the screen is determined based on the first information; Perform a first operation on the screen according to the target position.

12. The device according to claim 11, characterized in that, When the processor executes the program instructions, the controlled device performs the following steps: Determining the target position of the remote control device facing the screen based on the first information includes: Based on the location coordinates, determine the target position on the screen corresponding to the location coordinates.

13. The device according to claim 11 or 12, characterized in that, The first operation includes: displaying a pointer at the target location; or highlighting or shadowing the control corresponding to the target location.

14. A remote control device, characterized in that, It includes a processor and a memory, wherein the memory is used to store a computer program, the computer program including program instructions, which, when the processor executes the program instructions, cause the remote control device to perform the following steps: The binocular camera on the remote control device is calibrated in an offline state; When the remote control device is pointed at the screen of the controlled device, the binocular camera captures a first outline image and a second outline image containing the screen. The distortion caused by the binocular camera is removed by correcting the first contour image and the second contour image; A disparity map is obtained by calculating the matching points between the corrected first contour image and the second contour image through binocular matching. The position coordinates of the target position of the remote control device facing the screen are calculated using 3D coordinates based on the parallax map. The first information, which includes the location coordinates, is sent to the controlled device so that the controlled device determines the target location based on the first information and performs a first operation on the screen based on the target location.

15. The device according to claim 14, characterized in that, The first operation includes: displaying a pointer at the target location; or highlighting or shadowing the control corresponding to the target location.

Images