Man-machine interaction method, electronic equipment and related device

Abstract

The invention provides a man-machine interaction method, electronic equipment and a related device, and in the method, a cursor on an interface can move along with the movement of an object in a scene in which a user and the electronic equipment carry out air interaction. Wherein under the condition that the first moving speed of the cursor is smaller than or equal to a first threshold value, and / or the distance between the cursor and any target control on the interface is smaller than or equal to a second threshold value, it is indicated that the distance between the cursor and the control, needing to be operated by a user, on the interface is short; therefore, the electronic equipment can move the cursor at the second moving speed slower than the first moving speed, the cursor is prevented from missing the control required to be operated by the user, the cursor can accurately stay at the position of the control required to be operated by the user, and the user can conveniently operate the control required to be operated by the user.

Description

[0001] This application claims priority to the Chinese Patent Application No. 202411218760.X, filed on August 30, 2024, entitled “Human-computer interaction method, electronic device and related apparatus”, the entire content of which is incorporated herein by reference. TECHNICAL FIELD

[0002] Embodiments of the present application relate to the technical field of terminal, and in particular, to a human-computer interaction method, an electronic device and related apparatus. BACKGROUND

[0003] With the rise of smart large screens, vehicle-mounted entertainment large screens, virtual reality (VR), augmented reality (AR) and other devices, user-to-device interaction such as air gesture and handheld remote control interaction has become an important way of human-computer interaction. For example, a user can use air gesture to interact with a control on an interface of an electronic device, play a motion game, do exercise, work, and so on.

[0004] Currently, when a user uses air interaction to operate a control on a screen, the user needs to move carefully, and a careless movement can miss the control, especially when operating a small control. In particular, in the case of low gesture recognition accuracy and recognition jitter, the user can even be unable to operate the control, resulting in poor user experience. SUMMARY

[0005] Embodiments of the present application provide a human-computer interaction method, an electronic device and related apparatus, which can improve the user experience of air operation of a control on an interface.

[0006] In a first aspect, embodiments of the present application provide a human-computer interaction method. An execution subject of the method can be an electronic device or a chip in the electronic device. Hereinafter, the electronic device is taken as an example for description. In the method, in response to a first operation, the electronic device can display a cursor on an interface. The cursor corresponds to an object in a real space, and in response to movement of the object in the real space, the cursor can move on the interface. That is, the cursor can move on the interface in response to movement of the object in the real space. When the cursor meets a speed adjustment condition, the electronic device can continue to move the cursor at a second moving speed, which is less than a first moving speed. The speed adjustment condition includes that the first moving speed of the cursor is less than or equal to a first threshold value, and / or the distance between the cursor and any target control on the interface is less than or equal to a second threshold value. When the cursor does not meet the speed adjustment condition, the electronic device continues to move the cursor at the first moving speed.

[0007] The first movement speed is positively correlated with the movement speed of the object, and the first movement speed is a movement speed determined based on the movement speed of the object and without adjustment. The first movement speed being a movement speed without adjustment can be understood as follows: the first movement speed is determined based on an initial ratio and the movement speed of the object, and the first movement speed determined based on the initial ratio and the movement speed of the object can be referred to as an initial movement speed, which is a movement speed without adjustment, such as the second movement speed being a movement speed with adjustment.

[0008] In the scenario in which the user performs the air interaction with the electronic device, the cursor on the interface moves with the movement of the object. When the cursor meets the speed adjustment condition (for example, when the first movement speed of the cursor is less than or equal to a first threshold value, and / or the distance between the cursor and any target control on the interface is less than or equal to a second threshold value), it is indicated that the cursor is close to the control for the user demand operation on the interface, and therefore the electronic device can move the cursor at a second movement speed slower than the first movement speed, so as to avoid missing the control for the user demand operation by the cursor, and enable the cursor to accurately stop at the position of the control for the user demand operation, thereby facilitating the user to operate the control for the user demand operation. When the cursor does not meet the speed adjustment condition, it is indicated that the cursor is far away from the control for the user demand operation on the interface, and therefore the electronic device can continue to move the cursor at a relatively large speed, such as the first movement speed, so that the cursor can quickly approach the control for the user demand operation.

[0009] In a possible implementation, the first movement speed is determined based on the movement speed of the object, specifically: the first movement speed is obtained based on the movement speed of the object and an initial ratio. For example, the initial ratio can be a ratio of the movement speed of the object to the movement speed of the cursor (the first movement speed), and in this example, the electronic device can take the quotient of the movement speed of the object and the initial ratio as the first movement speed. For example, the initial ratio can be a ratio of the movement speed of the cursor (such as the first movement speed) to the movement speed of the object, and in this example, the electronic device can take the product of the movement speed of the object and the initial ratio as the first movement speed.

[0010] In this implementation, the first movement speed is obtained based on the movement speed of the object and the initial ratio, and when the cursor does not meet the speed adjustment condition, the electronic device can continue to move the cursor at the first movement speed without adjusting the first movement speed of the cursor.

[0011] In a possible implementation, when the cursor meets the speed adjustment condition, the position of the cursor on the interface corresponds to a first control on the interface, and the second moving speed is related to the size of the first control. For example, the second moving speed is positively related to the size of the first control. For example, the first control has a first size, the second moving speed corresponding to the first control is a first speed, the first control has a second size, and the second moving speed corresponding to the first control is a second speed. The first size is greater than the second size, and the first speed is greater than the second speed, but both the first speed and the second speed are less than the first moving speed.

[0012] In a possible implementation, the position of the cursor on the interface corresponding to the first control on the interface specifically means that the position of the cursor on the interface is within a preset range of the first control. The preset range surrounds the first control, and the second moving speed corresponding to the first control is a moving speed within the preset range.

[0013] In this implementation, for a control with a smaller size, the electronic device can control the cursor to move at a slower second moving speed, avoid missing the control due to too fast movement of the cursor, and enable the cursor to accurately stop at the position of the control, so as to facilitate the user to operate the control.

[0014] In a possible implementation, when the cursor meets the speed adjustment condition, before the electronic device continues to move the cursor at the second moving speed, the electronic device can further determine a first control in a target control corresponding to the cursor according to the position of the cursor on the interface and the position of the target control on the interface. The electronic device can obtain the second moving speed corresponding to the first control according to the ratio corresponding to the first control and the moving speed of the object. The ratio corresponding to the first control is different from the initial ratio. In this implementation, the electronic device can adjust the first moving speed of the cursor to the second moving speed according to the ratio corresponding to the first control.

[0015] The ratio corresponding to the first control and the method for the electronic device to obtain the ratio corresponding to the target control are described in detail below. The target control includes the first control.

[0016] In a possible implementation, the ratio corresponding to the first control includes a first ratio and a second ratio, the first ratio is used to determine a first sub-moving speed, the second ratio is used to determine a second sub-moving speed, and the second moving speed corresponding to the first control includes the first sub-moving speed and the second sub-moving speed. The ratio corresponding to the first control is related to the distance between the cursor and a preset position in the first control.

[0017] In this implementation, the first control can correspond to multiple ratios, and the ratio is related to the distance between the cursor and the preset position in the first control. In this way, the electronic device can determine the moving speed of the cursor according to the distance between the cursor and the preset position in the first control, and the ratio corresponding to the first control. Because the first control can correspond to multiple ratios, the moving speed of the cursor can also be multiple, so that the electronic device can gradually adjust the moving speed of the cursor, and the user's operation experience can be improved.

[0018] In this example, when the distance between the position of the cursor on the interface and the preset position is a first distance, the electronic device can determine a first sub-moving speed according to the first ratio and the moving speed of the object, and control the cursor to move at the first sub-moving speed. When the distance between the position of the cursor on the interface and the preset position is a second distance, the electronic device can determine a second sub-moving speed according to the second ratio and the moving speed of the object, and move the cursor at the second sub-moving speed. Wherein, the first distance is greater than the second distance, and the first sub-moving speed is greater than the second sub-moving speed.

[0019] In this implementation, because the moving speed of the cursor changes from the first moving speed to the first sub-moving speed, and then to the second sub-moving speed, when the cursor is close to the first control, the electronic device can gradually reduce the moving speed of the cursor, which can avoid the problem that the moving speed of the cursor is too fast and misses the first control. On the other hand, the moving speed of the cursor gradually decreases, rather than suddenly decreases, and the user's operation experience is good.

[0020] The method that the electronic device obtains the ratio corresponding to the target control is introduced as follows:

[0021] In a possible implementation, the electronic device can obtain layout information of the interface, and the layout information includes the size of each control on the interface. The electronic device can obtain the ratio corresponding to the target control according to the size of the target control.

[0022] The electronic device can call a first application programming interface (API) to obtain the layout information. Alternatively, the electronic device can take a screenshot of the interface to obtain a first image. The electronic device can input the first image into a first model to obtain layout information output by the first model, and the first model is used to obtain the size and position of each control in the image.

[0023] For example, for the first type of application, the electronic device can read the layout information of the interface of the first type of application through the auxiliary service. For example, for the second type of application, the electronic device cannot read the layout information of the interface of the second type of application through the auxiliary service, and in this example, the electronic device can obtain the layout information of the interface by taking a screenshot and using the first model.

[0024] In a possible implementation, the electronic device can obtain a first image by capturing the interface, and input the first image into a third model to obtain the position of the target control on the interface and the ratio corresponding to the target control, the third model being configured to obtain the position of the target control in the image and the ratio corresponding to the target control.

[0025] The method in this way is suitable for various types of applications and has a wide range of applications.

[0026] In a possible implementation, when the cursor meets the speed adjustment condition, before the electronic device continues to move the cursor at the second moving speed, the electronic device can further determine a first control in the target control corresponding to the cursor according to the position of the cursor on the interface and the position of the target control on the interface. The electronic device can obtain a second moving speed corresponding to the first control according to an adjustment ratio corresponding to the first control and the first moving speed. For example, the electronic device can multiply the adjustment ratio corresponding to the first control and the first moving speed to obtain the second moving speed.

[0027] The adjustment ratio corresponding to the first control and the method for obtaining the adjustment ratio corresponding to the target control are described in detail below. The target control includes the first control.

[0028] Similar to the ratio corresponding to the first control, the adjustment ratio corresponding to the first control includes a first ratio and a second ratio, the first ratio being used to determine a first sub-moving speed and the second ratio being used to determine a second sub-moving speed, and the second moving speed corresponding to the first control including the first sub-moving speed and the second sub-moving speed. The adjustment ratio corresponding to the first control is related to the distance between the cursor and a preset position in the first control.

[0029] In this implementation, the adjustment ratio corresponding to the first control can be multiple, and the adjustment ratio is related to the distance between the cursor and the preset position in the first control. In this way, the electronic device can determine the moving speed of the cursor according to the distance between the cursor and the preset position in the first control and the adjustment ratio corresponding to the first control. Because the adjustment ratio corresponding to the first control can be multiple, the moving speed of the cursor can also be multiple, so that the electronic device can gradually adjust the moving speed of the cursor and improve the operation experience of the user.

[0030] When the distance between the position of the cursor on the interface and the preset position is the first distance, the electronic device can determine a first sub-moving speed of the cursor according to the first ratio and the first moving speed, and control the cursor to move at the first sub-moving speed. When the distance between the position of the cursor on the interface and the preset position is the second distance, the electronic device can determine a second sub-moving speed of the cursor according to the second ratio and the first moving speed, and control the cursor to move at the second sub-moving speed. The first distance is greater than the second distance, and the first sub-moving speed is greater than the second sub-moving speed.

[0031] In this implementation, because the moving speed of the cursor changes from the first moving speed to the first sub-moving speed and then to the second sub-moving speed, when the cursor is close to the first control, the electronic device can gradually reduce the moving speed of the cursor, which can avoid the problem that the moving speed of the cursor is too fast and misses the first control, and the moving speed of the cursor gradually decreases instead of suddenly decreasing, thus providing a good user operation experience.

[0032] The method for the electronic device to obtain the adjustment ratio corresponding to the target control is described below.

[0033] In a possible implementation, the electronic device can obtain layout information of the interface, the layout information including the size of each control on the interface. The electronic device can obtain the adjustment ratio corresponding to the target control according to the size of the target control.

[0034] The electronic device can call a first application programming interface (API) to obtain the layout information. Alternatively, the electronic device can take a screenshot of the interface to obtain a first image, and input the first image into a first model to obtain layout information output by the first model, the first model being configured to obtain the size and position of each control in an image.

[0035] In a possible implementation, the electronic device can take a screenshot of the interface to obtain a first image. The electronic device can input the first image into a second model to obtain the position of the target control on the interface and the adjustment ratio corresponding to the target control, the second model being configured to obtain the position of the target control in an image and the adjustment ratio corresponding to the target control.

[0036] The method in this implementation is applicable to various types of applications and has a wide range of applications.

[0037] In a possible implementation, the first moving speed is an unadjusted moving speed, which can also be understood as follows: The first moving speed is not a moving speed obtained by adjusting an initial moving speed based on a ratio (or adjustment coefficient) of the control. For example, a second moving speed (including the first sub-moving speed and the second sub-moving speed) is obtained by adjusting based on a ratio (or adjustment coefficient) of the first control. The second moving speed (including the first sub-moving speed and the second sub-moving speed) can be referred to as an adjusted moving speed.

[0038] In a possible implementation, in response to the interface being updated, the electronic device can acquire the position of the target control in the updated interface and the adjustment ratio corresponding to the target control, so that the electronic device can control the moving speed of the cursor according to the position of the target control in the updated interface and the adjustment ratio corresponding to the target control, to improve the operation experience of the user.

[0039] In a possible implementation, the object can be a body part of the user or an input device of the electronic device. Specifically, the body part of the user is specifically a hand of the user.

[0040] In a possible implementation, in response to the first operation, the electronic device can display the cursor on the interface. The cursor corresponds to the object in the real space, and the cursor can move on the interface in response to the movement of the object in the real space. That is, the cursor can move on the interface in response to the movement of the object in the real space. When the cursor meets the speed adjustment condition, the electronic device can continue to move the cursor at a second moving speed, and the second moving speed is less than the first moving speed. The speed adjustment condition includes that the first moving speed of the cursor is less than or equal to a first threshold, and / or the cursor is within a preset range of any target control on the interface. When the cursor does not meet the speed adjustment condition, the electronic device continues to move the cursor at the first moving speed.

[0041] The first moving speed is positively correlated with the moving speed of the object, and the first moving speed is a moving speed determined based on the moving speed of the object and without adjustment. The first moving speed being a moving speed without adjustment can be understood as that the first moving speed is determined based on an initial ratio and the moving speed of the object, wherein the first moving speed determined based on the initial ratio and the moving speed of the object can be referred to as an initial moving speed, and the initial moving speed is a moving speed without adjustment, such as the second moving speed being a moving speed with adjustment.

[0042] The preset range of the target control surrounds the target control, and the preset ranges of any two target controls on the interface do not have an intersection. The preset range of the target control surrounding the target control can be understood as that the boundary of the preset range of the target control can completely overlap the boundary of the target control, or the boundary of the preset range of the target control and the boundary of the target control do not completely overlap, and the boundary of the preset range of the target control surrounds the boundary of the target control. The preset ranges of any two target controls on the interface do not have an intersection, which can be understood as that the preset ranges of any two target controls on the interface do not completely overlap.

[0043] In this embodiment, in a scenario where a user interacts with an electronic device remotely, the cursor on the interface moves along with the object. Specifically, if the cursor's first moving speed is greater than a first threshold, and / or the cursor is not within a preset range of any target control on the interface, it indicates that the cursor is far from the control the user wants to operate on. Therefore, the electronic device can continue moving the cursor at the first moving speed so that the cursor can quickly approach the control the user wants to operate on. Conversely, if the cursor's first moving speed is less than or equal to the first threshold, and / or the cursor is within a preset range of any target control on the interface, it indicates that the cursor is close to the control the user wants to operate on. Therefore, the electronic device can move the cursor at a second moving speed, slower than the first moving speed, to prevent the cursor from missing the control the user wants to operate on, enabling the cursor to accurately stop at the position of the control the user wants to operate on, facilitating the user's operation of the control.

[0044] In this implementation, the adjustment ratio corresponding to the first control includes a first ratio and a second ratio. The first ratio is used to determine a first sub-movement speed, and the second ratio is used to determine a second sub-movement speed. The second movement speed corresponding to the first control includes both the first and second sub-movement speeds. The adjustment ratio corresponding to the first control is related to the area of ​​the cursor within a preset range of the first control, and different areas within the preset range of the first control are in an enclosing relationship. For example, when the cursor is in the first area within the preset range of the first control, the cursor moves at the first sub-movement speed; when the cursor is in the second area within the preset range of the first control, the cursor moves at the second sub-movement speed. The first area encloses the second area, and the first sub-movement speed is greater than the second sub-movement speed.

[0045] In this implementation, because the cursor's movement speed changes from a first movement speed to a first sub-movement speed, and then to a second sub-movement speed, to improve the user experience and avoid inconsistent movement speeds, the first sub-movement speed can be greater than the second sub-movement speed when the first area surrounds the second area. In other words, as the cursor approaches the first control, the electronic device can gradually reduce the cursor's movement speed. This avoids the problem of the cursor moving too fast and missing the first control, and the gradual reduction in cursor movement speed, rather than a sudden decrease, provides a better user experience.

[0046] In one possible implementation, when the control is circular, the electronic device determines whether the cursor is within a preset range of the control based on the distance between the cursor and the control. For example, the distance between the cursor and the control can be understood as the distance between the cursor and a preset position (such as the center position) of the control. For example, when the cursor is within the preset range of the control, the distance between the cursor and the control is less than or equal to a second threshold. When the cursor is not within the preset range of the control, the distance between the cursor and the control is greater than the second threshold.

[0047] In this example, when the control is circular, the speed adjustment conditions may include: a first movement speed less than or equal to a first threshold, and / or, the cursor being within a preset range of any target control on the interface; or, the speed adjustment conditions may include: a first movement speed less than or equal to a first threshold, and / or, the distance between the cursor and any target control on the interface being less than or equal to a second threshold. The speed adjustment conditions can be referred to the relevant descriptions above.

[0048] In a second aspect, embodiments of this application provide an electronic device, including a processor and a memory, wherein the memory is used to store code instructions and the processor is used to execute the code instructions to perform the methods described in the first aspect or any possible implementation thereof.

[0049] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program or instructions that, when executed on a computer, cause the computer to perform the methods described in the first aspect or any possible implementation thereof.

[0050] Fourthly, embodiments of this application provide a computer program product including a computer program, which, when run on a computer, causes the computer to perform the methods described in the first aspect or any possible implementation of the first aspect.

[0051] Fifthly, this application provides a chip or chip system including at least one processor and a communication interface. The communication interface and the at least one processor are interconnected via a circuit. The at least one processor is used to run computer programs or instructions to perform the methods described in the first aspect or any possible implementation thereof. The communication interface in the chip can be an input / output interface, pins, or circuits, etc.

[0052] In one possible implementation, the chip or chip system described above in this application further includes at least one memory storing instructions. The memory can be an internal storage unit of the chip, such as a register or cache, or it can be a storage unit of the chip itself (e.g., read-only memory, random access memory, etc.).

[0053] It should be understood that the second to fifth aspects of this application correspond to the technical solutions of the first aspect of this application, and the beneficial effects achieved by each aspect and the corresponding feasible implementation are similar, and will not be repeated here. Attached Figure Description

[0054] Figure 1 This is a schematic diagram of a human-computer interaction scenario applicable to the embodiments of this application;

[0055] Figure 2 A schematic diagram showing the user's hand image and the interface displayed on the smart screen;

[0056] Figure 3 A diagram illustrating how the cursor moves to follow the user's hand.

[0057] Figure 4 This is a schematic diagram of another human-computer interaction scenario to which the embodiments of this application are applicable;

[0058] Figure 5 This is a schematic diagram of another human-computer interaction scenario to which the embodiments of this application are applicable;

[0059] Figure 6 A schematic diagram of the structure of an electronic device provided in an embodiment of this application;

[0060] Figure 7 A software structure block diagram of an electronic device provided in an embodiment of this application;

[0061] Figure 8 A flowchart illustrating one embodiment of the human-computer interaction method provided in this application;

[0062] Figure 9 A schematic diagram illustrating the movement of the cursor following the user's hand, provided in an embodiment of this application;

[0063] Figure 10 A schematic diagram illustrating the size and second moving speed of the control provided in an embodiment of this application;

[0064] Figure 11 A flowchart illustrating another embodiment of the human-computer interaction method provided in this application;

[0065] Figure 12 A schematic diagram illustrating the ratio of the control provided in an embodiment of this application;

[0066] Figure 13 A schematic diagram of cursor movement provided in an embodiment of this application;

[0067] Figure 14 This is a flowchart illustrating another embodiment of the human-computer interaction method provided in this application. Detailed Implementation

[0068] With the rise of smart screens, in-vehicle entertainment screens, virtual reality (VR), and augmented reality (AR) devices, air-to-air interaction between users and electronic devices, such as air gesture interaction and handheld remote control interaction, has become an important human-computer interaction method. Figure 1 This is a schematic diagram of a human-computer interaction scenario applicable to the embodiments of this application. Taking the user interacting with the smart screen 10 using air gestures as an example, refer to... Figure 1 The smart screen 10 includes a camera 11' and a display screen 12'. The display screen 12' is used to display the interface of the smart screen 10. The camera 11' is used to capture the user's gestures.

[0069] Reference Figure 1 The smart screen 10 can display a main page 101, which may include a user identifier 11 and a content recommendation area 12. For example, the content recommendation area 12 may include: a new album recommendation control 121, a viewing history control 122, a fitness control 123, a sports event control 124, a weather control 125, a video control 126, etc. In this embodiment, the user can operate the corresponding control through gestures, enabling the smart screen 10 to display an interface triggered by "operating the control".

[0070] For example, when a user is positioned in front of the smart screen 10, the camera 11' can capture an image of the user's hand. For instance, the image of the user's hand captured by the camera 11' would look like... Figure 2 As shown in a, the smart screen 10 can determine the key points of the user's hand based on the image of the user's hand. For example, the key points of the user's hand can be the center of the user's palm or a certain finger of the hand. The key points of the user's hand can be preset, and this application embodiment does not limit this. Figure 2 In the example of 'a', the center of the user's palm is taken as the key point of the user's hand, and... Figure 2 In the diagram, 'a' is represented by a black dot to indicate the key point of the user's hand.

[0071] After determining the key points of the user's hand, the Smart Screen 10 can map these key points onto its display screen 12'. For example, the Smart Screen 10 can display a cursor 11A corresponding to the key points of the hand on the main page 101. In other words, cursor 11A is a virtual representation of the key points of the hand on the page. Here's a brief explanation of how the Smart Screen 10 maps the user's key points of the hand onto its display screen 12':

[0072] Figure 2The value 'b' in the image represents the main page 101 of the smart screen 10. In some embodiments, the smart screen 10 can acquire the first coordinates A(x, y) of the key points of the user's hand in the image coordinate system. The image coordinate system can be understood as the coordinate system of the image captured by the camera 11'. For example, refer to... Figure 2 In the image coordinate system, the origin O(0,0) can be the lower left corner of the image, the X-axis can be the direction from the lower left corner to the lower right corner, and the Y-axis can be the direction from the lower left corner to the upper left corner.

[0073] The smart screen 10 maps the key points of the user's hand onto the display screen 12'. For example, if the smart screen 10 displays cursor 11A on the main page 101, the smart screen 10 can obtain the second coordinate A'(x1, y1) of cursor 11A in the interface coordinate system. Here, the interface coordinate system can be understood as the coordinate system in which the interface of the smart screen 10 is located. For example, refer to... Figure 2 In the context of 'b', the interface coordinate system can have its origin O'(0, 0) at the bottom left corner of the interface, with the X-axis pointing from the bottom left corner to the bottom right corner and the Y-axis pointing from the bottom left corner to the top left corner. It is understood that this application does not impose restrictions on the setting of the image coordinate system and the interface coordinate system. Figure 2 The image coordinate system and interface coordinate system in the example are used for illustration.

[0074] In some embodiments, in practical application scenarios, the field of view of camera 11' is relatively large, and the user's hand can move in a specific area of ​​the field of view. In order to improve the user experience, smart screen 10 can also map the dashed frame area in the image to the entire area of ​​the interface, or smart screen 10 can also map the dashed frame area in the image to the dashed frame area in the interface.

[0075] For example, the coordinate system of the image can be based on the bottom left corner of the dashed box in the image as the origin O(0,0), with the direction from the bottom left corner to the bottom right corner as the X-axis and the direction from the bottom left corner to the top left corner as the Y-axis. Similarly, the coordinate system of the interface can be based on the bottom left corner of the dashed box in the interface as the origin O(0,0), with the direction from the bottom left corner to the bottom right corner as the X-axis and the direction from the bottom left corner to the top left corner as the Y-axis. In this example, the mapping relationship between the pixels of the image and the pixels of the interface is as follows: the mapping relationship between the pixels in the dashed box area of ​​the image and the pixels in the dashed box area of ​​the interface.

[0076] After acquiring the first coordinate A(x, y) and the second coordinate A'(x1, y1), the smart screen 10 can store the corresponding mapping relationship between the first coordinate A(x, y) and the second coordinate A'(x1, y1). In other words, the smart screen 10 stores the mapping relationship between the first coordinate A(x, y) and the second coordinate A'(x1, y1). This storage of the mapping relationship between the first coordinate A(x, y) and the second coordinate A'(x1, y1) can be seen as the smart screen 10 mapping the key points of the user's hand to the cursor 11A. Following the movement of the user's hand in real space, the cursor 11A moves on page 101. The principle of how the cursor 11A follows the movement of the user's hand is briefly described below:

[0077] The image captured by camera 11' (such as an image of the user's hand) is composed of multiple pixels, which are the basic units that make up the image. Pixels determine the image's clarity and detail. Page 101 of the smart screen 10 is also composed of multiple pixels, which are the smallest display units on the display screen 12'. The number of pixels in the smart screen 10 is related to the resolution and clarity of the display screen 12'. The number of pixels in the image and the number of pixels in page 101 may be the same or different. In some embodiments, the smart screen 101 can obtain the mapping relationship between pixels in the image and pixels in page 101, which can be simply referred to as the pixel-to-pixel mapping relationship. For example, one pixel in the image is equivalent to N pixels in page 101, or one pixel in page 101 is equivalent to M pixels in the image. After obtaining the mapping relationship between pixels in the image and pixels in page 101, the smart screen 101 can obtain the ratio of the user's hand movement speed to the cursor 11A movement speed.

[0078] For example, taking the case where one pixel in an image corresponds to N pixels on page 101, if the key point of the hand in the image moves one pixel, the smart screen 10 needs to move the cursor 11A a distance of N pixels on page 101. In some embodiments, given the same movement time, the movement speed is proportional to the movement distance. Thus, the smart screen 10 can determine that the ratio of the user's hand movement speed to the cursor 11A movement speed can be considered as 1:N. It should be noted that 1 in the ratio can be considered as one pixel in the image, and N can be considered as N pixels on page 101. It is conceivable that the smart screen 10 can also determine the ratio of the cursor 11A movement speed to the user's hand movement speed, such as N:1. In some embodiments, the ratio of the user's hand movement speed to the cursor 11A movement speed, or the ratio of the cursor 11A movement speed to the user's hand movement speed, can be referred to as the first ratio relationship, which is determined by the smart screen 10 based on the mapping relationship between pixels.

[0079] For example, following the movement of the user's hand, the smart screen 10 detects that the key points of the user's hand have moved 3 pixels in the image. Based on the mapping relationship between pixels, the smart screen 10 can move the cursor 11A by 3N pixels on page 101. Similarly, following the movement of the user's hand, the smart screen 10 detects that the key points of the user's hand have moved 10 pixels in the image. Based on the mapping relationship between pixels, the smart screen 10 can move the cursor 11A by 10N pixels on page 101. Furthermore, it should be noted that, for example, if the key points of the user's hand move from position A to position B in the image, the smart screen 10 can not only determine the distance the key points of the hand have moved in the image (e.g., 3 pixels), but also determine the direction of movement based on the two-dimensional coordinates of positions A and B. Accordingly, the smart screen 10 can control the cursor 11A to move a distance of 3N pixels on page 101 according to the distance and direction of movement of the key points of the hand in the image.

[0080] In some embodiments, the speed at which the user's hand moves in real space can be referred to as the first speed, and the speed at which the key points of the user's hand move in the image can be referred to as the second speed. The first speed and the second speed can be proportional. In some embodiments, the speed at which the cursor 11A moves on page 101 can be referred to as the third speed, and the second speed and the third speed are proportional, for example, the second speed and the third speed can have a first ratio relationship. In summary, the first speed and the third speed are also proportional, that is, based on the movement of the user's hand in real space, the movement of the cursor 11A on page 101 is proportional to the movement speed of the cursor 11A. In some embodiments, the ratio of the speed at which the user's hand moves in real space to the speed at which the cursor 11A moves on page 101 can be referred to as the second ratio relationship.

[0081] It should be understood that the speed of the user's hand movement in the following embodiments refers to the speed at which the user's hand moves in the image captured by the camera 11'.

[0082] Having explained the principle of how cursor 11A moves following the user's hand movement in the example above, the following describes how the user can manipulate the controls on page 101 using gestures:

[0083] Reference Figure 3 In the diagram, when the user's hand moves from position A to position B in the image, the smart screen 10 determines the cursor's movement distance and direction based on the hand's movement distance (AB) and direction. For example, following the hand's movement from position A to position B in the image, the smart screen 10 moves the cursor 11A's position on the page from position A' to position B'. The hand's movement speed and the cursor's movement speed are in a first ratio relationship.

[0084] Reference Figure 3In the diagram, position B' represents the location of video control 126. When cursor 11A moves to the location of video control 126, in response to user input, the smart screen 10 can display a video interface, such as... Figure 3 As shown in 'c', this describes how a user triggers the Smart Screen 10 to display the video interface:

[0085] For example, when the cursor 11A moves to the position of the video control 126, the user can use gestures to operate the video control 126, such as making a fist or waving a finger, etc., and this application embodiment does not limit this. Alternatively, for example, when the cursor 11A moves to the position of the video control 126, the user can say the voice command "click" to instruct the smart screen 10 to display the interface (video interface) triggered by "clicking the video control 126". This application embodiment does not limit the user's operation of triggering the control, and this application embodiment focuses on the process of the cursor 11A moving according to the user's hand movement.

[0086] Figure 4 This is a schematic diagram illustrating another human-computer interaction scenario applicable to the embodiments of this application. Taking the user interacting with the smart screen 10 using a remote control as an example, refer to... Figure 4 The smart screen 10 includes a camera 11' and a display screen 12'. The display screen 12' is used to display the interface of the smart screen 10. The camera 11' is used to capture images of the user holding the remote control 41.

[0087] When the user holds the remote control 41, the smart screen 10 can display cursor 11A on the display screen 12'. Figure 4 Taking display screen 12' as an example, which shows page 101. Figure 2 and Figure 3 Similarly, in the example above, as the remote control 41 moves, the cursor 11A moves on page 101. The principle behind the movement of cursor 11A on the page can be found in [reference needed]. Figure 2 and Figure 3 The description in [the document] is as follows. In this example, when the cursor 11A moves to the location of the control, the user can also interact with the control, as can be seen in [the document]. Figure 3 As described in the document. In some embodiments, when the cursor 11A moves to the location of the control, the user can also press a button on the remote control 41 to operate the control on the page 101.

[0088] Figure 5 This is a schematic diagram illustrating another human-computer interaction scenario applicable to the embodiments of this application. Taking the user interacting with the smart screen 10 using sensors as an example, refer to... Figure 5The smart screen 10 includes a camera 11' and a display screen 12'. The display screen 12' is used to display the interface of the smart screen 10. The camera 11' is used to capture images of the user using the sensor 51. The sensor 51 can be, for example, a fitness ring, a game controller, etc.

[0089] For example, when a user is playing a game using a game controller 51, the smart screen 10 can display a cursor 11A on the display screen 12'. For instance, Figure 5 The display screen 12' shows the game page 501, and the cursor 11A corresponding to the game controller 51 can be displayed on page 501. Figure 2 and Figure 3 Similarly, in the example above, as the game controller 51 moves, the cursor 11A moves on the game 501. The principle behind the movement of cursor 11A on the page can be found in [reference needed]. Figure 2 and Figure 3 The description in [the document] is as follows. In this example, when the cursor 11A moves to the location of the control, the user can also interact with the control, as can be seen in [the document]. Figure 3 As described in the description. In some embodiments, when the cursor 11A moves to the location of the control, the user can also press the button deployed on the fitness ring 51 to operate the control on the page 101.

[0090] In summary, users can interact with electronic devices through air gestures, remote controls, and sensors (such as fitness rings and game controllers). In some embodiments, remote controls and sensors (such as fitness rings and game controllers) can be referred to as input devices of the electronic device. In this application embodiment, the electronic device can be an electronic device including a camera and a display screen. Exemplarily, the electronic device can be referred to as user equipment (UE), terminal, etc. For example, the electronic device can be a smart screen, smart TV, virtual reality (VR) terminal device, augmented reality (AR) terminal device, in-vehicle device, mobile phone, tablet, personal digital assistant (PDA), handheld device with wireless communication function, computing device, or wearable device, wireless terminal in industrial control, wireless terminal in smart home, etc. The form of the electronic device is not specifically limited in this application embodiment.

[0091] Currently, when users interact with controls on the screen using air gestures, they need to move carefully, and it's easy to miss the control, especially when operating smaller controls. Particularly when gesture recognition accuracy is low or there is shaky recognition, users may even be unable to operate the control, resulting in a poor user experience.

[0092] For example, taking user interaction with a smart screen using air gestures as an example, assuming one pixel in an image is equivalent to two pixels on the page, if the user's hand moves one pixel in the image, the smart screen can move the cursor two pixels on the page. For instance, if a control on the page is large, such as occupying 20 pixels, when the smart screen moves the cursor to the control, even if the user's hand moves another 8 pixels, the cursor moves 16 pixels to remain on the control. Therefore, when the user manipulates the control with gestures, even with extra movement, they can still operate the control without needing to carefully move their hand. Conversely, if a control on the page is small, such as occupying 2 pixels, if the user's hand moves one pixel in the image, the cursor moves two pixels on the page, which would cross the control. This would require the user to carefully move 0.5 pixels to position the cursor on the control, making operation inconvenient.

[0093] Furthermore, with low recognition accuracy and shaky recognition, small controls become more difficult for users to operate. For example, current electronic devices can use algorithms to identify key points of a hand in an image. Visual algorithms need to consider various lighting conditions and environmental influences, and they currently suffer from shaky recognition issues. For instance, when the user's hand is not moving, the cursor's position on the page should theoretically remain unchanged. However, due to shaky recognition and other problems, the cursor's position on the page will jitter, making it more difficult for users to operate smaller controls.

[0094] Accordingly, this application provides a human-computer interaction method. When it is determined that a user needs to operate a control, the electronic device can reduce the cursor's movement speed, thereby reducing the cursor's sensitivity so that the cursor can land on the control. For example, referring to the description in the above embodiments, one pixel in an image corresponds to N pixels on a page. If the hand moves one pixel in the image, the cursor needs to move N pixels on the page. Alternatively, the ratio of the hand's movement speed to the cursor's movement speed is 1:N. In this application embodiment, when a user needs to operate the video control 126, the movement of the hand in the image by one pixel can be adjusted to move the cursor N' pixels on the page, where N' is less than N, i.e., the ratio of the hand's movement speed to the cursor's movement speed is 1:N'. In other words, the electronic device adjusts the ratio of the hand's movement speed to the cursor's movement speed, thus reducing the cursor's movement speed. This gives the user the impression that the cursor sensitivity is lower; even if the user moves their hand quickly, the cursor moves very slowly, making it easier for the user to operate the control.

[0095] In this example, for instance, if the control occupies 2 pixels, and the electronic device does not adjust the ratio of hand movement speed to cursor movement speed (i.e., the cursor movement speed is not adjusted), if the user's hand moves 1 pixel in the image, the cursor moves 2 pixels on the page. Thus, even if the user carefully moves 1 pixel, the cursor will miss the control. However, in this embodiment, when the user needs to operate the cursor, the electronic device can adjust the ratio of hand movement speed to cursor movement speed. For example, the electronic device can adjust the ratio to 1:0.5, meaning the cursor movement speed is reduced. In this way, if the user's hand moves 1 pixel in the image, the cursor moves 0.5 pixels on the page. The slower cursor movement speed prevents the user from missing the control, making it easier for the user to operate it.

[0096] Before introducing the human-computer interaction method provided in the embodiments of this application, the electronic device provided in the embodiments of this application will be introduced first. (Refer to...) Figure 6 The electronic device provided in this application embodiment may include: a processor 61, a camera 62, and a display screen 63.

[0097] Camera 62 is used to capture images of an object. The object can be, for example, a user's body part or an input device of an electronic device. For example, a user's body part can be their hand, eyes, head, knee, or foot. An input device of an electronic device can be understood as a device that can establish a connection with the electronic device, through which the user can input commands to the electronic device. For example, an input device of an electronic device can include, but is not limited to, a remote control, keyboard, mouse, and sensors (such as a fitness ring or game controller). In some embodiments, the input device of an electronic device can be connected to the electronic device via wired or wireless connection; this application embodiment does not limit the connection method between the input device and the electronic device.

[0098] The display screen 63 is used to display the interface of the electronic device, and the interface may include at least one operable control.

[0099] Processor 61 is used to map objects onto display screen 62 based on images captured by camera 62, and to display the cursor corresponding to the object on display screen 62. As the object moves in real space, processor 61 can control the movement of the cursor on the interface, as can be seen from... Figure 2-5 Description of the example in the text.

[0100] In this embodiment, the processor 61 is further configured to reduce the cursor movement speed, i.e., reduce the cursor's response sensitivity, and slow down the cursor movement speed to facilitate user operation of the controls, as described in the following embodiments. It should be understood that the steps performed by the electronic device in the following embodiments can be considered as steps performed by the processor 61.

[0101] In some embodiments, the electronic device in this application may further include a memory 64. The memory 64 includes high-speed random-access memory (RAM) and may also include non-volatile memory (NVM), such as at least one disk storage device. The memory 64 may store various instructions for performing various processing functions and implementing the method steps of this application.

[0102] In some embodiments, the electronic device in this application may further include: a power supply 65, a communication bus 66, and a communication port 67. The communication port 67 is used to enable communication between the electronic device and other peripherals (such as input devices of the electronic device). In this application embodiment, the memory 64 is used to store computer-executable program code, which includes instructions; when the processor 61 executes the instructions, the instructions cause the processor 61 of the electronic device to perform the actions described in the above method embodiments. The implementation principle and technical effects are similar and will not be repeated here.

[0103] Understandable Figure 6 The structures shown do not constitute a specific limitation on the electronic device. In other embodiments of this application, the electronic device may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0104] In some embodiments, the software system of an electronic device may adopt a layered architecture, event-driven architecture, microkernel architecture, microservice architecture, or cloud architecture. Figure 7 This is a software architecture block diagram of an electronic device provided in an embodiment of this application. The layered architecture divides the software system of the electronic device into several layers, each with a clear role and division of labor. Layers communicate with each other through software interfaces. In some embodiments, the electronic device may include an application layer and a system layer.

[0105] The application layer can include a series of application packages. (See reference...) Figure 7 Application packages can include: audio applications, fitness applications, sports event applications, weather applications, and video applications, etc.

[0106] For example, taking a smart screen as an electronic device, an audio application, fitness application, sports event application, weather application, and video application can be installed on the smart screen. Figure 1 As shown, application controls can be displayed on page 101 of the smart screen. In this embodiment, users can operate the application controls on the smart screen through air interaction, enabling the smart screen to display the application interface.

[0107] Reference Figure 7 The system layer can include: interaction modules and accessibility services (AS).

[0108] The interaction module is used to process data when users interact with electronic devices remotely and to respond to user actions.

[0109] In some embodiments, the interaction module may include, for example, a gesture interaction module, a remote control interaction module, and a sensor interaction module. The gesture interaction module processes data when a user interacts with the electronic device using air gestures; the remote control interaction module processes data when a user interacts with the electronic device while holding a remote control; and the sensor interaction module processes data when a user interacts with the electronic device while holding a remote control. This application does not limit the method of air interaction between the user and the electronic device, and correspondingly, it does not limit the modules deployed in the interaction module. The following embodiments use the interaction module as an example for explanation.

[0110] The interaction module maps objects to a cursor on the interface based on images captured by the electronic device's camera. Following the movement of the object in real space, the interaction module can control the cursor's movement on the interface, as described in the following embodiments.

[0111] In some embodiments, the interaction module may be deployed in an electronic device in the form of an application or service, and this application embodiment does not limit this.

[0112] The AS (Application System) can read the layout information of the application's interface. The layout information may include the position and size of controls within the interface. In some embodiments, the interface may include operable controls and non-operable controls. In this application embodiment, the controls operated by the user can be all controls on the interface, or only operable controls. In response to user operation of a control, the electronic device can respond, such as displaying the interface triggered by the operation of the control, or displaying a pop-up, menu, or prompt message. This application embodiment does not limit the response method of the electronic device after the user operates an operable control. In some embodiments, operable controls include, for example, buttons, input boxes, sliders, drop-down menus, etc., on the interface. Non-operable controls include, for example, labels, read-only text boxes, static images, and icons, etc., on the interface.

[0113] In some embodiments, an electronic device may deploy a first type of application. The application server (AS) can read the layout information of the interface of the first type of application. In this example, the interaction module can call a first interface to obtain the layout information of the interface of the first type of application from the AS. The first interface is used to interface with the AS, or in other words, the AS has a first interface deployed on it.

[0114] In some embodiments, a second type of application may be deployed in the electronic device. The AS cannot read the layout information of the interface of the second type of application. Accordingly, the interaction module cannot obtain the layout information of the interface of the second type of application from the AS through the first interface.

[0115] For the second type of application, in some embodiments, a first model can be pre-deployed in the electronic device. In this example, because the interaction module cannot obtain the layout information of the interface of the second type of application from the AS through the first interface, the interaction module can take a screenshot of the interface of the second type of application and obtain the layout information of the interface through the first model.

[0116] In some embodiments, a second model may be pre-deployed in the electronic device. The interaction module can take screenshots of the interface of the second type of application and obtain the ratio corresponding to each control in the interface through the second model, as described in the following embodiments.

[0117] The human-computer interaction method provided in this application will be described below with reference to specific embodiments. These embodiments can be combined with each other, and the same or similar concepts or processes may not be described again in some embodiments.

[0118] Figure 8 This is a flowchart illustrating one embodiment of the human-computer interaction method provided in this application. (Refer to...) Figure 8 The human-computer interaction method provided in this application embodiment may include:

[0119] S801, in response to the first operation, displays a cursor on the interface. The cursor corresponds to an object in the real space. In response to the object moving in the real space, the cursor moves on the interface.

[0120] The object can be a user's body part or an input device of an electronic device. For example, a user's body part can be their hands, eyes, head, knees, or feet. Input devices of electronic devices can include, but are not limited to, remote controls, keyboards, mice, and sensors (such as ring devices or game controllers).

[0121] The first operation is to trigger the electronic device to map an object to a cursor on the interface. In response to the first operation, the electronic device can display the cursor corresponding to the object on the interface. In some embodiments, there can be at least one object, and correspondingly, there can also be at least one cursor for each object. In this embodiment, a single cursor is used as an example to illustrate the process of the cursor moving with the object. When there are multiple objects and their corresponding cursors, the method of the cursor moving with the object in this embodiment can be referred to.

[0122] For example, taking a user's hand as the object, the first operation can be: the user performs a preset gesture. Preset gestures include, for example, waving, or having the palm face the electronic device for a preset duration, such as 2 seconds. It should be understood that having the palm face the electronic device for a preset duration means that the palm is within the field of view of the electronic device's camera for the preset duration. For example, taking a remote control as the object, the first operation can be: the user holding the remote control facing the electronic device for a preset duration, or the user pressing a preset button on the remote control, etc. This application does not limit the first operation; depending on the electronic device or its configuration, the first operation by which the user triggers the display of a cursor on the interface of the electronic device can be different.

[0123] For example, refer to Figure 9 In step 'a', if the user faces the smart screen and their palm is facing the screen for a preset duration, the smart screen can display cursor 91 on interface 901. Interface 901 can be referred to in the description on page 101.

[0124] It is understood that the cursor is represented by an arrow in the accompanying drawings of this application embodiment. The cursor may also be a dot, a hand icon, etc., and this application embodiment does not limit this. In some embodiments, users can customize the shape, size, color, and other information of the cursor on the electronic device.

[0125] S802, when the cursor does not meet the speed adjustment conditions, the cursor continues to move at a first moving speed. The speed adjustment conditions include: the first moving speed of the cursor is less than or equal to a first threshold, and / or, the distance between the cursor and any target control on the interface is less than or equal to a second threshold, the first moving speed is positively correlated with the moving speed of the object, and the first moving speed is a moving speed determined based on the moving speed of the object and not adjusted.

[0126] The cursor moves across the interface at a first movement speed, which is positively correlated with the movement speed of the object. For example, as the object's movement speed increases, the first movement speed also increases.

[0127] In some embodiments, the first moving speed is positively correlated with the object's moving speed, specifically by being obtained based on a ratio between the object's moving speed and an initial value. In this embodiment, the electronic device can determine the cursor's moving speed based on the object's moving speed and the initial ratio, and the electronic device can control the cursor to move on the interface at that speed. The object's moving speed can be the object's moving speed within an image captured by a camera.

[0128] The first movement speed, which is the unadjusted movement speed, can be understood as follows: the first movement speed is determined based on the initial ratio and the object's movement speed. This first movement speed, determined based on the initial ratio and the object's movement speed, can be called the initial movement speed, which can be understood as the unadjusted movement speed. For example, the second movement speed is the adjusted movement speed.

[0129] The initial ratio can be understood as the first ratio relationship in the above embodiments. Specifically, the initial ratio can be the ratio of the object's moving speed in the image to the cursor's moving speed. In some embodiments, the ratio of the object's moving speed in the image to the cursor's moving speed can be understood as the value obtained by dividing the object's moving speed in the image by the cursor's moving speed, or the value obtained by dividing the cursor's moving speed by the object's moving speed in the image.

[0130] Reference Figure 2 As described in the illustrated embodiments, the initial ratio can be related to the mapping relationship between the pixels of the image and the pixels of the interface. For example, the initial ratio can be represented by the ratio of the object's movement speed in the image to the cursor's movement speed; for instance, the initial ratio could be 1:N.

[0131] In this embodiment, the cursor moves along with the object. The electronic device can determine the first moving speed of the cursor based on the object's moving speed and an initial ratio. When the cursor does not meet the speed adjustment conditions, the electronic device can control the cursor to continue moving on the interface at the first moving speed. The speed adjustment conditions include: the cursor's first moving speed is less than or equal to a first threshold, and / or, the distance between the cursor and any target control on the interface is less than or equal to a second threshold. The cursor not meeting the speed adjustment conditions can include the following situations: the cursor's first moving speed is greater than the first threshold, and / or, the distance between the cursor and any target control on the interface is greater than the second threshold. The following describes a scenario where the cursor moves with the object when the speed adjustment conditions are not met:

[0132] Firstly:

[0133] In some embodiments, the electronic device may determine whether the cursor is close to the control that the user needs to operate based on the cursor's first movement speed.

[0134] Typically, a user's operating habits are as follows: when the cursor is not close to the control the user wants to operate on, the user can quickly move the object to enable the cursor to move quickly so that the cursor can quickly approach the control the user wants to operate on. When the cursor is close to the control the user wants to operate on, the user can slowly move the object to enable the cursor to move slowly so that the cursor can accurately stop at the position of the control the user wants to operate on, avoiding the cursor missing the control the user wants to operate on. Therefore, in this embodiment of the application, the electronic device, based on the user's operating habits, can determine that: when the cursor's first moving speed is greater than a first threshold, it indicates that the cursor's moving speed is fast, indicating that the cursor is far from the control the user wants to operate on. When the cursor's first moving speed is less than or equal to the first threshold, it indicates that the cursor's moving speed is slow, indicating that the cursor is close to the control the user wants to operate on.

[0135] In this embodiment of the application, when the first moving speed of the cursor is greater than the first threshold, because the cursor is far away from the control that the user needs to operate, the electronic device can continue to move the cursor at a faster speed, i.e., the first moving speed, so that the cursor can quickly approach the control that the user needs to operate.

[0136] Secondly:

[0137] In some embodiments, the electronic device can determine whether the cursor is close to the control that the user needs to operate based on the cursor's position on the interface.

[0138] In this embodiment, when the cursor moves on the interface, the electronic device can determine the cursor's position on the interface. In some embodiments, the electronic device can obtain the layout information of the interface, which may include the position and size of each control on the interface. The electronic device can determine the distance between the cursor and any target control on the interface based on the cursor's position on the interface and the position of each target control on the interface. In this embodiment, the electronic device can determine whether the cursor is close to the control that the user needs to operate based on the distance between the cursor and any target control on the interface. In some embodiments, the target control may be each control on the interface, or each operable control on the interface, or a control on the interface with a size larger than a preset size, or a control on the interface with a user operation frequency greater than a preset frequency, etc.

[0139] Specifically, if the distance between the cursor and any target control on the interface is greater than the second threshold, it indicates that the cursor is far from the control on the interface that the user needs to operate. The electronic device can continue to move the cursor at the first moving speed so that the cursor can quickly approach the control that the user needs to operate.

[0140] Thirdly:

[0141] As described in "Firstly" and "Secondly" above, the electronic device determines whether the cursor is close to the control the user wants to operate on based on the cursor's initial movement speed or the distance between the cursor and the target control on the interface. In some embodiments, some users may have the habit of moving objects slowly or quickly regardless of whether the cursor is close to the control they want to operate on, thus the cursor's movement speed will be correspondingly faster or slower. In some embodiments, although the distance between the cursor and the target control on the interface is close, the control the user wants to operate on is not that target control. To address these situations, in order to more accurately determine whether the cursor is close to the control the user wants to operate on, in some embodiments, the electronic device may combine the cursor's initial movement speed and the distance between the cursor and the target control on the interface to determine whether the cursor is close to the control the user wants to operate on.

[0142] In some embodiments, when the cursor's first moving speed is greater than a first threshold and the distance between the cursor and any target control on the interface is greater than a second threshold, that is, when the user moves the object at a relatively fast speed, the cursor moves quickly on the interface, and the distance between the cursor and any target control on the interface is relatively far, it indicates that the cursor has not yet approached the control that the user needs to operate. The electronic device can continue to move the cursor at the first moving speed so that the cursor can quickly approach the control that the user needs to operate.

[0143] S803: When the cursor meets the speed adjustment conditions, continue to move the cursor at a second moving speed, which is less than the first moving speed.

[0144] The cursor speed adjustment condition can be understood as follows: the cursor's first movement speed is less than or equal to a first threshold, and / or the distance between the cursor and any target control on the interface is less than or equal to a second threshold.

[0145] Referring to the description in S802, the following description can be consulted:

[0146] Firstly:

[0147] In this embodiment of the application, when the first moving speed of the cursor is less than or equal to the first threshold, it indicates that the cursor is close to the control that the user needs to operate on. In order to prevent the cursor from missing the control that the user needs to operate on, the electronic device can continue to move the cursor at a slower speed than the first moving speed, such as a second moving speed, so that the cursor can accurately stop at the position of the control that needs to be operated on.

[0148] The second moving speed is less than the first moving speed. In some embodiments, the electronic device may have a preset algorithm to reduce the first moving speed to the second moving speed, and the electronic device may determine the second moving speed based on the algorithm and the first moving speed. For example, the second moving speed may be half of the first moving speed, or the electronic device may calculate the second moving speed based on a preset formula, and the second moving speed may be less than the first moving speed.

[0149] In this embodiment, when the cursor is close to the control that the user needs to operate, the electronic device can reduce the cursor's first moving speed, enabling the cursor to move at a smaller second moving speed, i.e., reducing the cursor's sensitivity. This prevents the cursor from missing the control the user needs to operate, allowing the cursor to accurately stop at the position of the control. For small controls on the interface, the method provided in this embodiment can prevent the cursor from missing the control, enabling the cursor to accurately stop at the position of the control, so that the user can operate the control.

[0150] In some embodiments, when the first movement speed of the cursor is less than or equal to a first threshold, the position of the cursor on the interface corresponds to the position of the first control on the interface. This correspondence between the cursor position on the interface and the first control on the interface can also be simply referred to as "the cursor corresponds to the first control," which can be understood as the cursor being within a preset range of the first control.

[0151] In some embodiments, the preset range of the first control can be equal to the size of the first control, and the boundary of the preset range of the first control is the boundary of the first control. In this example, the cursor being within the preset range of the first control can be understood as: the cursor being inside the first control, or the cursor being on the first control.

[0152] In some embodiments, the preset range of the first control is larger than the size of the first control, and the preset range of the first control surrounds the first control. The cursor being within the preset range of the first control can be understood as the distance between the cursor and the first control being less than or equal to a second threshold.

[0153] In some embodiments, the second movement speed may be related to the size of the first control. For example, the second movement speed may be positively correlated with the size of the first control; for instance, the smaller the size of the first control, the smaller the second movement speed. In this embodiment, the electronic device can determine the second movement speed based on the size of the first control. Specifically, the smaller the size of the first control, the smaller the second movement speed. Thus, for smaller controls, the electronic device can control the cursor movement with a slower second movement speed, preventing the cursor from moving too quickly and missing the control, enabling the cursor to accurately stop at the control's position, facilitating user operation of the control.

[0154] For example, taking a rectangle as the first control, refer to... Figure 10 In the context of 'a', when the first control is of the first size, the second movement speed is V1. (Refer to...) Figure 10 In the case of b, when the first control is of the second size, the second size is smaller than the first size, and the second movement speed can be V2, where V2 is less than V1.

[0155] Secondly:

[0156] In this embodiment, when the distance between the cursor and any target control on the interface is less than or equal to a second threshold, assuming that any target control is a first control on the interface, the electronic device can determine that the cursor is close to the first control. The electronic device can then move the cursor at a slower speed than the first moving speed, such as a second moving speed, so that the cursor can accurately stop at the position of the first control. The second moving speed can be referred to the description in "one" of S803.

[0157] Thirdly:

[0158] In this embodiment of the application, when the first moving speed of the cursor is less than or equal to a first threshold, and the distance between the cursor and any target control (such as the first control) on the interface is less than or equal to a second threshold, the electronic device can continue to move the cursor at a slower speed than the first moving speed, such as a second moving speed, so that the cursor can accurately stop at the position of the first control.

[0159] Taking a ratio of 1:N between the hand's movement speed and the cursor's movement speed as an example, see... Figure 9In the context of point 'a', when cursor 91 is at position A on interface 901, if the user needs to operate user identifier 11, and cursor 91 is far from the user identifier, the user can quickly move their hand, and correspondingly, cursor 91 also moves quickly on interface 901. Taking the movement speed of cursor 91 on interface 901 as the first movement speed as an example, the electronic device can detect that the first movement speed of cursor 91 is greater than a first threshold, and the distance of cursor 91 from any target control on interface 901 is greater than a second threshold. The electronic device can continue to move cursor 91 at the first movement speed so that cursor 91 can quickly reach user identifier 11. For example... Figure 9 The target control on the interface can be understood as any control on the interface.

[0160] Reference Figure 9 In point b, when cursor 91 moves to position B, it is relatively close to user identifier 11. To avoid missing user identifier 11, the user can move their hand slowly. Correspondingly, the movement speed of cursor 91 also slows down. Taking the first movement speed of cursor 91 as an example, if the electronic device detects that the first movement speed of cursor 91 is less than or equal to a first threshold, and the distance between cursor 91 and user identifier 11 is less than or equal to a second threshold, the electronic device can continue moving cursor 91 using the second movement speed.

[0161] For example, as cursor 91 moves from position A to position B, for every pixel the user's hand moves in the image, cursor 91 moves N pixels on interface 91. When cursor 91 reaches position B, for every pixel the user's hand moves in the image, cursor 91 moves N' pixels on interface 91, where N' is less than N. Thus, with the same hand movement speed, cursor 91 moves slower; that is, in the same movement time, cursor 91 moves a shorter distance. In other words, cursor 91's sensitivity is reduced, preventing it from missing user identifier 11, allowing the user to operate user identifier 11 smoothly.

[0162] As mentioned in the above embodiment, the second moving speed is positively correlated with the size of the first control; the smaller the size of the first control, the smaller the second moving speed. Figure 9Taking user identifier 11 as an example, and video control 126 as an example of the first control, when the electronic device detects that the first moving speed of cursor 91 is less than or equal to a first threshold, and the distance between cursor 91 and video control 126 is less than or equal to a second threshold, the electronic device can use a second moving speed to continue moving cursor 91. For example, for every pixel the user's hand moves in the image, cursor 91 moves N” pixels on interface 91, where N” is less than N. The size of video control 126 is larger than the size of user identifier 11, and the second moving speed corresponding to video control 126 can be greater than the second moving speed corresponding to user identifier 11. Correspondingly, for example, N” is greater than N'.

[0163] It should be understood that Figure 9 The thickness of the solid arrowhead indicates the movement speed of cursor 91. The thicker the solid arrowhead, the faster the cursor moves; the thinner the solid arrowhead, the slower the cursor moves.

[0164] In this embodiment, in a scenario where a user interacts with an electronic device remotely, the cursor on the interface moves along with the object. Specifically, if the cursor's first moving speed is greater than a first threshold, and / or the distance between the cursor and any target control on the interface is greater than a second threshold, it indicates that the cursor is far from the control the user needs to operate on. Therefore, the electronic device can continue moving the cursor at the first moving speed so that the cursor can quickly approach the control the user needs to operate on. Conversely, if the cursor's first moving speed is less than or equal to the first threshold, and / or the distance between the cursor and any target control on the interface is less than or equal to the second threshold, it indicates that the cursor is close to the control the user needs to operate on. Therefore, the electronic device can move the cursor at a second moving speed, slower than the first moving speed, to prevent the cursor from missing the control the user needs to operate on, enabling the cursor to accurately stop at the position of the control the user needs to operate on, facilitating user operation of the control.

[0165] The following details the process of "continuing to move the cursor at a second speed when the cursor's first moving speed is less than or equal to a first threshold, and / or the distance between the cursor and any target control on the interface is less than or equal to a second threshold":

[0166] In some embodiments, the electronic device may store the position of a target control on the interface and the ratio corresponding to the target control. The ratio can be understood as the ratio of the object's movement speed to the cursor's movement speed. It should be understood that the ratio corresponding to the control is different from the initial ratio. Although both the initial ratio and the ratio corresponding to the control are used to obtain the cursor's movement speed based on the object's speed, the initial ratio is used when the cursor is not close to the control the user wants to operate on, while the ratio corresponding to the control is used when the cursor is close to the control the user wants to operate on.

[0167] In other words, when the electronic device determines that the cursor is not close to the control that the user wants to operate on, the electronic device can determine the cursor's movement speed based on the initial ratio and the object's movement speed, and control the cursor to continue moving based on that movement speed. When the electronic device determines that the cursor is close to the control that the user wants to operate on (such as the first control), the electronic device can determine the cursor's movement speed based on the ratio corresponding to the first control and the object's movement speed, and control the cursor to continue moving based on that movement speed.

[0168] In some embodiments, if the cursor's first movement speed is less than or equal to a first threshold, and / or the distance between the cursor and any target control on the interface is less than or equal to a second threshold, the electronic device can determine that the cursor is approaching a target control. The target control is a control whose distance from the cursor is less than or equal to the second threshold.

[0169] The ratios corresponding to different controls on the interface can be the same or different. In this embodiment, the ratio corresponding to a control is related to the size of the control.

[0170] In this embodiment, when the cursor's first moving speed is less than or equal to a first threshold, and / or the distance between the cursor and any target control on the interface is less than or equal to a second threshold, the electronic device can determine the first control among the target controls corresponding to the cursor based on the cursor's position on the interface and the position of each control on the interface. The electronic device can determine the ratio corresponding to the first control from the stored ratios of target controls. Furthermore, the electronic device can obtain the second moving speed corresponding to the first control based on the ratio corresponding to the first control and the object's moving speed. For example, taking the ratio corresponding to the first control as the quotient of the object's moving speed and the cursor's moving speed, the electronic device can divide the object's moving speed by the ratio corresponding to the first control to obtain the second moving speed corresponding to the first control. In this embodiment, after obtaining the second moving speed corresponding to the first control, the electronic device can continue to move the cursor at the second moving speed.

[0171] In some embodiments, the electronic device may store the position of a target control on the interface and the corresponding adjustment ratio of the target control, the adjustment ratio of the target control being used to adjust the first movement speed. In this example, when the electronic device determines that the cursor is not close to the control that the user needs to operate on, the electronic device may determine the movement speed of the cursor based on the initial ratio and the movement speed of the object, and control the cursor to continue moving based on the movement speed. When the electronic device determines that the cursor is close to the control that the user needs to operate on (such as the first control), the electronic device may determine the movement speed of the cursor based on the adjustment ratio of the first control and the first movement speed, and control the cursor to continue moving based on the movement speed.

[0172] The adjustment ratios for different controls on the interface can be the same or different. In this embodiment, the adjustment ratio of a control is related to the size of the control.

[0173] In this embodiment, when the cursor's first moving speed is less than or equal to a first threshold, and / or the distance between the cursor and any target control on the interface is less than or equal to a second threshold, the electronic device can determine the first control among the target controls corresponding to the cursor based on the cursor's position on the interface and the position of each control on the interface. The electronic device can determine the adjustment ratio corresponding to the first control from the stored adjustment ratios of the target controls. Furthermore, the electronic device can obtain the second moving speed corresponding to the first control based on the adjustment ratio corresponding to the first control and the first moving speed. The second moving speed is equal to the first moving speed multiplied by the adjustment ratio corresponding to the first control. In this embodiment, after obtaining the second moving speed corresponding to the first control, the electronic device can continue to move the cursor at the second moving speed.

[0174] It is understood that, in some embodiments, the electronic device can obtain the position of the target control on the target interface and the adjustment ratio corresponding to the target control, and store the position of the target control on the target interface and the adjustment ratio corresponding to the target control. In actual use, the cursor movement speed is adjusted based on the adjustment ratio corresponding to the target control.

[0175] In some embodiments, after obtaining the position of the target control on the target interface and the corresponding adjustment ratio of the target control, the electronic device can obtain the ratio corresponding to the target control based on the initial ratio and the corresponding adjustment ratio, and store the ratio corresponding to the target control. In this example, during actual use, the electronic device can use the ratio corresponding to the target control and the object's movement speed to obtain the cursor's movement speed.

[0176] For example, refer to Figure 9 The description, taking user identifier 11 and video control 126 on the interface as examples, shows that the ratio corresponding to user identifier 11 can be 1:N', and the ratio corresponding to video control 126 can be 1:N". The ratio corresponding to user identifier 11 can be obtained based on the adjustment ratio and initial ratio of user identifier 11, and the ratio corresponding to video control 126 can be obtained based on the adjustment ratio and initial ratio of video control 126. Specifically, when the first control is user identifier 11, the electronic device can determine the second movement speed of the cursor based on the ratio of user identifier 11 (which can be 1:N') and the speed of the object. Similarly, when the first control is video control 126, the electronic device can determine the second movement speed of the cursor based on the ratio of video control 126 (which can be 1:N") and the speed of the object.

[0177] Reference Figure 9 During the movement of cursor 91 from position A to position B, cursor 91 moves at a first speed. Upon reaching position B, cursor 91 begins moving at a second speed. Because the second speed is less than the first speed, the cursor's movement speed suddenly decreases. In some embodiments, to reduce the jarring feeling during cursor operation, a control can correspond to multiple ratios, and the cursor's movement speed gradually decreases based on these ratios. This way, as the cursor approaches a control, its movement speed can be sequentially reduced according to the multiple ratios corresponding to that control, rather than suddenly becoming very small, thus improving the user experience.

[0178] The following example uses the adjustment ratio corresponding to the first control. For instance, the adjustment ratio corresponding to the first control may include a first ratio and a second ratio. It should be understood that this example uses two ratios for the adjustment ratio corresponding to the first control:

[0179] The adjustment ratio of the first control is related to the distance between the cursor and a preset position within the first control. For example, the preset position within the first control might be its center. For instance, when the distance between the cursor and the center of the first control is a first distance, the adjustment ratio of the first control is a first ratio; when the distance between the cursor and the center of the first control is a second distance, the adjustment ratio of the first control is a second ratio. The first distance can be greater than the second distance.

[0180] The electronic device can determine a first sub-moving speed based on a first ratio and a first moving speed. In other words, the first ratio is used to determine the first sub-moving speed. Similarly, the electronic device can determine a second sub-moving speed based on a second ratio, the first moving speed, and the moving speed of the object. In other words, the second ratio is used to determine the second sub-moving speed. In some embodiments, the second moving speed corresponding to the first control may include both the first sub-moving speed and the second sub-moving speed.

[0181] Because the adjustment ratio corresponding to the first control is related to the distance between the cursor and the preset position in the first control, in this embodiment, when the cursor distance from the preset position in the first control is different, the electronic device can use the corresponding adjustment ratio to determine the cursor's movement speed. For example, when the distance between the cursor's position on the interface and the preset position is a first distance, the electronic device can determine the cursor's movement speed as a first sub-movement speed based on the first ratio and the first movement speed, and the electronic device can continue to move the cursor at the first sub-movement speed. The first sub-movement speed is equal to the product of the first ratio and the first movement speed. When the distance between the cursor's position on the interface and the preset position is a second distance, the electronic device can determine the cursor's movement speed as a second sub-movement speed based on the second ratio and the first movement speed, and the electronic device can continue to move the cursor at the second sub-movement speed. The second sub-movement speed is equal to the product of the second ratio and the first movement speed.

[0182] In this design, because the cursor's movement speed changes from a first movement speed to a first sub-movement speed, and then to a second sub-movement speed, to improve the user experience and avoid inconsistent movement speeds, the first sub-movement speed can be greater than the second sub-movement speed when the first distance is greater than the second distance. In other words, as the cursor approaches the primary control, the electronic device can gradually reduce the cursor's movement speed. This avoids the cursor moving too quickly and missing the primary control, and the gradual, rather than abrupt, reduction in cursor speed provides a better user experience.

[0183] It should be understood that, in the above embodiments, the adjustment ratio corresponding to the first control includes two ratios as an example. In some embodiments, the adjustment ratio corresponding to the first control may include more or fewer ratios.

[0184] The above embodiments describe a method for pre-storing the position of a target control on an interface and its corresponding adjustment ratio in an electronic device. In some embodiments, the electronic device can obtain the position of the target control on the interface and its corresponding adjustment ratio. The following describes a method for an electronic device to obtain the position of a target control on the interface and its corresponding adjustment ratio. It should be understood that the method for an electronic device to obtain the position of a target control on the interface and its corresponding ratio can also refer to the description in the following embodiments:

[0185] Method 1:

[0186] Reference Figure 11 Methods for electronic devices to obtain the position of a target control on the interface and the corresponding adjustment ratio of the target control may include:

[0187] S1101, The electronic device obtains the layout information of the interface, which includes the size of each control on the interface.

[0188] The layout information of the interface can include the position and size of each control on the interface. After obtaining the layout information, the electronic device can adjust the target controls accordingly based on their size.

[0189] In this context, the electronic device can identify the target control within the controls of the interface. For example, the target control can be each control on the interface, or each operable control on the interface, or a control on the interface with a size larger than a preset size, or a control on the interface with a user operation frequency greater than a preset frequency, etc.

[0190] The following section first introduces methods for electronic devices to obtain interface layout information:

[0191] Firstly, in some embodiments, reference is made to... Figure 7 As described in the text, for the first type of application, AS can read the layout information of the interface of the first type of application, and the interaction module can call the first interface to obtain the layout information of the interface of the first type of application from AS.

[0192] Secondly, in some embodiments, for the second type of application, the AS cannot read the layout information of the interface of the second type of application. Correspondingly, the interaction module cannot obtain the layout information of the interface of the second type of application from the AS through the first interface. In this embodiment, a first model can be pre-deployed in the electronic device. The first model is used to output the layout information of the interface based on the image of the interface. The training process of the first model is not described in detail in this embodiment; in some embodiments, the training data of the first model can be the image of the interface.

[0193] In this embodiment, the electronic device can take a screenshot of the interface to obtain a first image. The electronic device can then input the first image into a first model to obtain the layout information of the interface output by the first model.

[0194] In summary, in some embodiments, the electronic device may include a first type of application and / or a second type of application. When the electronic device obtains the layout information of the interface, it can select the corresponding method to obtain the layout information of the interface according to the type of application to which the interface belongs. For example, when the interface is the interface of a first type of application, the interaction module can call the first interface to obtain the layout information of the interface of the first type of application from the AS. When the interface is the interface of a second type of application, the interaction module can take a screenshot of the interface to obtain a first image. The interaction module can input the first image into the first model to obtain the layout information of the interface output by the first model.

[0195] S1102, the electronic device obtains the adjustment ratio corresponding to the target control based on the size of the target control on the interface.

[0196] The target controls on the interface vary in size. In this embodiment, the electronic device can determine the adjustment ratio of the target control based on the distance from a preset position within the target control and the size of the target control. It should be understood that there can be multiple target controls on the interface. The following explanation uses a single target control as an example, with the preset position within the target control serving as its center position:

[0197] For example, taking page 101 of the smart screen as an example, Figure 12 The 'a' in the text can be referenced. Figure 1 As described in the present application, when an electronic device obtains the adjustment ratio corresponding to a target control, it can obtain the adjustment ratio within a preset range of the target control. In some embodiments, the preset range of the target control is greater than the size of the target control (e.g., the preset range of the target control surrounds the target control), or the preset range of the target control is equal to the size of the target control (e.g., the boundary of the preset range of the target control is the boundary of the target control), or the preset range of the target control is smaller than the size of the target control (e.g., the target control surrounds the preset range of the target control). The preset range of the target control can be predefined. Figure 12 In example b, the target control's preset range is larger than the target control's size. Figure 12 In b, the default range of the target control is represented by a dashed box.

[0198] This section uses user identifier 11 and video control 126 on page 101 as examples to illustrate how an electronic device can obtain the adjustment ratio corresponding to a target control:

[0199] Reference Figure 12 The enlarged view of user identifier 11 in b is shown. User identifier 11 is circular, and the center position of user identifier 11 is a preset position. Figure 12 In Figure b, the center position of the user identifier 11 is represented by a black dot. In some embodiments, the electronic device can start from the dashed frame of a preset range of the user identifier 11 and move towards the center position of the user identifier 11, setting n concentric circles with the same center as the circle of the user identifier 11. The radius of each concentric circle is different, such as... Figure 12 The dotted circle in b is shown. Each concentric circle has a different radius, so the distance of each concentric circle from the center of the user identifier 11 (i.e., the center) is different.

[0200] In some embodiments, the electronic device may have n+1 preset adjustment ratios, which are related to the distance from the center of the user identifier 11. For example, the farther the distance from the center of the user identifier 11, the larger the adjustment ratio. In some embodiments, starting from the center of the user identifier 11, the preset range of the user identifier 11 may be divided into n+1 parts from the inside out, and the boundary of each part may correspond to an adjustment ratio.

[0201] For example, taking n as 3, the preset ratio can be 0.9, 0.7, 0.5, and 0.3. Among these, the dashed frame of the preset range of user identifier 11 is furthest from the center of user identifier 11, and the dashed frame of the preset range of user identifier 11 can be adjusted to a ratio of 0.9 accordingly. Figure 12 In the diagram, the distances of the annulus of circle 1 and circle 2 from the center of user identifier 11 decrease sequentially. Therefore, it can be determined that the adjustment ratio corresponding to the circle (boundary) of user identifier 11 is 0.7, the adjustment ratio corresponding to circle 1 is 0.5, and the adjustment ratio corresponding to circle 2 is 0.3.

[0202] In other words, the adjustment ratio corresponding to the first control can include multiple ratios, and the adjustment ratio corresponding to the first control is related to the area of ​​the cursor within the preset range of the first control, with different areas within the preset range of the first control forming an enclosing relationship. For example, refer to... Figure 12 In option b, the area between the dashed border of the user identifier 11 and the circle of the user identifier 11 can be considered as region 1 within the preset range of the user identifier 11, with an adjustment ratio of 0.9. The area between the circle of the user identifier 11 and the annulus of circle 1 can be considered as region 2 within the preset range of the user identifier 11, with an adjustment ratio of 0.7. The area between the annulus of circle 1 and the annulus of circle 2 can be considered as region 3 within the preset range of the user identifier 11, with an adjustment ratio of 0.5. The area between the annulus of circle 2 and the center of the user identifier 11 can be considered as region 4 within the preset range of the user identifier 11, with an adjustment ratio of 0.3.

[0203] In this example, the adjustment ratio corresponding to user identifier 11 can include multiple ratios, such as 0.9, 0.7, 0.5, and 0.3. Furthermore, the adjustment ratio corresponding to user identifier 11 is related to the area within the preset range of user identifier 11. For example, the adjustment ratios corresponding to areas 1 to 4 within the preset range of user identifier 11 are all different. Additionally, different areas within the preset range of user identifier 11 are in an enclosing relationship; for example, area 1 encloses area 2, area 2 encloses area 3, and area 3 encloses area 4.

[0204] In this example, different areas within the preset range of user identifier 11 correspond to different adjustment ratios. These different adjustment ratios are used to determine different movement speeds. When the cursor's movement speed is the first movement speed, when the cursor moves into area 1, the electronic device can determine the cursor's movement speed in area 1 as (first movement speed × 0.9) based on the first movement speed and the adjustment ratio 0.9. Within area 1, the electronic device controls the cursor movement at a speed of (first movement speed × 0.9). When the cursor moves into area 2, the electronic device can determine the cursor's movement speed in area 2 as (first movement speed × 0.7) based on the first movement speed and the adjustment ratio 0.7. Within area 2, the electronic device controls the cursor movement at a speed of (first movement speed × 0.7). Similarly, when the cursor moves into area 3, the electronic device can determine the cursor's movement speed in area 3 as (first movement speed × 0.5) based on the first movement speed and the adjustment ratio 0.5. Within area 3, the electronic device controls the cursor movement at a speed of (first movement speed × 0.5). Similarly, when the cursor moves to area 4, the electronic device can determine the cursor's movement speed in area 4 as (first movement speed × 0.3) based on the first movement speed and the adjustment ratio of 0.3. In area 4, the electronic device controls the cursor movement at a speed of (first movement speed × 0.3).

[0205] In some embodiments, when the first control is circular, the adjustment ratio corresponding to the first control can also be said to be related to the distance between the cursor and a preset position in the first control. For example, the preset position in the first control may be, for example, the center position of the first control. Taking the first control as user identifier 11, and the adjustment ratio corresponding to user identifier 11 including a first ratio and a second ratio as an example. For example, when the distance between the cursor and the center position of the first control is a first distance (e.g., the cursor is within area 1), the adjustment ratio corresponding to the first control is the first ratio; when the distance between the cursor and the center position of the first control is a second distance (e.g., the cursor is within area 3), the adjustment ratio corresponding to the first control is the second ratio. The first distance can be greater than the second distance.

[0206] The electronic device can determine a first sub-moving speed (e.g., first moving speed × 0.9) based on a first ratio and a first moving speed. Alternatively, the first ratio can be used to determine the first sub-moving speed. Similarly, the electronic device can determine a second sub-moving speed (e.g., first moving speed × 0.5) based on a second ratio, the first moving speed, and the object's moving speed. Alternatively, the second ratio can be used to determine the second sub-moving speed. In some embodiments, the second moving speed corresponding to the first control may include both the first sub-moving speed and the second sub-moving speed.

[0207] In some embodiments, taking an initial ratio of 1:N as an example, the ratio corresponding to the dashed frame of the preset range of user identifier 11 can be (1:N)×0.9, the ratio corresponding to the circle of user identifier 11 can be (1:N)×0.7, the ratio corresponding to the annulus of circle 1 can be (1:N)×0.5, and the ratio corresponding to the annulus of circle 2 can be (1:N)×0.3. Thus, in the region between the dashed frame of the preset range of user identifier 11 and the circle of user identifier 11, the ratio corresponding to user identifier 11 can be (1:N)×0.9; in the region between the circle of user identifier 11 and the annulus of circle 1, the ratio corresponding to user identifier 11 can be (1:N)×0.7; in the region between the annulus of circle 1 and the annulus of circle 2, the ratio corresponding to user identifier 11 can be (1:N)×0.5; and in the region between the annulus of circle 2 and the center position of user identifier 11, the ratio corresponding to user identifier 11 can be (1:N)×0.3.

[0208] Similarly, refer to Figure 12 The image shows an enlarged view of video control 126 in section b. Video control 126 is rectangular, and its center position is a preset position. In some embodiments, the electronic device can start from the dashed frame of the preset range of video control 126 and set two rectangles with the same center position as the rectangle of video control 126, moving towards the center position of video control 126. Each rectangle has a different size, such as... Figure 12 The dotted-dash rectangle in b is shown. (Refer to...) Figure 12 In the diagram, b represents rectangle 1 and rectangle 2, respectively.

[0209] The adjustment ratio corresponding to the dashed frame of the preset range of the video control 126 can be 0.9, the adjustment ratio corresponding to the rectangle of the video control 126 can be 0.7, the adjustment ratio corresponding to the edge of rectangle 1 can be 0.5, and the adjustment ratio corresponding to the edge of rectangle 2 can be 0.3.

[0210] In other words, the adjustment ratio corresponding to the first control can include multiple ratios, and the adjustment ratio corresponding to the first control is related to the area of ​​the cursor within the preset range of the first control, with different areas within the preset range of the first control forming an enclosing relationship. For example, refer to... Figure 12In section b, the area between the dashed border of the preset range of video control 126 and the rectangle of video control 126 can be considered as region 1A within the preset range of video control 126, with an adjustment ratio of 0.9. The area between the rectangle of video control 126 and the edge of rectangle 1 can be considered as region 2A within the preset range of video control 126, with an adjustment ratio of 0.7. The area between the edge of rectangle 1 and the edge of rectangle 2 can be considered as region 3A within the preset range of video control 126, with an adjustment ratio of 0.5. The area between the edge of rectangle 2 and the center of video control 126 can be considered as region 4A within the preset range of video control 126, with an adjustment ratio of 0.3.

[0211] In this example, the adjustment ratio corresponding to the video control 126 can include multiple ratios, such as 0.9, 0.7, 0.5, and 0.3. Furthermore, the adjustment ratio corresponding to the video control 126 is related to the area within the preset range of the video control 126 where the cursor is located. For example, the adjustment ratios corresponding to areas 1A to 4A within the preset range of the video control 126 are all different. Additionally, different areas within the preset range of the video control 126 are in an enclosing relationship; for example, area 1A encloses area 2A, area 2A encloses area 3A, and area 3A encloses area 4A.

[0212] In this example, different areas within the preset range of the video control 126 correspond to different adjustment ratios. These different adjustment ratios are used to determine different movement speeds. When the cursor's movement speed is the first movement speed, when the cursor moves into area 1A, the electronic device can determine the cursor's movement speed within area 1A as (first movement speed × 0.9) based on the first movement speed and the adjustment ratio 0.9. Within area 1A, the electronic device controls the cursor movement at a speed of (first movement speed × 0.9). When the cursor moves into area 2A, the electronic device can determine the cursor's movement speed within area 2A as (first movement speed × 0.7) based on the first movement speed and the adjustment ratio 0.7. Within area 2A, the electronic device controls the cursor movement at a speed of (first movement speed × 0.7). Similarly, when the cursor moves into area 3A, the electronic device can determine the cursor's movement speed within area 3A as (first movement speed × 0.5) based on the first movement speed and the adjustment ratio 0.5. Within area 3A, the electronic device controls the cursor movement at a speed of (first movement speed × 0.5). Similarly, when the cursor moves to area 4A, the electronic device can determine the cursor's movement speed in area 4A as (first movement speed × 0.3) based on the first movement speed and the adjustment ratio of 0.3. In area 4A, the electronic device controls the cursor movement at a speed of (first movement speed × 0.3).

[0213] In some embodiments, different controls on the interface may correspond to different numbers of adjustment ratios, or, provided that the number of adjustment ratios corresponding to different controls is the same, the specific adjustment ratio values ​​corresponding to different controls may be different.

[0214] Thus, in the area between the dashed frame of the preset range of the video control 126 and the rectangle of the video control 126, the ratio corresponding to the video control 126 can be (1:N)×0.9; in the area between the rectangle of the video control 126 and the edge of rectangle 1, the ratio corresponding to the user identifier 11 can be (1:N)×0.7; in the area between the edge of rectangle 1 and the edge of rectangle 2, the ratio corresponding to the video control 126 can be (1:N)×0.5; and in the area between the edge of rectangle 2 and the center position of the video control 126, the ratio corresponding to the video control 126 can be (1:N)×0.3.

[0215] Using the same method, the electronic device can obtain the adjustment ratio (or proportion) corresponding to each target control on the interface, which is related to the distance from the center of the control. For example, the adjustment ratio corresponding to each control on page 101 can be as follows: Figure 12 As shown in c, Figure 12 In section 'c', the adjustment ratio of the target control on the interface is represented by an adjustment scale chart. This means that for a control on page 101, Figure 12 In the 'c' section, different shades of gray represent the adjustment ratios of the controls. (See reference...) Figure 12 In the example of the cursor being a hand icon, the control of the cursor's movement speed by the electronic device when the cursor moves in the direction of the arrow can be referenced. Figure 12 As shown in d. (Refer to...) Figure 12 In the example of d, as the distance from the center of the first control decreases, the distance the cursor moves within the same time period decreases, and correspondingly, the control ratio of the electronic device over the cursor's movement speed (such as the degree of decrease in movement speed) increases. Figure 12 In the diagram, the horizontal axis of 'd' represents the distance the cursor moves, and the vertical axis represents the control ratio of the electronic device over the cursor's movement speed. The larger the control ratio of the electronic device over the cursor's movement speed, the more the cursor's movement speed is reduced.

[0216] For example, the adjustment ratio of the target control on page 101 can also be as follows: Figure 12 As shown in e, Figure 12 The 'd' in the diagram represents the adjustment scale of the target control on the interface in a 3D view. (See reference...) Figure 12In the figure, 'e' represents the adjustment ratio, which varies depending on the size of the control. For any given control, the distance from its center affects the adjustment ratio. The closer the control is to its center, the smaller the adjustment ratio, and the greater the change in the ratio compared to its initial value. In some embodiments, the control's ratio can be viewed as the control ratio of the electronic device to the cursor's movement speed. Figure 12 In the 'e', ​​it is represented as "the control ratio of the cursor movement speed".

[0217] Method 2:

[0218] In some embodiments, a second model may be pre-deployed in the electronic device. The second model is used to output the position of a target control on the interface, and the corresponding adjustment ratio of the target control, based on the image of the interface. In this embodiment, the electronic device can take a screenshot of the interface to obtain a first image. The electronic device can then input the first image into the second model to obtain the position of the target control on the interface and the corresponding adjustment ratio of the target control.

[0219] Understandably, since the electronic device in this example is based on the second model, obtaining the position of the target control on the interface and the corresponding adjustment ratio of the target control, this method can be applied to both the first type of application and the second type of application.

[0220] In summary, after obtaining the position of the target control on the interface and its corresponding adjustment ratio, the electronic device can adjust the cursor's movement speed based on the cursor's movement speed and / or its distance from the first control. (See reference...) Figure 13 In clause 'a', when the cursor's movement speed is X cm / s, greater than a first threshold, and the cursor's distance from the first control is greater than a second threshold, the electronic device can continue moving the cursor at a speed of X cm / s. When the cursor's movement speed is Y cm / s, less than or equal to the first threshold, and the cursor's distance from the first control is less than or equal to the second threshold, refer to... Figure 13 In the context of option b, if the cursor is within the preset range of the first control, the electronic device can determine the adjustment ratio of the first control as ratio a based on the distance L1 between the cursor and the center position of the first control. After obtaining the adjustment ratio of the first control, the electronic device can obtain the second moving speed Y×a cm / s of the cursor and continue to move the cursor at Y×a cm / s. (Refer to...) Figure 13In the context of the first control, as the cursor moves, the distance between the cursor and the center of the first control decreases. The electronic device can determine the adjustment ratio of the first control as a ratio b based on the distance L2 between the cursor and the center of the first control. After obtaining the adjustment ratio b of the first control, the electronic device can obtain the second movement speed Y×b cm / s of the cursor and continue to move the cursor at Y×b cm / s. Here, b is less than a. The cursor movement speeds X cm / s and Y cm / s can both be considered as the first movement speed.

[0221] In this embodiment, when the cursor is within a preset range of the first control, the object's movement speed can be increased or decreased, and the first movement speed obtained based on the initial ratio and the object's movement speed can be increased or decreased. In this embodiment, when the cursor is within the preset range of the first control, the electronic device can continue to use the first movement speed (calculated from the initial ratio and the object's movement speed) and a speed adjustment threshold for judgment to perform corresponding operations. For example, when the cursor is within the preset range of the first control and moves at a second movement speed, if the user suddenly increases the object's movement speed, the first movement speed obtained by the electronic device based on the initial ratio and the object's movement speed will increase. Where the first movement speed is greater than the first threshold, and / or the distance between the cursor and any target control on the interface is greater than the second threshold, the electronic device can continue to move the cursor at the first movement speed.

[0222] In summary, in the embodiments of this application, the electronic device can obtain the position of the target control on the interface and the adjustment ratio corresponding to the target control. Thus, during the movement of the cursor, the electronic device can adjust the movement speed of the cursor based on the movement speed of the cursor, and / or the position of the cursor from the first control and the adjustment ratio corresponding to the control. The technical effects of the embodiments of this application can be referred to the description in the above embodiments.

[0223] In some embodiments, the first movement speed, which is an unadjusted movement speed, can also be understood as follows: the first movement speed is not the movement speed after adjusting the initial movement speed based on the ratio (or adjustment coefficient) of the control, but a movement speed determined based on the initial ratio and the movement speed of the object. For example, the second movement speed (including the first sub-movement speed and the second sub-movement speed) is obtained by adjusting the ratio (or adjustment coefficient) of the first control, and this second movement speed can be referred to as the adjusted movement speed.

[0224] In some embodiments, during interface updates, the electronic device can obtain the position of the scaling control and the corresponding scaling ratio in the updated interface, facilitating the control of the cursor's movement speed on the updated interface. Interface updates can be understood as interface changes. Interface changes include interface transitions or user swiping, and this embodiment does not limit the scope of these changes.

[0225] In some embodiments, Figure 14 This is a flowchart illustrating another embodiment of the human-computer interaction method provided in this application. (Refer to...) Figure 14 The human-computer interaction method provided in this application embodiment may include:

[0226] S1401, in response to the first operation, displays a cursor on the interface, the cursor corresponds to an object in the real space, and in response to the object moving in the real space, the cursor moves on the interface.

[0227] S1401 can be referred to in the description in S801.

[0228] S1402, when the cursor does not meet the speed adjustment conditions, the cursor continues to move at the first moving speed. The speed adjustment conditions include: the first moving speed is less than or equal to the first threshold, and / or, the cursor is within the preset range of any target control on the interface, the first moving speed is positively correlated with the moving speed of the object, the first moving speed is a moving speed determined based on the moving speed of the object and not adjusted, the preset range of the target control surrounds the target control, and the preset ranges of any two target controls do not intersect.

[0229] S1403: When the cursor meets the speed adjustment condition, continue to move the cursor at the second moving speed, which is less than the first moving speed.

[0230] In S1402, "the first moving speed is positively correlated with the moving speed of the object, and the first moving speed is a moving speed determined based on the moving speed of the object and not adjusted", which can be referred to in the description in S802.

[0231] In some embodiments, the speed adjustment conditions include: a first movement speed less than or equal to a first threshold, and / or, the cursor being within a preset range of any target control on the interface. The speed adjustment condition "the first movement speed less than or equal to the first threshold" can be referred to the description in S802.

[0232] Referring to the description in S802, in addition to determining whether the cursor is close to the control that the user needs to operate on based on the cursor's movement speed and / or the distance between the cursor and any target control on the interface, in some embodiments, the electronic device may also determine whether the cursor is close to the control that the user needs to operate on based on whether the cursor is within a preset range of any target control on the interface. The target controls on the interface can be referred to in the description in S802.

[0233] In this example, the preset range of the target control can surround the target control. For instance, the boundary of the preset range of the target control can completely overlap with the boundary of the target control, meaning the preset range of the target control can be the same size as the target control. Alternatively, the boundary of the preset range of the target control can not overlap with the boundary of the target control, and the boundary of the preset range of the target control surrounds the boundary of the target control, meaning the preset range of the target control is larger than the target control. Furthermore, the preset ranges of any two target controls on the interface do not intersect. This setting is intended to facilitate control of the cursor's movement speed when the cursor is within the preset range of any target control on the interface. The fact that the preset ranges of any two target controls on the interface do not intersect can also be understood as: the preset ranges of any two target controls on the interface do not overlap at all.

[0234] In some embodiments, when all controls on the interface are target controls, the preset range of a target control can surround the target control, and the preset ranges of any two target controls on the interface do not intersect. In some embodiments, when the controls on the interface include non-target controls, i.e., there are both target controls and non-target controls on the interface, in this example, the preset range of the target control and the non-target controls on the interface do not intersect, and the preset ranges of any two target controls do not intersect.

[0235] In some embodiments, referring to the description in Method 1 of the above embodiments, the electronic device can obtain the layout information of the interface. The layout information of the interface may include the position and size of each control on the interface. Correspondingly, the layout information of the interface may include the size of the target control's position. The position of the target control can be represented by the number of pixels occupied by the target control on the interface. When the electronic device determines that the cursor is located within the number of pixels occupied by the target control on the interface, the electronic device can determine that the cursor is on the target control.

[0236] Similarly, after obtaining the size of the target control's position, the electronic device can obtain the number of pixels occupied by the target control's preset range on the interface. It should be understood that the preset range of the target control is pre-defined, such as 1.1 times the size of the target control. In this example, when the electronic device determines that the cursor's position on the interface is within the pixels occupied by the target control's preset range, the electronic device can determine that the cursor is within the target control's preset range.

[0237] Specifically, when the cursor is not within the preset range of any target control on the interface, it indicates that the cursor is far from the control on the interface that the user needs to operate on. The electronic device can continue to move the cursor at a first moving speed so that the cursor can quickly approach the control that the user needs to operate on. Conversely, when the cursor is within the preset range of any target control on the interface, it indicates that the cursor is close to the target control. The electronic device can reduce the cursor's first moving speed, enabling the cursor to move at a smaller second moving speed, that is, reduce the cursor's sensitivity. This can prevent the cursor from missing the target control and enable the cursor to accurately stop at the target control.

[0238] Similarly, electronic devices can also determine whether the cursor is close to the control that the user needs to operate based on the cursor's movement speed and whether the cursor is within the preset range of any target control on the interface. The combination of these two conditions can make the electronic device's judgment more accurate and facilitate precise control of the cursor's speed.

[0239] In some embodiments, when the speed adjustment conditions include a first moving speed less than or equal to a first threshold and the cursor being within a preset range of any target control on the interface, the electronic device can determine that the cursor is approaching a target control on the interface when the first moving speed of the cursor is less than or equal to the first threshold and the cursor is within the preset range of any target control on the interface. The electronic device can then reduce the first moving speed of the cursor to prevent the cursor from missing the target control and enable the cursor to accurately stop at the target control. When the cursor does not meet the speed adjustment conditions, for example, when the first moving speed of the cursor is greater than the first threshold and / or the cursor is not within the preset range of any target control on the interface, the electronic device can determine that the cursor is far from the control on the interface that the user needs to operate on. The electronic device can then continue to move the cursor at the first moving speed so that the cursor can quickly approach the control that the user needs to operate on.

[0240] In some embodiments, the speed adjustment conditions in S1402 are applicable to controls of various shapes, such as rectangles, squares, circles, etc.

[0241] In some embodiments, when the control is circular, the electronic device determines whether the cursor is within a preset range of the control based on the distance between the cursor and the control. For example, the distance between the cursor and the control can be understood as the distance between the cursor and a preset position (such as the center position) of the control. For example, when the cursor is within the preset range of the control, the distance between the cursor and the control is less than or equal to a second threshold. When the cursor is not within the preset range of the control, the distance between the cursor and the control is greater than the second threshold.

[0242] In this example, when the control is circular, the speed adjustment conditions may include: a first movement speed less than or equal to a first threshold, and / or, the cursor being within a preset range of any target control on the interface; or, the speed adjustment conditions may include: a first movement speed less than or equal to a first threshold, and / or, the distance between the cursor and any target control on the interface being less than or equal to a second threshold.

[0243] In this embodiment, in a scenario where a user interacts with an electronic device remotely, the cursor on the interface moves along with the object. Specifically, if the cursor's first moving speed is greater than a first threshold, and / or the cursor is not within a preset range of any target control on the interface, it indicates that the cursor is far from the control the user wants to operate on. Therefore, the electronic device can continue moving the cursor at the first moving speed so that the cursor can quickly approach the control the user wants to operate on. Conversely, if the cursor's first moving speed is less than or equal to the first threshold, and / or the cursor is within a preset range of any target control on the interface, it indicates that the cursor is close to the control the user wants to operate on. Therefore, the electronic device can move the cursor at a second moving speed, slower than the first moving speed, to prevent the cursor from missing the control the user wants to operate on, enabling the cursor to accurately stop at the position of the control the user wants to operate on, facilitating the user's operation of the control.

[0244] It should be noted that the information and data of the interface involved in this application (including but not limited to data used for analysis, data stored, data displayed, etc.) are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of the relevant data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0245] This application provides an electronic device, which includes a processor and a memory; the memory stores computer-executable instructions; the processor executes the computer-executable instructions stored in the memory, causing the electronic device to perform the above-described method.

[0246] This application provides a chip. The chip includes a processor, which is used to call a computer program in memory to execute the technical solutions in the above embodiments. Its implementation principle and technical effects are similar to those in the related embodiments described above, and will not be repeated here.

[0247] This application also provides a computer-readable storage medium. The computer-readable storage medium stores a computer program. When the computer program is executed by a processor, it implements the methods described above. The methods described in the above embodiments can be implemented wholly or partially by software, hardware, firmware, or any combination thereof. If implemented in software, the functionality can be stored as one or more instructions or code on or transmitted over the computer-readable medium. The computer-readable medium can include computer storage media and communication media, and can also include any medium that can transfer a computer program from one place to another. The storage medium can be any target medium accessible by a computer.

[0248] In one possible implementation, a computer-readable medium may include random access memory (RAM), read-only memory (ROM), compact discread-only memory (CD-ROM) or other optical disc storage, magnetic disk storage or other magnetic storage devices, or any other medium targeted to carry or to store required program code in the form of instructions or data structures, and accessible by a computer. Furthermore, any connection is appropriately referred to as a computer-readable medium. For example, if software is transmitted from a website, server, or other remote source using coaxial cable, fiber optic cable, twisted pair, digital subscriber line (DSL), or wireless technologies such as infrared, radio, and microwave, then coaxial cable, fiber optic cable, twisted pair, DSL, or wireless technologies such as infrared, radio, and microwave are included in the definition of medium. As used herein, disks and optical discs include optical discs, laser discs, optical discs, digital versatile discs (DVDs), floppy disks, and Blu-ray discs, where disks typically reproduce data magnetically, while optical discs optically reproduce data using lasers. Combinations of the above should also be included within the scope of computer-readable media.

[0249] This application provides a computer program product, which includes a computer program that, when run, causes a computer to perform the above-described method.

[0250] It should be noted that the modules or components described in the above embodiments can be one or more integrated circuits configured to implement the above methods, such as one or more application-specific integrated circuits (ASICs), one or more digital signal processors (DSPs), or one or more field-programmable gate arrays (FPGAs), etc. Furthermore, when a module is implemented through processing element scheduler code, the processing element can be a general-purpose processor, such as a central processing unit (CPU) or other processors capable of calling program code, such as a controller. Additionally, these modules can be integrated together to implement a system-on-a-chip (SOC).

[0251] In the above embodiments, implementation can be achieved, in whole or in part, through software, hardware, firmware, or any combination thereof. When implemented in software, it can be implemented, in whole or in part, as a computer program product. A computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the flow or function according to the embodiments of this application is generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., a solid-state disk (SSD)).

[0252] The term "multiple" in this document refers to two or more. The term "and / or" is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. Furthermore, the character " / " in this document generally indicates an "or" relationship between the preceding and following related objects; in formulas, " / " indicates a "division" relationship. Additionally, it should be understood that in the description of this application, words such as "first" and "second" are used only for descriptive purposes and should not be construed as indicating or implying relative importance or order.

[0253] It is understood that the various numerical designations used in the embodiments of this application are merely for descriptive convenience and are not intended to limit the scope of the embodiments of this application.

[0254] It is understood that, in the embodiments of this application, the order of the above-mentioned process numbers does not imply the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of this application.

Claims

1. A human-machine interaction method, characterized in that, The method is applied to an electronic device, and the method comprises: in response to a first operation, displaying a cursor on an interface, the cursor corresponding to an object in a real space, the cursor moving on the interface in response to movement of the object in the real space; when the cursor does not satisfy a speed adjustment condition, continuing to move the cursor at a first moving speed, the speed adjustment condition comprising: the first moving speed being less than or equal to a first threshold, and / or the cursor being within a preset range of any target control on the interface, the first moving speed being positively correlated with the moving speed of the object, the first moving speed being a moving speed determined based on the moving speed of the object and not being adjusted, the preset range of a target control surrounding the target control, and the preset ranges of any two target controls not having an intersection; when the cursor satisfies the speed adjustment condition, continuing to move the cursor at a second moving speed, the second moving speed being less than the first moving speed.

2. The method of claim 1, wherein, The first moving speed being determined based on the moving speed of the object specifically comprises: the first moving speed being obtained based on the moving speed of the object and an initial ratio.

3. The method according to claim 1 or 2, characterized in that, When the cursor satisfies the speed adjustment condition, the position of the cursor on the interface corresponding to a first control on the interface, the second moving speed being related to the size of the first control.

4. The method of claim 3, wherein, The second moving speed being positively correlated with the size of the first control.

5. The method according to claim 3 or 4, characterized in that, The position of the cursor on the interface corresponding to the first control on the interface specifically comprises: the position of the cursor on the interface being within a preset range of the first control.

6. The method according to any one of claims 3-5, characterized in that, Before the cursor continues to move at the second moving speed, the method further comprises: determining, according to the position of the cursor on the interface and the positions of target controls on the interface, that the cursor corresponds to the first control among the target controls; obtaining, according to the ratio corresponding to the first control and the moving speed of the object, the second moving speed corresponding to the first control.

7. The method according to any one of claims 3-5, characterized in that, Before the cursor continues to move at the second moving speed, the method further comprises: determining, according to the position of the cursor on the interface and the positions of target controls on the interface, that the cursor corresponds to the first control among the target controls; obtaining, according to the adjustment ratio corresponding to the first control and the first moving speed, the second moving speed corresponding to the first control.

8. The method of claim 7, wherein, The adjustment ratio corresponding to the first control comprises a first ratio and a second ratio, the first ratio being used to determine a first sub-moving speed, the second ratio being used to determine a second sub-moving speed, and the second moving speed corresponding to the first control comprising the first sub-moving speed and the second sub-moving speed; The adjustment ratio corresponding to the first control being related to a region within a preset range of the first control in which the cursor is located, different regions within the preset range of the first control being in a surrounding relationship.

9. The method of claim 8, wherein, The cursor continuing to move at the second moving speed comprises: when the cursor is in a first region within the preset range of the first control, moving the cursor at the first sub-moving speed; When the cursor is in a second region in a preset range of the first control, the cursor is moved at a second sub-moving speed, the first region surrounds the second region, and the first sub-moving speed is greater than the second sub-moving speed.

10. The method according to any one of claims 3-9, characterized in that, The method further includes: obtaining layout information of the interface, the layout information including sizes of each control on the interface; obtaining an adjustment ratio corresponding to the target control according to a size of the target control.

11. The method of claim 10, wherein, The obtaining of the layout information of the interface includes: calling a first application programming interface (API) to obtain the layout information.

12. The method of claim 10, wherein, The obtaining of the layout information of the interface includes: capturing a screenshot of the interface to obtain a first image; inputting the first image into a first model to obtain the layout information output by the first model, the first model being configured to obtain sizes and positions of each control in an image.

13. The method of any one of claims 3-9, wherein, The method further includes: capturing a screenshot of the interface to obtain a first image; inputting the first image into a second model to obtain a position of a target control on the interface and an adjustment ratio corresponding to the target control, the second model being configured to obtain the position of the target control and the adjustment ratio corresponding to the target control.

14. The method according to any one of claims 10-13, characterized in that, The method further includes: in response to an interface update, obtaining a position of a target control in an updated interface and an adjustment ratio corresponding to the target control.

15. The method of any one of claims 1-14, wherein, The object can be a body part of a user or an input device of the electronic device.

16. The method of claim 15, wherein, The body part of the user is specifically a hand of the user.

17. An electronic device, comprising: The electronic device includes one or more processors and a memory; The memory is coupled to the one or more processors, and the memory is configured to store computer program code including computer instructions, and the one or more processors are configured to invoke the computer instructions to cause the electronic device to perform the method in any one of claims 1-16.

18. A chip system, characterized by The chip system is applied to an electronic device, and the chip system includes one or more processors configured to invoke computer instructions to cause the electronic device to perform the method in any one of claims 1-16.

19. A computer-readable storage medium, characterized in that, The computer-readable storage medium includes computer instructions configured to cause an electronic device to perform the method in any one of claims 1-16 when the computer instructions are run on the electronic device.

20. A computer program product, characterised in that, The computer program product includes computer program code configured to cause an electronic device to perform the method in any one of claims 1-16 when the computer program code is run on the electronic device.

Images