Method of controlling a cursor and electronic device

Abstract

This application provides a method and electronic device for controlling a cursor. The method is applied to a first device, which is connected to a second device and a cursor input device. The method includes: the first device displaying the cursor on a first display screen of the first device at a first display-to-control ratio according to a first operation by a user on the cursor input device, the first display-to-control ratio being related to information on the first display screen and information on the cursor input device; responding to a second operation by the user on the cursor input device, the second operation instructing the cursor to be moved to a second display screen of the second device; and the first device displaying the cursor on the second display screen at a second display-to-control ratio according to a third operation by the user on the cursor input device, the second display-to-control ratio being related to information on the second display screen and information on the cursor input device. Through this method and electronic device, the display-to-control ratio can be dynamically adjusted when the display screen showing the cursor changes, thereby enabling dynamic adjustment of the cursor movement speed and improving cursor responsiveness.

Description

Technical Field

[0001] This application relates to the field of cursor control, and more specifically, to a method and electronic device for controlling a cursor. Background Technology

[0002] As the ecosystem of smart devices continues to develop, the types of displays and cursor input devices connected to smart devices are increasing, allowing users to control the movement of the cursor on the display device through the cursor input device.

[0003] However, when smart devices are connected to different display devices and / or different cursor input devices, the cursor movement may become unresponsive, affecting the user experience. Summary of the Invention

[0004] This application provides a method and electronic device for controlling a cursor. Through this method and electronic device, the electronic device can adaptively adjust the display-to-control ratio according to the information of the display screen and the cursor input device connected to it. When the user uses the cursor input device to control the movement of the cursor on the display screen, the cursor can be displayed on the display screen according to the adaptively adjusted display-to-control ratio. In this way, when the display screen displaying the cursor changes, the electronic device can dynamically adjust the display-to-control ratio, thereby dynamically adjusting the current cursor movement speed, improving the cursor responsiveness, and giving the user a better cursor usage experience in different peripheral scenarios.

[0005] In a first aspect, a method for controlling a cursor is provided, the method being applied to a first device, the first device being connected to a second device and a cursor input device, the method comprising: the first device displaying the cursor on a first display screen of the first device at a first display ratio according to a first operation by a user on the cursor input device, the first display ratio being related to information on the first display screen and information on the cursor input device; responding to a second operation by the user on the cursor input device, the second operation being used to instruct the cursor to be moved to a second display screen of the second device for display; and the first device displaying the cursor on the second display screen of the second device at a second display ratio according to a third operation by the user on the cursor input device, the second display ratio being related to information on the second display screen and information on the cursor input device.

[0006] In this embodiment, the electronic device can adaptively adjust the display control ratio based on the information of the connected display screen and the information of the cursor input device. When the user uses the cursor input device to control the movement of the cursor on the display screen, the cursor can be displayed on the display screen according to the adaptively adjusted display control ratio. In this way, when the display screen displaying the cursor changes, the electronic device can dynamically adjust the display control ratio, thereby dynamically adjusting the current cursor movement speed, improving the responsiveness of the cursor, and giving the user a better cursor usage experience in different peripheral scenarios.

[0007] In conjunction with the first aspect, in one possible implementation, the information of the first display screen includes one or more of the following: screen size, screen resolution, and pixel density unit; the information of the second display screen includes one or more of the following: screen size, screen resolution, and pixel density unit.

[0008] In conjunction with the first aspect, in one possible implementation, when the cursor input device is a touchpad, the information of the cursor input device includes one or more of the following: the size of the touchpad, the reporting rate, the resolution, and the pixel density unit; when the cursor input device is a mouse, the information of the cursor input device includes the dots per inch (DPI) and / or the reporting rate of the mouse.

[0009] In conjunction with the first aspect, in one possible implementation, before displaying the cursor on the first display screen of the first device at a first display-to-control ratio, the method further includes: the first device acquiring information about the first display screen and information about the cursor input device; and the first device determining the first display-to-control ratio based on the information about the first display screen and the information about the cursor input device.

[0010] In this embodiment, the electronic device can adaptively adjust the display-to-control ratio based on the information of the connected display screen and the information of the cursor input device. When the user uses the cursor input device to control the movement of the cursor on the display screen, the cursor can be displayed on the display screen according to the adaptively adjusted display-to-control ratio. This enables dynamic adjustment of the current cursor movement speed, improves the responsiveness of the cursor, and provides the user with a better cursor usage experience in different peripheral scenarios.

[0011] In conjunction with the first aspect, in one possible implementation, before displaying the cursor on the second display screen of the second device at a second display-to-control ratio, the method further includes: the first device acquiring information about the second display screen and information about the cursor input device; and the first device determining the second display-to-control ratio based on the information about the second display screen and the information about the cursor input device.

[0012] In this embodiment, the electronic device can adaptively adjust the display control ratio based on the information of the connected display screen and the information of the cursor input device. When the user uses the cursor input device to control the movement of the cursor on the display screen, the cursor can be displayed on the display screen according to the adaptively adjusted display control ratio. In this way, when the display screen displaying the cursor changes, the electronic device can dynamically adjust the display control ratio, thereby dynamically adjusting the current cursor movement speed, improving the responsiveness of the cursor, and giving the user a better cursor usage experience in different peripheral scenarios.

[0013] In conjunction with the first aspect, in one possible implementation, the first device displays a cursor on a first display screen of the first device at a first display-to-control ratio based on a first operation by a user on the cursor input device, comprising: the first device receiving a first movement distance sent by the second device, the first movement distance being the movement distance generated by the cursor input device in response to the first operation; the first device determining a second movement distance of the cursor based on the first display-to-control ratio and the first movement distance; and the first device displaying the cursor on the first display screen based on the second movement distance.

[0014] In this embodiment, the electronic device can adaptively adjust the display control ratio based on the information of the connected display screen and the cursor input device. When the user uses the cursor input device to control the movement of the cursor on the display screen, the electronic device can determine the cursor's movement information on the display screen based on the adaptively adjusted display control ratio and the movement information of the cursor input device, thereby determining the cursor's display position. The cursor is then displayed on the display screen based on its display position. When the display screen showing the cursor changes, the electronic device can dynamically adjust the display control ratio, thereby achieving dynamically adjustable cursor movement speed. This means that the cursor's display position can be determined more accurately, improving the cursor's responsiveness and providing users with a better cursor usage experience in different peripheral scenarios.

[0015] In conjunction with the first aspect, in one possible implementation, the first travel includes a first offset and a second offset, the first offset being the horizontal travel of the cursor input device in response to the first operation, and the second offset being the vertical travel of the cursor input device in response to the first operation; the second travel includes a third offset and a fourth offset, the third offset being the horizontal travel of the cursor, and the fourth offset being the vertical travel of the cursor; the first device determines the second travel of the cursor based on the first display-to-control ratio and the first travel, including: the first device determining the third offset based on the first display-to-control ratio and the first offset; and the first device determining the fourth offset based on the first display-to-control ratio and the second offset.

[0016] In conjunction with the first aspect, in one possible implementation, the first device displays a cursor on the first display screen according to the second stroke, including: the first device determines a second coordinate position based on a first coordinate position, the third offset, and the fourth offset, the first coordinate position being the current display position of the cursor; the first device controls the first display screen to display the cursor at the second coordinate position.

[0017] In this embodiment of the application, the electronic device can adaptively adjust the display control ratio according to the information of the display screen and the information of the cursor input device connected to it. When the user uses the cursor input device to control the movement of the cursor on the display screen, the electronic device can determine the movement information of the cursor on the display screen according to the adaptively adjusted display control ratio and the movement information of the cursor input device. Then, it can determine the target display position of the cursor according to the current display position and movement information of the cursor, and display the cursor on the display screen according to the target display position of the cursor.

[0018] In conjunction with the first aspect, in one possible implementation, the first device is a smartphone, tablet, or personal computer (PC).

[0019] In conjunction with the first aspect, in one possible implementation, the second device is an external display screen, projector, tablet computer, personal computer (PC), or smartphone.

[0020] In conjunction with the first aspect, in one possible implementation, the cursor input device is a touchpad or a mouse.

[0021] In a second aspect, an electronic device is provided, comprising a memory and a processor, wherein the memory is used to store computer program code, and the processor is used to execute the computer program code stored in the memory to implement the method in the first aspect or any possible implementation thereof.

[0022] Thirdly, a computer-readable storage medium is provided, which stores a computer program or instructions that, when executed, implement the method of the first aspect or any possible implementation thereof.

[0023] Fourthly, a chip is provided, wherein instructions are stored that, when executed on a device, cause the chip to perform the methods described in the first aspect or any possible implementation thereof.

[0024] Fifthly, a computer program product is provided, which stores a computer program or instructions that, when executed, implement the method in the first aspect or any possible implementation of the first aspect. Attached Figure Description

[0025] Figure 1 This is a schematic diagram of the structure of the electronic device provided in the embodiments of this application;

[0026] Figure 2 This is a software structure block diagram of the electronic device provided in the embodiments of this application;

[0027] Figure 3This is a schematic diagram of an application scenario provided by an embodiment of this application;

[0028] Figure 4 This is a schematic diagram of another application scenario provided by an embodiment of this application;

[0029] Figure 5 This is a schematic diagram of another application scenario provided by an embodiment of this application;

[0030] Figure 6 This is a schematic flowchart illustrating a method for controlling a cursor provided in the embodiments of the application;

[0031] Figure 7 This is a schematic flowchart illustrating yet another method for controlling a cursor provided in the application embodiments;

[0032] Figure 8 This is a schematic diagram of a system architecture for an electronic device used to control a cursor, provided in an embodiment of this application;

[0033] Figure 9 This is a schematic diagram illustrating the interaction between the first display screen, the first device, and the cursor input device provided in an embodiment of this application;

[0034] Figure 10 This is a schematic flowchart of another method 1000 for controlling a cursor provided in the embodiments of this application. Detailed Implementation

[0035] The technical solutions of this application will now be described with reference to the accompanying drawings. Obviously, the described embodiments are merely some embodiments of this application, and not all embodiments.

[0036] The technical solutions of the embodiments of this application will be described below with reference to the accompanying drawings. In the description of the embodiments of this application, unless otherwise stated, " / " means "or," for example, A / B can mean A or B; "and / or" in this text is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Furthermore, in the description of the embodiments of this application, "plural" or "multiple" refers to two or more than two.

[0037] Hereinafter, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this embodiment, unless otherwise stated, "a plurality of" means two or more.

[0038] The terminology used in the following embodiments is for the purpose of describing particular embodiments only and is not intended to be limiting of this application. As used in the specification and appended claims of this application, the singular expressions “a,” “an,” “the,” “the,” “the,” and “this” are intended to also include expressions such as “one or more,” unless the context clearly indicates otherwise. It should also be understood that in the following embodiments of this application, “at least one” and “one or more” refer to one, two, or more than two. The term “and / or” is used to describe the relationship between related objects, indicating that three relationships may exist; for example, A and / or B can indicate: A alone, A and B simultaneously, or B alone, where A and B can be singular or plural. The character “ / ” generally indicates that the preceding and following related objects are in an “or” relationship.

[0039] References to "one embodiment" or "some embodiments" as described in this specification mean that one or more embodiments of this application include a specific feature, structure, or characteristic described in connection with that embodiment. Therefore, the phrases "one embodiment," "some embodiments," "another embodiment," "other embodiments," etc., appearing in different parts of this specification do not necessarily refer to the same embodiment, but rather mean "one or more, but not all, embodiments," unless otherwise specifically emphasized. The terms "comprising," "including," "having," and variations thereof mean "including but not limited to," unless otherwise specifically emphasized.

[0040] The methods provided in this application can be applied to electronic devices such as mobile phones, tablets, wearable devices, in-vehicle devices, augmented reality (AR) / virtual reality (VR) devices, laptops, ultra-mobile personal computers (UMPCs), netbooks, and personal digital assistants (PDAs). This application does not impose any restrictions on the specific type of electronic device.

[0041] For example, Figure 1A schematic diagram of the structure of electronic device 100 is shown. Electronic device 100 may include a processor 110, an external memory interface 120, an internal memory 121, a universal serial bus (USB) interface 130, a charging management module 140, a power management module 141, a battery 142, antenna 1, antenna 2, a mobile communication module 150, a wireless communication module 160, an audio module 170, a speaker 170A, a receiver 170B, a microphone 170C, a headphone jack 170D, a sensor module 180, buttons 190, a motor 191, an indicator 192, a camera 193, a display screen 194, and a subscriber identification module (SIM) card interface 195, etc. The sensor module 180 may include a pressure sensor 180A, a gyroscope sensor 180B, a barometric pressure sensor 180C, a magnetic sensor 180D, an accelerometer sensor 180E, a distance sensor 180F, a proximity sensor 180G, a fingerprint sensor 180H, a temperature sensor 180J, a touch sensor 180K, an ambient light sensor 180L, a bone conduction sensor 180M, etc.

[0042] It is understood that the structures illustrated in the embodiments of this application do not constitute a specific limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may include more or fewer components than illustrated, or combine some components, or split some components, or have different component arrangements. The illustrated components may be implemented in hardware, software, or a combination of software and hardware.

[0043] Processor 110 may include one or more processing units, such as: application processor (AP), modem processor, graphics processing unit (GPU), image signal processor (ISP), controller, memory, video codec, digital signal processor (DSP), baseband processor, and / or neural network processing unit (NPU), etc. Different processing units may be independent devices or integrated into one or more processors.

[0044] The controller can be the nerve center and command center of the electronic device 100. The controller can generate operation control signals according to the instruction opcode and timing signals to complete the control of fetching and executing instructions.

[0045] The processor 110 may also include a memory for storing instructions and data. In some embodiments, the memory in the processor 110 is a cache memory. This memory can store instructions or data that the processor 110 has just used or that are used repeatedly. If the processor 110 needs to use the instruction or data again, it can retrieve it directly from the memory. This avoids repeated accesses, reduces the waiting time of the processor 110, and thus improves the efficiency of the system.

[0046] In some embodiments, the processor 110 may include one or more interfaces. Interfaces may include an inter-integrated circuit (I2C) interface, an inter-integrated circuit sound (I2S) interface, a pulse code modulation (PCM) interface, a universal asynchronous receiver / transmitter (UART) interface, a mobile industry processor interface (MIPI), a general-purpose input / output (GPIO) interface, a subscriber identity module (SIM) interface, and / or a universal serial bus (USB) interface, etc.

[0047] It is understood that the interface connection relationships between the modules illustrated in the embodiments of this application are merely illustrative and do not constitute a structural limitation on the electronic device 100. In other embodiments of this application, the electronic device 100 may also employ different interface connection methods or combinations of multiple interface connection methods as described in the above embodiments.

[0048] The charging management module 140 receives charging input from the charger. While charging the battery 142, the charging management module 140 can also supply power to the electronic device through the power management module 141.

[0049] The power management module 141 connects the battery 142, the charging management module 140, and the processor 110. The power management module 141 receives input from the battery 142 and / or the charging management module 140, providing power to the processor 110, internal memory 121, external memory, display screen 194, camera 193, and wireless communication module 160. The power management module 141 can also monitor parameters such as battery capacity, battery cycle count, and battery health status (leakage current, impedance). In some other embodiments, the power management module 141 may also be located within the processor 110. In other embodiments, the power management module 141 and the charging management module 140 may be housed in the same device. The wireless communication function of the electronic device 100 can be implemented through antenna 1, antenna 2, mobile communication module 150, wireless communication module 160, modem processor, and baseband processor.

[0050] Electronic device 100 implements display functions through a GPU, a display screen 194, and an application processor. The GPU is a microprocessor for image processing, connected to the display screen 194 and the application processor. The GPU is used to perform mathematical and geometric calculations and for graphics rendering. Processor 110 may include one or more GPUs, which execute program instructions to generate or modify display information.

[0051] Display screen 194 is used to display images, videos, etc. Display screen 194 includes a display panel. The display panel may be a liquid crystal display (LCD), an organic light-emitting diode (OLED), an active-matrix organic light-emitting diode (AMOLED), a flexible light-emitting diode (FLED), a miniature LED, a microLED, a quantum dot light-emitting diode (QLED), etc. In some embodiments, electronic device 100 may include one or N displays 194, where N is a positive integer greater than 1.

[0052] Electronic device 100 can perform shooting functions through ISP, camera 193, video codec, GPU, display 194 and application processor.

[0053] An NPU (Neural Processing Unit) is a computational processor for neural networks (NNs). By borrowing the structure of biological neural networks, such as the transmission patterns between neurons in the human brain, it can rapidly process input information and continuously learn on its own. NPUs enable intelligent cognitive applications in electronic devices, such as image recognition, facial recognition, speech recognition, and text understanding.

[0054] The external storage interface 120 can be used to connect an external memory card, such as a Micro SD card, to expand the storage capacity of the electronic device 100. The external memory card communicates with the processor 110 through the external storage interface 120 to perform data storage functions. For example, music, video, and other files can be saved on the external memory card.

[0055] Internal memory 121 can be used to store computer executable program code, which includes instructions. Processor 110 executes various functional applications and data processing of electronic device 100 by running the instructions stored in internal memory 121. Internal memory 121 may include a program storage area and a data storage area. The program storage area may store the operating system, at least one application required for a function (such as sound playback, image playback, etc.), etc. The data storage area may store data created during the use of electronic device 100 (such as audio data, phonebook, etc.). Furthermore, internal memory 121 may include high-speed random access memory and may also include non-volatile memory, such as at least one disk storage device, flash memory device, universal flash storage (UFS), etc.

[0056] Electronic device 100 can implement audio functions, such as music playback and recording, through audio module 170, speaker 170A, receiver 170B, microphone 170C, headphone jack 170D, and application processor.

[0057] Indicator 192 can be an indicator light, used to indicate charging status, power changes, or to indicate messages, missed calls, notifications, etc.

[0058] The SIM card interface 195 is used to connect a SIM card. The electronic device 100 interacts with the network through the SIM card to realize functions such as making calls and data communication.

[0059] It should be understood that the phone cards in the embodiments of this application include, but are not limited to, SIM cards, eSIM cards, universal subscriber identity modules (USIM), universal integrated circuit cards (UICC), etc.

[0060] The electronic device provided in this application embodiment can run an operating system (OS). This operating system can be various operating systems currently used in the industry, such as HarmonyOS based on OpenHarmony; or other operating systems such as Android™, iOS mobile operating system; it can also be various open-source operating systems or their derivatives, such as Linux OS, and other embedded operating systems; or it can be a future new operating system, such as an AI operating system based on artificial intelligence. An operating system is a set of interconnected system software programs that manage and control the operation of electronic devices, utilize and run hardware and software resources, and provide public services to organize user interaction. The operating system occupies a pivotal position in electronic devices, connecting to the physical devices at the hardware layer below and providing a runtime environment for application software above.

[0061] An operating system typically includes a kernel layer, a middleware layer, and an application layer. The application layer includes applications, which can include system applications and third-party applications. The middleware layer is a suite of software, or frameworks, that provides various services to application developers, such as databases, multimedia, and graphics, or capabilities like distributed scheduling and system expansion. For example, the middleware layer can also be broadly divided into a framework layer and / or a system service layer. The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. The system service layer includes the system's core capabilities, providing services to applications through the framework layer. The kernel layer is the layer between hardware and software. The kernel layer can include hardware drivers and the operating system kernel. In addition to providing hardware drivers, the kernel layer also supports functions such as memory management and system process management.

[0062] The electronic devices we use in our daily lives come in various types and forms, and are applied in a wide range of scenarios. Therefore, based on the different forms and functions of electronic devices, different application scenarios, and different user needs, the operating systems used in these devices may also differ. These operating systems share commonalities but also have their own unique characteristics. Different operating systems affect user experience, application ecosystem, and system performance. The basic functions implemented by the electronic device provided in this application can be achieved using a general-purpose operating system or a dedicated operating system.

[0063] To more clearly illustrate the implementation of the embodiments of this application under a specific operating system, exemplarily, Figure 2The architecture of HarmonyOS is shown, and those skilled in the art can deduce the implementation of the embodiments of this application under other specific operating systems, such as the implementation under the Android™ operating system.

[0064] like Figure 2 As shown, the software architecture of an electronic device can be divided into several layers. In some embodiments, from bottom to top, these layers are: kernel layer, system service layer, framework layer, and application layer. The layers communicate with each other through software interfaces. System functions can be tailored, added, or combined at the subsystem granularity in different device deployment scenarios, and each subsystem can also be tailored, added, or combined at the functional granularity.

[0065] (1) Kernel layer

[0066] The Kernel Abstraction Layer (KAL) provides basic kernel capabilities to upper layers by shielding the differences between multiple kernels, including but not limited to process / thread management, memory management, file system, network management, and peripheral device management.

[0067] Kernel Subsystem: Supports the selection of a suitable OS kernel for different resource-constrained devices, including but not limited to Linux kernel, HarmonyOS kernel, LiteOS, etc.

[0068] Driver Subsystem: The driver framework is the foundation for the open system hardware ecosystem, providing unified peripheral access capabilities and a framework for driver development and management. The driver framework includes: display drivers, camera drivers, audio drivers, Bluetooth drivers, sensor drivers, etc.

[0069] (2) System Service Layer

[0070] The system service layer comprises the core capabilities of the system, providing services to applications through the framework layer. This layer includes, but is not limited to, the following subsystems:

[0071] The system's basic capability subsystems provide fundamental capabilities for the operation, scheduling, and migration of distributed applications across multiple devices. For example, they may include distributed soft bus, distributed data management, distributed task scheduling, and the Ark multi-language runtime. They may also include multi-modal input subsystems, graphics subsystems, security subsystems, and AI subsystems.

[0072] Basic software service subsystems: provide common and general software services; for example, event notification subsystem, telephone service subsystem, multimedia subsystem, etc.

[0073] Enhanced software service subsystem suite: Provides differentiated capability-enhancing software services for different devices; for example, it may include proprietary business subsystems for smart screens, wearable devices, and IoT devices.

[0074] Hardware service subsystem set: provides hardware services; for example, it may include location service subsystem, user IAM (Identity and Access Management) subsystem, wearable proprietary hardware service subsystem, biometric identification, IoT proprietary hardware service subsystem, etc.

[0075] Distributed task scheduling enables distributed service management (discovery, synchronization, registration, and invocation), supporting remote startup, remote invocation, remote connection, and migration of applications across devices.

[0076] Distributed data management enables data synchronization, data storage, data sharing, and data access across all scenarios and devices.

[0077] The distributed soft bus provides communication-related capabilities for seamless interconnection between multiple devices, including: WLAN service capabilities, Bluetooth service capabilities, soft bus, inter-process communication RPC (Remote Procedure Call) and other communication capabilities.

[0078] Ark Multilingual Runtime is a unified compilation runtime platform designed to support the joint compilation and execution of multiple programming languages ​​and multiple chip platforms.

[0079] (3) Framework layer

[0080] The framework layer provides application programming interfaces (APIs) and programming frameworks for applications in the application layer. Examples include the ArkUI framework (which provides a complete infrastructure for UI development of system applications, including UI functionalities such as components, layouts, animations, and interactive events, as well as a real-time interface preview tool), the user application framework, and the Ability framework (an Ability is a lightweight application; the Ability framework schedules and manages the operation and lifecycle of Abilities). Different devices may run different operating systems, and therefore support different APIs.

[0081] The HarmonyOS API is a series of open capabilities provided to support HarmonyOS application development. The HarmonyOS API can be set at the framework layer or independently of the framework layer. Examples include: Audio API (audio service), Push API (push service), and Account API (account service).

[0082] (4) Application layer

[0083] Applications can include system apps and extended / third-party apps. System apps can include the desktop, control bar, settings, contacts, phone, camera, etc., while extended / third-party apps can include social apps, travel apps, etc.

[0084] It should be understood that the technical solutions in the embodiments of this application can be used in systems such as Android, iOS, and HarmonyOS.

[0085] The technical solutions of this application embodiment can be applied to electronic devices with voice functionality, for example. Exemplarily, they can be applied to televisions, desktop computers, laptops, in-vehicle screens, and portable electronic devices such as mobile phones, foldable screens, tablets, cameras, camcorders, video recorders, watches, bracelets, and smart glasses. They can also be applied to other electronic devices with voice functionality, and further to electronic devices in future networks or in future evolved public land mobile networks (PLMNs). The main application scenario is a scenario where users interact with electronic devices via voice.

[0086] As the ecosystem of smart devices (such as personal computers, PCs) continues to develop, the types of display devices and cursor input devices that smart devices can connect to are increasing. For example, in addition to the display screen that it has built into the PC, a PC can also connect to other external displays, as well as other smart devices with displays, such as projectors, smart screens, and tablets. It can also connect to external cursor input devices such as touchpads, mice, and remote controls. Users can control the movement of the cursor on the display screen by operating the cursor input device.

[0087] However, when smart devices are connected to different display screens and / or different cursor input devices, the speed at which the cursor moves varies greatly on different screens due to the limited display-to-control ratio between the cursor input device and the display screen. When the display screen and / or cursor input device are changed, the cursor on the display screen may not be responsive, which affects the user experience.

[0088] In some embodiments, a method for controlling cursor movement is provided, the method comprising: acquiring the relative motion of a control device within a first time period, the control device being placed inside a mobile vehicle, the reference object for the relative motion being the mobile vehicle, the control device being capable of controlling cursor movement based on inertia; and moving the cursor based on the relative motion.

[0089] In some embodiments, a cursor control method applied to a first electronic device is provided. The method includes: receiving a first instruction sent by a second electronic device, and displaying the cursor as stationary according to the first instruction; receiving a second instruction sent by the second electronic device, changing the cursor from stationary to pointing state according to the second instruction, and moving at least one of the cursor, the current focus of the cursor, and a list displayed on the screen of the first electronic device on the screen of the first electronic device; and displaying the cursor as stationary when the cursor or the list stops moving. Wherein, when the cursor is displayed as pointing, the cursor, the current focus of the cursor, and at least one item in the list are movable; when the cursor is displayed as stationary, the cursor, the current focus of the cursor, and the list are immovable; receiving a third instruction sent by the second electronic device, and playing the screen corresponding to the target focus where the cursor is located according to the third instruction.

[0090] These cursor control methods are all based on the display / control ratio manually configured by the display device and cursor input device. They do not support adjusting the control / display ratio according to the screen size and / or screen resolution of the display device. When the display device and / or cursor input device changes, the currently configured control / display ratio may no longer be applicable. Especially when the screen size of the display device increases significantly, the cursor movement speed is slow when the user controls the cursor movement through the cursor input device. That is, the cursor movement speed varies greatly on different screens, which can easily lead to the cursor not following the user's movements. To improve this situation, the user needs to manually reconfigure the display / control ratio, which is cumbersome.

[0091] In view of this, embodiments of this application provide a method and electronic device for controlling a cursor. Through this method and electronic device, the electronic device can adaptively adjust the display-to-control ratio according to the cursor display device and / or cursor input device connected to it. When the user uses the cursor input device to control the cursor movement on the cursor display device, the cursor can be displayed on the cursor display device according to the adaptively adjusted display-to-control ratio. In this way, when the cursor display device and / or cursor input device changes, the electronic device can dynamically adjust the display-to-control ratio, thereby dynamically adjusting the current cursor movement speed, improving the cursor responsiveness, and giving the user a better cursor usage experience in different peripheral scenarios.

[0092] The control-display ratio (C / D ratio) refers to the ratio of the travel distance of the moving elements (such as pointers or cursors) of the controller (i.e., the cursor input device) to that of the display (i.e., the display device). A high C / D ratio indicates low controller sensitivity, meaning that a larger control movement produces a smaller display movement; a low C / D ratio indicates high controller sensitivity, meaning that a smaller control movement produces a larger display movement.

[0093] The method provided in this application can be applied to electronic devices that can connect to devices with displays and cursor input devices, such as PCs, tablets, smartphones, and smart screens.

[0094] In some embodiments, the solutions provided in this application can be applied to scenarios where an electronic device has an external display screen, such as a PC connecting to the display screen of another PC, or a PC having its own display screen and then connecting another external display screen.

[0095] In some embodiments, the solutions provided in this application can be applied to scenarios where electronic devices project their screens, such as projecting the screen displayed on a PC onto a projector, or projecting the screen displayed on a PC onto a smart screen.

[0096] In some embodiments, the solutions provided in this application can be applied to scenarios where keyboards and mice are shared between electronic devices, such as sharing a keyboard and mouse between the display screen of one PC and the display screen of another PC. Devices with displays may include, for example, a display screen, a computer, a smart screen, a projector, etc.; cursor input devices may include, for example, a mouse, a pointing remote control, a touchpad, etc.

[0097] For example, taking the screen mirroring scenario of electronic devices as an example, Figure 3 A schematic diagram of an application scenario provided by an embodiment of this application is shown.

[0098] like Figure 3 As shown, taking electronic device 310 as a PC and electronic device 320 as a smart screen as an example, electronic device 310 is a screen projection device and electronic device 320 is the screen being projected. Electronic device 310 includes a touchpad 311 and a display screen 312. The display screen 312 displays a cursor 330. When the user's gesture moves from position A to position B on the touchpad 311, according to the display control ratio between the display screen 312 and the touchpad 311, the cursor 330 displayed on the display screen 312 is moved from position A1 to position B1.

[0099] The display control ratio between the display screen 312 and the touchpad 311 is related to the information on the display screen 312 and the information on the touchpad 311.

[0100] Correspondingly, when the user's gesture moves from position A to position B on the touchpad 311, the cursor 330 displayed on the screen of the electronic device 320 is moved from position A2 to position B2 according to the display control ratio between the display screen of the electronic device 320 and the touchpad 311.

[0101] The display-to-control ratio between the display screen and the touchpad 311 of the electronic device 320 is related to the information on the display screen and the information on the touchpad 311 of the electronic device 320.

[0102] In some embodiments, when the cursor input device changes from the touchpad 311 to the mouse 1, the electronic device 310 can determine the display control ratio between the display screen 312 and the mouse 1 based on the information of the display screen 312 and the mouse 1, and display the cursor 330 on the display screen 312 based on the display control ratio between the display screen 312 and the mouse 1; similarly, it can also determine the display control ratio between the display screen of the electronic device 320 and the mouse 1 based on the information of the display screen of the electronic device 320 and the mouse 1, and display the cursor 330 on the display screen of the electronic device 320 based on the display control ratio between the display screen of the electronic device 320 and the mouse 1.

[0103] For example, taking the scenario of an external screen for an electronic device as an example, Figure 4 This illustration shows another application scenario provided by an embodiment of this application.

[0104] Taking an electronic device as an example, such as a PC Figure 4 As shown in (a), electronic device 410 is a PC and electronic device 420 is an external display screen. Electronic device 410 includes a touchpad 411 and a display screen 412. Window 1 is displayed on the display screen 412 of electronic device 410 and window 2 is displayed on the display screen of electronic device 420. When the user's finger slides on the touchpad 411, the cursor 430 is displayed on the display screen 412 according to the display control ratio between the display screen 412 and the touchpad 411.

[0105] like Figure 4 As shown in (b), the user can move the cursor to the external display screen 420 by performing a right swipe operation on the touchpad 411. After the cursor is moved to the external display screen 420, when the user's finger slides on the touchpad 411, the cursor 430 is displayed on the external display screen 420 according to the display control ratio between the external display screen 420 and the touchpad 411.

[0106] For example, taking the scenario of sharing a keyboard and mouse on an electronic device as an example, Figure 5 This illustration shows another application scenario provided by an embodiment of this application.

[0107] Taking an electronic device as an example, such as a PC Figure 5As shown in (a), electronic device 510 is a PC and electronic device 520 is a tablet. Electronic device 510 includes a touchpad 511 and a display screen 512. Window 1 is displayed on the display screen 412 of electronic device 510, and a video application interface is displayed on the display screen of electronic device 520. When the user's finger slides on the touchpad 511, the cursor 530 is displayed on the display screen 512 according to the display control ratio between the display screen 512 and the touchpad 511.

[0108] Users can operate the interface of electronic device 520 through the touchpad 511 of electronic device 510.

[0109] like Figure 5 As shown in (b), the user can move the cursor 530 to the display screen of the electronic device 520 by performing a right swipe operation on the touchpad 511. After the cursor 530 is displayed on the display screen of the electronic device 520, when the user's finger slides on the touchpad 511, the cursor 530 is displayed on the display screen of the electronic device 520 according to the display control ratio between the display screen of the electronic device 520 and the touchpad 511. For example, the user can trigger the "Home" control in the video application interface displayed on the display screen of the electronic device 520 by performing a swipe and click operation on the touchpad 511.

[0110] In some embodiments, after the cursor 530 is moved from the display screen 512 to the display screen of the electronic device 520, the user can also operate the interface of the electronic device 520 through the keyboard of the electronic device 510.

[0111] In some embodiments, when a mouse is connected to the electronic device 510, after the cursor 530 is moved from the display screen 512 to the display screen of the electronic device 520, the user can also perform interface operations on the electronic device 520 using the mouse connected to the electronic device 510.

[0112] For example, Figure 6 A schematic flowchart of a cursor control method 600 provided in an embodiment of this application is shown. Figure 6 As shown, the method 600 includes:

[0113] S601: The first device connects to the second device and the cursor input device.

[0114] In some embodiments, the second device itself may be a display device, such as a display screen external to the first device.

[0115] In some embodiments, the second device may be an electronic device with a display screen, such as a PC, a smart screen, a tablet, or other electronic devices.

[0116] In some embodiments, the cursor input device may be a mouse, a touchpad, a pointing remote control, or other device capable of controlling the cursor displayed on the screen.

[0117] In some embodiments, the relationship between the second display screen of the second device and the first device is that the second display screen is an external display screen of the first device.

[0118] In some embodiments, the relationship between the second device and the first device is that the first device is the screen projection device and the second device is the screen being projected onto.

[0119] In some embodiments, the relationship between the second device and the first device is that the first device and the second device share a set of mouse and keyboard, that is, the first device and the second device share a set of mouse and keyboard, that is, the user can use a cursor input device to control the cursor to be displayed on the first display screen and control the cursor to be displayed on the second display screen.

[0120] S602: Based on the user's first operation on the cursor input device, the first device displays the cursor on the first display screen of the first device with a first display-to-control ratio, the first display-to-control ratio being related to the information on the first display screen and the information on the cursor input device.

[0121] The first display-to-control ratio is the control-to-display ratio between the first display screen and the cursor input device.

[0122] When the cursor input device is a touchpad, the first operation may be, for example, the user's finger sliding on the touchpad; when the cursor input device is a mouse, the first operation may be, for example, the user moving the mouse on a flat surface.

[0123] In some embodiments, the information of the first display screen of the first device may include one or more of the following: screen size, screen resolution, and pixel density unit (pixels per inch, PPI) of the first display screen, and may also include other information of the first display screen, which is not limited in this application.

[0124] In some embodiments, the first device determines the PPI of the first display screen based on the size information of the first display screen and the resolution of the first display screen. For example, the ratio between the resolution of the first display screen and the size of the first display screen can be regarded as the PPI of the first display screen.

[0125] In some embodiments, when the cursor input device is a touchpad, the information of the cursor input device may include one or more of the following: the size of the touchpad, the reporting rate, the resolution, and the pixel density unit, and may also include other information of the touchpad, which is not limited in this application.

[0126] PPI stands for the number of pixels per inch.

[0127] The reporting rate refers to the frequency at which the touchpad reports the movement to the first device.

[0128] In some embodiments, when the cursor input device is a mouse, the information of the cursor input device includes the mouse's dots per inch (DPI) and / or reporting rate, and may also include other information of the mouse, which is not limited in this application.

[0129] The reporting rate refers to the frequency at which the mouse reports its movement to the first device.

[0130] S603: Responds to a second operation by the user on the cursor input device, the second operation being used to instruct the cursor to be moved to a second display screen of the second device.

[0131] When the cursor input device is a touchpad, the second operation can be, for example, the user's finger sliding on the touchpad; when the cursor input device is a mouse, the second operation can be, for example, the user moving the mouse on a flat surface.

[0132] S604: The first device displays the cursor on the second display screen of the second device at a second display control ratio according to the user's third operation on the cursor input device. The second display control ratio is related to the information on the second display screen and the information on the cursor input device.

[0133] The second display-to-control ratio is the control-to-display ratio between the second display screen and the cursor input device.

[0134] When the cursor input device is a touchpad, the third operation can be, for example, the user's finger sliding on the touchpad; when the cursor input device is a mouse, the third operation can be, for example, the user moving the mouse on a flat surface.

[0135] In some embodiments, the information of the second display screen of the second device may include one or more of the following: screen size, screen resolution, and pixel density unit of the second display screen, and may also include other information of the second display screen, which is not limited in this application.

[0136] In some embodiments, when the cursor input device is changed, the first device may also redetermine the display control ratio based on the information of the display screen currently displaying the cursor and the information of the replaced cursor input device.

[0137] In this embodiment, when the display screen and / or cursor input device changes, the electronic device can dynamically adjust the display-to-control ratio between the cursor input device and the display screen based on the information of the display screen and / or the cursor input device. This can also be understood as dynamically adjusting the cursor movement speed based on the information of the display screen and / or the cursor input device, improving cursor responsiveness. Users do not need to manually adjust the display-to-control ratio when the cursor input device or the display screen is changed, thereby improving the user experience.

[0138] For example, Figure 7 A schematic flowchart of another method 700 for controlling a cursor provided in an embodiment of this application is shown. Figure 7 As shown, the method 700 includes:

[0139] S701: The first device connects to the second device and the cursor input device. When the cursor is displayed on the first display screen of the first device, the first device determines the first display-to-control ratio between the first display screen and the cursor input device based on the information of the first display screen and the information of the cursor input device.

[0140] The explanation of the information on the first display screen and the cursor input device is as follows: Figure 6 The embodiments shown have been described in detail, and for the sake of brevity, they will not be repeated here.

[0141] In some embodiments, the screen size of the first display screen is negatively correlated with the first display-to-control ratio (V / C), that is, the larger the screen size of the first display screen, the smaller the first V / C, and the smaller the screen size of the first display screen, the larger the first V / C.

[0142] In some embodiments, the PPI of the first display screen is negatively correlated with the first display-to-control ratio, that is, the larger the PPI of the first display screen, the smaller the first display-to-control ratio, and the smaller the PPI of the first display screen, the larger the first display-to-control ratio.

[0143] In one implementation, the first device determines the first display control ratio based on the information of the first display screen and the information of the cursor input device, which may include: the first device correcting the display control ratio configured in the first device based on the information of the first display screen and the information of the cursor input device to obtain the first display control ratio.

[0144] S702: In response to user operation 1 on cursor input device, the first device obtains movement 1 from cursor input device.

[0145] When the cursor input device is a touchpad, operation 1 can be, for example, the user's finger sliding on the touchpad; when the cursor input device is a mouse, operation 1 can be, for example, the user moving the mouse on a plane.

[0146] In one example, the first device is a PC, the first display screen is the PC's display screen, and the second device can be an external display screen of the PC or an electronic device with a display screen, such as a projection screen, a large screen device, a tablet, a smartphone, etc.; the cursor input device can be a mouse, a touchpad, a pointing remote control, etc.

[0147] Among them, stroke 1 refers to the motion distance generated by the user using the cursor input device.

[0148] Taking a touchpad as an example of a cursor input device, stroke 1 refers to the distance a user slides on the touchpad using their finger or stylus.

[0149] Taking a mouse as the cursor input device as an example, stroke 1 refers to the distance the user travels when moving the mouse on a plane.

[0150] For example, the motion 1 may include a first offset and a second offset, wherein the first offset refers to the offset in the horizontal direction generated by the user using the cursor input device, and the second offset refers to the offset in the vertical direction generated by the user using the cursor input device.

[0151] In some embodiments, in response to a user's operation 1 on the cursor input device, the cursor input device sends a first offset and a second offset to the first device.

[0152] S703: The first device obtains the stroke 2 corresponding to the cursor based on the first display control ratio and stroke 1.

[0153] Among them, stroke 2 refers to the distance that the cursor displayed on the first display screen needs to move when the user generates stroke 1 by using the cursor input device.

[0154] For example, the motion 2 may include a third offset and a fourth offset, wherein the third offset refers to the horizontal offset of the cursor on the first display screen relative to the currently displayed coordinate position; similarly, the fourth offset refers to the vertical offset of the cursor on the first display screen relative to the currently displayed coordinate position.

[0155] In one example, the third offset is obtained based on the first offset and the first display control ratio, and the fourth offset is obtained based on the second offset and the first display control ratio.

[0156] S704: The first device controls the display of the cursor on the first display screen according to the stroke 2 corresponding to the cursor.

[0157] Specifically, the first device determines the next display coordinates of the cursor based on the current display coordinates of the cursor, the third offset, and the fourth offset, and displays the cursor on the first display screen based on the next display coordinates.

[0158] In one example, the current display coordinates of the cursor are (X1, Y1), the first offset generated by the user's operation 1 on the cursor input device is dx, the second offset is dy, the display-to-control ratio between the cursor input device and the first display screen is A, the third offset of the cursor is dx' = Adx, the fourth offset of the cursor is dy' = Ady, and the next display coordinates of the cursor are (X1+dx', Y1+dy').

[0159] S705: In response to the user's operation 2 on the cursor input device, the cursor is displayed on the second display screen of the second device.

[0160] When the cursor input device is a touchpad, operation 2 can be, for example, the user's finger sliding on the touchpad; when the cursor input device is a mouse, operation 2 can be, for example, the user moving the mouse on a flat surface.

[0161] S706: When the cursor is displayed on the second display screen, the first device determines the second display-to-control ratio between the second display screen and the cursor input device based on the information of the second display screen and the information of the cursor input device.

[0162] The explanation of the information on the second display screen and the cursor input device is as follows: Figure 6 The embodiments shown have been described in detail, and for the sake of brevity, they will not be repeated here.

[0163] S707: In response to the user's operation 3 on the cursor input device, the first device obtains the movement 3 from the cursor input device.

[0164] When the cursor input device is a touchpad, operation 3 can be, for example, the user's finger sliding on the touchpad; when the cursor input device is a mouse, operation 3 can be, for example, the user moving the mouse on a flat surface.

[0165] In one example, the first device is a PC, the first display screen is the PC's display screen, and the second display screen can be an external display screen of the PC or a display screen of other electronic devices, such as a projector screen, a large-screen device display screen, a tablet display screen, a smartphone display screen, etc.; the cursor input device can be a mouse, touchpad, directional remote control, etc.

[0166] Among them, motion stroke 3 refers to the motion stroke generated by the user using the cursor input device.

[0167] Taking a touchpad as an example of a cursor input device, stroke 3 refers to the distance a user slides on the touchpad using their finger or stylus.

[0168] Taking a mouse as the cursor input device as an example, stroke 3 refers to the travel distance of the mouse when the user operates it on a plane.

[0169] For example, the motion 3 may include a fifth offset and a sixth offset, the fifth offset being the horizontal offset generated by the user using the cursor input device, and the sixth offset being the vertical offset generated by the user using the cursor input device.

[0170] In some embodiments, in response to user operation 3 on the cursor input device, the cursor input device sends a fifth offset and a sixth offset to the first device.

[0171] S708: The first device obtains the stroke 4 corresponding to the cursor based on the second display control ratio and the stroke 3.

[0172] Among them, stroke 4 refers to the distance the cursor displayed on the second display screen needs to move when the user generates stroke 3 by using the cursor input device.

[0173] For example, the motion 4 may include a seventh offset and an eighth offset, wherein the seventh offset refers to the horizontal offset of the cursor on the second display screen relative to the currently displayed coordinate position; similarly, the eighth offset refers to the vertical offset of the cursor on the second display screen relative to the currently displayed coordinate position.

[0174] In one example, the seventh offset is obtained based on the fifth offset and the second display control ratio, and the eighth offset is obtained based on the sixth offset and the second display control ratio.

[0175] S709: The first device controls the display of the cursor on the second display screen according to the stroke 4 corresponding to the cursor.

[0176] Specifically, the first device determines the next display coordinates of the cursor based on the current display coordinates of the cursor, the seventh offset, and the eighth offset, and displays the cursor on the second display screen based on the next display coordinates.

[0177] In one example, the current display coordinates of the cursor are (X2, Y2), the fifth offset generated by the user's operation 3 on the cursor input device is dx1, the sixth offset is dy1, the display control ratio between the cursor input device and the first display screen is B, the seventh offset of the cursor is dx1' = Bdx1, the fourth offset of the cursor is dy1' = Bdy1, and the next display coordinates of the cursor are (X2+dx1', Y2+dy1').

[0178] In some embodiments, when the display screen of the first device is changed from the first display screen to the second display screen, or when the cursor is changed from being displayed on the first display screen of the first device to being displayed on the second display screen of the second device, the first device obtains information from the second display screen, determines a second display control ratio between the second display screen and the cursor input device based on the information from the second display screen and the information from the cursor input device, and controls the cursor to be displayed on the second display screen based on the second display control ratio during the process of the user controlling the cursor movement on the second display screen using the cursor input device.

[0179] In some embodiments, when the cursor input device connected to the first device is changed from cursor input device 1 to cursor input device 2, the first device obtains the information of cursor input device 2, determines the third display control ratio between the first display screen and cursor input device 2 based on the information of the first display screen and the information of cursor input device 2, and controls the cursor to be displayed on the first display screen based on the third display control ratio during the process of the user using cursor input device 2 to control the movement of the cursor on the first display screen.

[0180] In some embodiments, when the display screen of the first device is changed from the first display screen to the second display screen, or the cursor is changed from being displayed on the first display screen of the first device to being displayed on the second display screen of the second device, and the connected cursor input device is changed from cursor input device 1 to cursor input device 2, the first device obtains the information of the second display screen and the information of the cursor input device 2, determines the fourth display control ratio between the second display screen and the cursor input device 2 based on the information of the second display screen and the information of the cursor input device 2, and during the process of the user using the cursor input device 2 to control the movement of the cursor on the second display screen, the first device controls the cursor to be displayed on the second display screen based on the fourth display control ratio.

[0181] For example, Figure 8 This illustration shows a system architecture diagram of an electronic device for controlling a cursor, according to an embodiment of this application. Figure 8 As shown, the system architecture includes an application layer, a framework layer, a kernel layer, and a hardware layer.

[0182] The application layer includes one or more applications, such as one or more third-party applications, and one or more system applications; the framework layer includes the Windows Manager Service (WMS), the input device management library (libinput), the multimodal input subsystem (multimodalinput), the rendering service (render_srevice), and the hardware composer (HWC); the kernel layer includes input device drivers and display interfaces, such as the EDP (embedded display port) display interface; the hardware layer includes touchpads, mice, and monitors, such as liquid crystal displays (LCDs).

[0183] EDP ​​is a digital video interface standard designed specifically for embedded devices, primarily used to connect motherboards and built-in displays.

[0184] Specifically:

[0185] The window management service is used to update the information of the peripheral display screen when it is replaced.

[0186] Among them, the replacement of the peripheral display screen can refer to the replacement of the display screen that displays the cursor, for example, the cursor is changed from being displayed on the first display screen of the electronic device to being displayed on the second display screen that is connected to the electronic device.

[0187] In some embodiments, the information of the peripheral display screen may include the screen size information of the peripheral display screen, the resolution and / or PPI of the peripheral display screen, and other information of the peripheral display screen, which are not limited in this application.

[0188] The window management service is also used to send updated screen information to the multimodal input subsystem.

[0189] In some embodiments, the framework layer also includes a display management service for detecting changes in screen information and, when a change in screen information is detected, notifying the window management service to update the screen information.

[0190] The input device management library is used to obtain information about the cursor input device.

[0191] In some embodiments, the information of the cursor input device may include the attribute information of the cursor input device. For example, when the cursor input device is a touchpad, the information of the cursor input device may include one or more of the following: the size of the touchpad, the resolution of the touchpad, and the PPI of the touchpad.

[0192] The input device management library is also used to send the acquired cursor input device information to the multi-mode input subsystem.

[0193] The multi-mode input subsystem is used to obtain updated screen information from the window management service and to obtain information about the cursor input device from the input device management library.

[0194] The multi-mode input subsystem is also used to dynamically determine the display control ratio based on the updated screen information and the information of the cursor input device.

[0195] In some embodiments, when the cursor input device is a mouse, the multi-mode input subsystem can dynamically determine the display-to-control ratio based on the updated screen information.

[0196] For example, the multi-mode input subsystem corrects the configured display-to-control ratio based on the updated screen information, thereby dynamically determining the display-to-control ratio.

[0197] In some embodiments, when the cursor input device is a touchpad, the multi-mode input subsystem dynamically determines the display-to-control ratio based on the updated screen information and the information of the cursor input device.

[0198] For example, the multi-mode input subsystem corrects the configured display-to-control ratio based on the updated screen information and the information of the cursor input device, thereby dynamically determining the display-to-control ratio.

[0199] The multi-mode input subsystem is also used to adjust the speed of cursor movement on the display screen according to the dynamically determined display control ratio.

[0200] In some embodiments, the multi-mode input subsystem obtains the third and fourth offsets of the cursor based on the dynamically determined display-control ratio and the first and second offsets obtained from the cursor input device.

[0201] The multi-mode input subsystem is also used to determine the second coordinate position of the cursor based on the current coordinate position of the cursor, the third offset, and the fourth offset; and to send the second coordinate position to the rendering service.

[0202] The current coordinate position of the cursor refers to the current display position of the cursor, that is, the display position when the cursor's coordinates are not superimposed on the horizontal offset and the vertical offset.

[0203] A rendering service used to render the cursor layer at a second coordinate position on the display screen.

[0204] The hardware mixer is used to synthesize a cursor based on the cursor layer rendered by the rendering service and output it to the display in the hardware layer through the kernel layer's display interface.

[0205] A monitor used to display the cursor at a second coordinate position on the display screen.

[0206] For example, taking a mouse or touchpad as the cursor input device, combined with Figure 9 This application describes the interaction between the first display screen, the first device, and the cursor input device provided in the embodiments of this application.

[0207] like Figure 9 As shown, the first device includes a display management service, a window management service, an input device driver, an input device management library, a multi-mode input subsystem, a hardware mixer, a display interface, and a monitor. The first display screen can be an external display screen for the first device, and the mouse and / or touchpad are external cursor input devices for the first device. The interaction process includes:

[0208] When the display management service detects a change in screen information, such as when the display screen showing the cursor is changed from the display screen of the first device to the first display screen connected to the first device, it notifies the window management service to update the screen information.

[0209] When screen information changes, the window management service updates the information of the display screen. That is, after the display screen showing the cursor is changed to the first display screen, the updated display screen information is the information of the first display screen.

[0210] In some embodiments, the information of the first display screen may include the screen size information of the first display screen, the resolution and / or PPI of the first display screen, and other information of the first display screen, which are not limited in this application.

[0211] The window management service sends information from the first display screen to the multi-mode input subsystem.

[0212] The input device management library obtains information about the mouse or touchpad through the input device driver.

[0213] The input device management library sends the acquired mouse or touchpad information to the multi-mode input subsystem.

[0214] The multi-mode input subsystem obtains information about the first display screen from the window management service and information about the mouse or touchpad from the input device management library.

[0215] The multi-mode input subsystem dynamically determines the display-to-control ratio based on information from the first display screen and the mouse or touchpad.

[0216] The multi-mode input subsystem adjusts the cursor movement speed on the first display screen according to the dynamically determined display-control ratio.

[0217] In some embodiments, the multi-mode input subsystem obtains the third and fourth offsets of the cursor based on the dynamically determined display-to-control ratio, the first offset, and the second offset obtained from the touchpad or mouse.

[0218] For an explanation of the first offset, second offset, third offset, and fourth offset, please refer to [link to relevant documentation]. Figure 4 For the sake of brevity, the relevant descriptions in the illustrated embodiments will not be repeated here.

[0219] The multi-mode input subsystem determines the second coordinate position of the cursor based on the current coordinate position of the cursor, the third offset, and the fourth offset.

[0220] The current coordinate position of the cursor refers to the current display position of the cursor, that is, the display position when the cursor's coordinates are not superimposed on the horizontal offset and the vertical offset.

[0221] In one example, taking a touchpad as the cursor input device, the touchpad reports the first horizontal offset and the second vertical offset (dx, dy) generated by the user using the touchpad to the multi-mode input subsystem. The multi-mode input subsystem obtains information about the first display screen (e.g., display ppi) through the window management service and information about the touchpad (input_params) through the input device management library to dynamically calculate the display-to-control ratio between the touchpad and the first display screen.

[0222] The formula for calculating the display-to-control ratio in electronic devices is:

[0223] gain = f(x) (1)

[0224] The formula for determining the dynamic display control ratio based on the information from the first display screen and the touchpad is:

[0225] gain'=f(display_ppi)*f(input_params)*Gain(2)

[0226] Based on the dynamic display control ratio, the first offset, and the second offset, the formulas for obtaining the third offset of the cursor coordinates in the horizontal direction and the fourth offset of the cursor coordinates in the vertical direction (dx', dy') are as follows:

[0227] dx'=dx*gain';dy'=dx*gain'(3)

[0228] The formula for determining the cursor's second coordinate position (X2, Y2) based on the cursor's current coordinate position (X1, Y1), the third offset, and the fourth offset (dx', dy') is as follows:

[0229] X2=X1+dx';Y2=Y1+dy'(4)

[0230] The hardware mixer draws the cursor at a second coordinate position on the first display screen and outputs the drawn cursor to the display screen through the display interface.

[0231] The cursor is displayed at the second coordinate position on the first display screen.

[0232] In this embodiment, the electronic device can obtain information about the current screen and the cursor input device in real time, and dynamically adjust the display control ratio based on the information about the current screen and the cursor input device. This allows the electronic device to dynamically adjust the display control ratio when the screen displaying the cursor changes or the cursor input device is changed, so that the movement speed of the cursor on the display screen matches the peripheral scene of the electronic device, improving the responsiveness of the cursor during user control of cursor movement.

[0233] For example, Figure 10 A schematic flowchart of another method 1000 for controlling a cursor provided in an embodiment of this application is shown. Figure 10 As shown, the method 1000 includes:

[0234] S1001: When the display management service detects a change in screen information, it notifies the window management service to update the screen information.

[0235] S1002: When screen information changes, the window management service updates the information on the display screen and sends the updated information to the multi-mode input subsystem.

[0236] S1003: The input device management library obtains information about the cursor input device through the input device driver and sends the obtained information about the cursor input device to the multi-mode input subsystem.

[0237] S1004: The multi-mode input subsystem obtains the updated display screen information from the window management service, obtains the cursor input device information from the input device management library, and then determines the first display control ratio based on the updated display screen information and the cursor input device information.

[0238] S1005: The multi-mode input subsystem determines the cursor display position based on the first display control ratio.

[0239] In some embodiments, the multi-mode input subsystem adjusts the first offset and the second offset from the cursor input device according to the first display control ratio to obtain the third offset of the cursor coordinates in the horizontal direction and the fourth offset of the cursor coordinates in the vertical direction; then, the second coordinate position of the cursor is determined according to the current coordinate position of the cursor, the third offset and the fourth offset, that is, the display position of the cursor.

[0240] The current coordinate position of the cursor refers to the current display position of the cursor, that is, the display position when the cursor's coordinates are not superimposed on the horizontal offset and the vertical offset.

[0241] S1006: The display module obtains the cursor display position from the multi-mode input subsystem.

[0242] In some embodiments, the display module may include one or more of the rendering service, hardware mixer, display interface, and display described above.

[0243] S1007: The display module displays the cursor on the screen according to the cursor's position.

[0244] One or more modules or units described herein can be implemented in software, hardware, or a combination of both. When any of the above modules or units are implemented in software, the software exists as computer program instructions and is stored in memory. A processor can be used to execute the program instructions and implement the above method flow. The processor can include, but is not limited to, at least one of the following: a central processing unit (CPU), a microprocessor, a digital signal processor (DSP), a microcontroller unit (MCU), or an artificial intelligence processor, etc., and various computing devices that run software. Each computing device may include one or more cores for executing software instructions to perform calculations or processing. The processor can be built into a SoC (System-on-a-Chip) or an application-specific integrated circuit (ASIC), or it can be a separate semiconductor chip. In addition to the cores within the processor for executing software instructions to perform calculations or processing, it may further include necessary hardware accelerators, such as field-programmable gate arrays (FPGAs), PLDs (programmable logic devices), or logic circuits that implement dedicated logic operations.

[0245] When the modules or units described herein are implemented in hardware, the hardware may be any one or any combination of a CPU, microprocessor, DSP, MCU, artificial intelligence processor, ASIC, SoC, FPGA, PLD, application-specific digital circuit, hardware accelerator, or non-integrated discrete device, which may run the necessary software or perform the above method flow independently of software.

[0246] When the modules or units described herein are implemented using software, they can be implemented, in whole or in part, in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, all or part of the processes or functions described in the embodiments of this application are generated. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another. For example, the computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center via wired (e.g., coaxial cable, fiber optic, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) means. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that integrates one or more available media. The available medium can be a magnetic medium (e.g., floppy disk, hard disk, magnetic tape), an optical medium (e.g., DVD), or a semiconductor medium (e.g., solid-state disk (SSD)).

[0247] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0248] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0249] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0250] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0251] In addition, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit.

[0252] If the aforementioned functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, or a portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0253] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for controlling a cursor, characterized in that, The method is applied to a first device, the first device being connected to a second device and a cursor input device, the method comprising: The first device displays the cursor on the first display screen of the first device at a first display-to-control ratio according to the user’s first operation on the cursor input device. The first display-to-control ratio is related to the information on the first display screen and the information of the cursor input device. Responding to a second operation by the user on the cursor input device, the second operation instructing the cursor to be moved to a second display screen of the second device; The first device displays the cursor on the second display screen at a second display ratio based on the user's third operation on the cursor input device. The second display ratio is related to the information on the second display screen and the information on the cursor input device.

2. The method according to claim 1, characterized in that, The information of the first display screen includes one or more of the following: screen size, screen resolution, and pixel density unit; the information of the second display screen includes one or more of the following: screen size, screen resolution, and pixel density unit.

3. The method according to claim 1 or 2, characterized in that, When the cursor input device is a touchpad, the information of the cursor input device includes one or more of the following: the size of the touchpad, the reporting rate, the resolution, and the pixel density unit. When the cursor input device is a mouse, the information of the cursor input device includes the dots per inch (DPI) and / or the reporting rate of the mouse.

4. The method according to any one of claims 1 to 3, characterized in that, Before displaying the cursor on the first display screen of the first device at a first display-to-control ratio, the method further includes: The first device acquires information from the first display screen and information from the cursor input device; The first device determines the first display-to-control ratio based on the information from the first display screen and the information from the cursor input device.

5. The method according to any one of claims 1 to 4, characterized in that, Before displaying the cursor on the second display screen of the second device at a second display ratio, the method further includes: The first device acquires information from the second display screen and information from the cursor input device; The first device determines the second display control ratio based on the information from the second display screen and the information from the cursor input device.

6. The method according to any one of claims 1 to 5, characterized in that, The first device displays the cursor on its first display screen at a first display-to-control ratio according to a first operation performed by the user on the cursor input device, including: The first device receives a first movement sent by the second device, wherein the first movement is the movement distance generated by the cursor input device in response to the first operation; The first device determines the second stroke of the cursor based on the first display control ratio and the first stroke. The first device displays a cursor on the first display screen according to the second movement.

7. The method according to claim 6, characterized in that, The first travel includes a first offset and a second offset, wherein the first offset is the horizontal travel of the cursor input device in response to the first operation, and the second offset is the vertical travel of the cursor input device in response to the first operation; the second travel includes a third offset and a fourth offset, wherein the third offset is the horizontal travel of the cursor, and the fourth offset is the vertical travel of the cursor. The first device determines the second stroke of the cursor based on the first display control ratio and the first stroke, including: The first device determines the third offset based on the first display control ratio and the first offset; The first device determines the fourth offset based on the first display control ratio and the second offset.

8. The method according to claim 7, characterized in that, The first device displays a cursor on the first display screen according to the second stroke, including: The first device determines the second coordinate position based on the first coordinate position, the third offset, and the fourth offset, wherein the first coordinate position is the current display position of the cursor; The first device controls the first display screen to display the cursor at the second coordinate position.

9. The method according to any one of claims 1 to 8, characterized in that, The first device is a smartphone, tablet computer, or PC.

10. The method according to any one of claims 1 to 9, characterized in that, The second device is an external display screen, projector, tablet computer, PC, or smartphone.

11. The method according to any one of claims 1 to 10, characterized in that, The cursor input device is a touchpad or a mouse.

12. An electronic device, characterized in that, include: One or more processors; One or more memory units; And one or more computer programs, wherein the one or more computer programs are stored in the one or more memories, the one or more computer programs including instructions that, when executed by the one or more processors, cause the electronic device to perform the method as described in any one of claims 1 to 11.

13. A computer-readable storage medium, characterized in that, The storage medium stores a program or instructions that, when executed, implement the method as described in any one of claims 1 to 11.

14. A chip, characterized in that, The chip stores instructions that, when executed, implement the method as described in any one of claims 1 to 11.

15. A computer program product, characterized in that, The computer program product stores a program or instructions that, when executed, implement the method as described in any one of claims 1 to 11.

Images