A device remote control method, apparatus, device, and storage medium

Abstract

This application discloses a method, apparatus, device, and storage medium for remote device control. The method includes: responding to a remote control trigger operation, determining a target remote control mode based on the trigger position corresponding to the remote control trigger operation, the current default mode, and the current mouse state; if the target remote control mode is a touch screen control mode, calculating a target page scroll offset based on touch screen operation data; comparing the page scroll offset threshold corresponding to the touch screen control mode with the target page scroll offset to determine the final page scroll offset and generating a touch screen remote control command; if the target remote control mode is a mouse control mode, determining mouse operation data based on the remote control trigger operation and generating a mouse remote control command based on the mouse operation data, which can improve the flexibility of remote device control methods and the efficiency of remote control operations.

Description

Technical Field

[0001] This application belongs to the field of remote control technology, specifically relating to a method, apparatus, device, and storage medium for remote control of equipment. Background Technology

[0002] With the development of internet and smart terminal technologies, cross-device remote control has become a frequently used technology. The operational coordination between the control end and the controlled end, as well as the accuracy of command transmission, directly determine the control effect. Therefore, how to perform flexible and accurate remote control of equipment has become a key technical issue for improving the effect and experience of remote control.

[0003] In existing technologies, remote device control primarily involves setting up a virtual mouse on the display screen of the main control device to simulate computer mouse operations. The main control device receives operation commands from the virtual mouse to determine the control commands for the controlled device and then sends these commands to the controlled device. However, existing technologies suffer from inflexibility and low efficiency in remote device control. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, device, and storage medium for remote device control, which solves the problems of inflexible remote device control methods and low control efficiency in the prior art. By determining the target remote control mode through the trigger position corresponding to the remote control trigger operation, the current default mode, and the current mouse state, and by calculating the target page scroll offset based on touch screen operation data to generate touch screen remote control commands and generating mouse remote control commands based on mouse operation data, the purpose of seamless switching of control modes can be achieved, thereby improving the flexibility of remote device control methods and the efficiency of remote control operations.

[0005] In a first aspect, embodiments of this application provide a method for remotely controlling a device, the method comprising:

[0006] In response to a remote control trigger operation, obtain the current default mode and the current mouse state, and determine the target remote control mode based on the trigger position, the current default mode, and the current mouse state corresponding to the remote control trigger operation. When the target remote control mode is touch screen control mode, the touch screen operation data is determined according to the remote control trigger operation, and the target page scroll offset is calculated based on the touch screen operation data; Compare the page scroll offset threshold corresponding to the touch screen control mode with the target page scroll offset. Determine the final page scroll offset based on the comparison result and generate touch screen remote control commands based on the final page scroll offset. When the target remote control mode is mouse control mode, mouse operation data is determined based on the remote control trigger operation, and mouse remote control commands are generated based on the mouse operation data.

[0007] Furthermore, touchscreen operation data is determined based on remote control trigger operations, and the target page scroll offset is calculated based on the touchscreen operation data, including: Determine multiple operation states corresponding to the remote control trigger operation, and determine the starting position coordinates and ending position coordinates of the touch screen operation based on the multiple operation states; Calculate the target page scroll offset based on the coordinates of the starting and ending positions of the touch operation.

[0008] Furthermore, multiple operation states corresponding to the remote control trigger operation are determined, and the starting and ending coordinates of the touch screen operation are determined based on these multiple operation states, including: Identify the operation timestamp corresponding to each operation state, and determine the target touch screen operation corresponding to the remote control trigger operation based on the operation state and operation timestamp; The starting point and ending point coordinates of the target touch screen operation are determined based on the operation position coordinates corresponding to each operation state.

[0009] Furthermore, the current mouse state includes the mouse button display state; Before comparing the page scroll offset threshold corresponding to the touch screen control mode with the target page scroll offset, the method also includes: Get the actual size of the page content and the preset touch screen visible size, and determine the initial page scroll offset threshold corresponding to the touch screen control mode based on the actual size of the page content and the preset touch screen visible size; Determine the page black border coverage size corresponding to the mouse button display state, and determine the final page scroll offset threshold corresponding to the touch screen control mode based on the initial page scroll offset threshold and the page black border coverage size.

[0010] Furthermore, the target remote control mode is determined based on the trigger location corresponding to the remote control trigger operation, the current default mode, and the current mouse state, including: When the current default mode is non-mouse control mode, the mouse display area of ​​the current page is determined according to the mouse display state and mouse display mode in the current mouse state. Determine the relationship between the trigger position corresponding to the remote control trigger operation and the position of the current page mouse display area, and determine the target remote control mode based on the position relationship.

[0011] Furthermore, the current page's mouse display area includes multiple mouse button areas, and the mouse operation data includes operation position coordinates and operation type parameters; Generate remote mouse control commands based on mouse operation data, including: The target mouse button area is determined from multiple mouse button areas based on the operation position coordinates, and the target operation state corresponding to the target mouse button area is determined based on the operation type parameter. Identify the target operation event corresponding to the target operation state, and generate remote mouse control commands based on the target operation event.

[0012] Furthermore, the current page mouse display area includes a mouse button display area or a mouse icon display area, and multiple mouse button areas include a left-click area, a middle-click area, and a right-click area.

[0013] Secondly, embodiments of this application provide a remote control device for equipment, the device comprising: The control mode determination module is used to respond to remote control trigger operations, obtain the current default mode and the current mouse state, and determine the target remote control mode based on the trigger position, the current default mode and the current mouse state corresponding to the remote control trigger operation. The offset calculation module is used to determine the touch screen operation data based on the remote control trigger operation when the target remote control mode is touch screen control mode, and to calculate the target page scroll offset based on the touch screen operation data. The touch screen control command generation module is used to compare the page scroll offset threshold corresponding to the touch screen control mode with the target page scroll offset, determine the final page scroll offset based on the comparison result, and generate touch screen remote control commands based on the final page scroll offset. The mouse control command generation module is used to determine mouse operation data based on remote control trigger operations when the target remote control mode is mouse control mode, and to generate mouse remote control commands based on the mouse operation data.

[0014] Thirdly, embodiments of this application provide an electronic device including a processor, a memory, and a program or instructions stored in the memory and executable on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the method described in the first aspect.

[0015] Fourthly, embodiments of this application provide a readable storage medium on which a program or instructions are stored, which, when executed by a processor, implement the steps of the method described in the first aspect.

[0016] Fifthly, embodiments of this application also provide a computer program product comprising a computer program stored in a computer-readable storage medium, wherein at least one processor of the device reads from the computer-readable storage medium and executes the computer program, causing the device to perform the method described in the first aspect.

[0017] In this embodiment, in response to a remote control trigger operation, the current default mode and the current mouse state are obtained. The target remote control mode is determined based on the trigger position corresponding to the remote control trigger operation, the current default mode, and the current mouse state. If the target remote control mode is a touchscreen control mode, touchscreen operation data is determined based on the remote control trigger operation, and the target page scroll offset is calculated based on the touchscreen operation data. The relationship between the page scroll offset threshold corresponding to the touchscreen control mode and the target page scroll offset is compared. The final page scroll offset is determined based on the comparison result, and a touchscreen remote control command is generated based on the final page scroll offset. If the target remote control mode is a mouse control mode, mouse operation data is determined based on the remote control trigger operation, and a mouse remote control command is generated based on the mouse operation data. The above-mentioned remote control method solves the problems of inflexible remote control and low efficiency in existing technologies. By determining the target remote control mode based on the trigger position, current default mode, and current mouse state, and by calculating the target page scroll offset based on touch screen operation data to generate touch screen remote control commands and mouse remote control commands based on mouse operation data, the goal of seamless switching of control modes can be achieved, thus improving the flexibility and efficiency of remote control. Attached Figure Description

[0018] Figure 1 This is a flowchart of a device remote control method provided in an embodiment of this application; Figure 2 This is a flowchart of another device remote control method provided in the embodiments of this application; Figure 3 This is a flowchart of generating remote mouse control commands provided in an embodiment of this application; Figure 4 This is a schematic diagram of the virtual mouse page display provided in an embodiment of this application; Figure 5 This is a structural block diagram of a remote control device for an equipment provided in an embodiment of this application; Figure 6 This is a structural block diagram of the electronic device provided in the embodiments of this application. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this application clearer, specific embodiments of this application are described in detail below with reference to the accompanying drawings. It is understood that the specific embodiments described herein are merely for explaining this application and not for limiting it. Furthermore, it should be noted that, for ease of description, only the parts relevant to this application are shown in the drawings, not all of them. Before discussing exemplary embodiments in more detail, it should be mentioned that some exemplary embodiments are described as processes or methods depicted as flowcharts. Although the flowcharts describe operations (or steps) as sequential processes, many of these operations can be performed in parallel, concurrently, or simultaneously. Furthermore, the order of the operations can be rearranged. The process can be terminated when its operation is completed, but may also have additional steps not included in the drawings. The process can correspond to a method, function, procedure, subroutine, subprogram, etc.

[0020] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0021] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0022] First, this solution can be used in scenarios involving cross-device remote control, particularly in scenarios where a master mobile device can perform highly collaborative and precise cross-device remote control of a controlled mobile device or computer. By determining the target remote control mode based on the trigger position, current default mode, and current mouse state corresponding to the remote control trigger operation, and by calculating the target page scroll offset based on touchscreen operation data to generate touchscreen remote control commands and generating mouse remote control commands based on mouse operation data, seamless switching of control modes can be achieved, improving the flexibility and efficiency of remote device control. Based on the above use cases, it is understood that the executing entity for each step in this solution can be the master mobile device used for remote control. This master mobile device refers to any electronic device with data computing, processing, and storage capabilities, such as mobile phones and tablet computers; this application does not limit this.

[0023] The following description, in conjunction with the accompanying drawings, details a method, apparatus, device, and storage medium for remote device control provided in this application, through specific embodiments and application scenarios.

[0024] Figure 1 This is a flowchart of a device remote control method provided in an embodiment of this application. Figure 1 As shown, the specific steps include the following: S101, in response to a remote control trigger operation, obtains the current default mode and the current mouse state, and determines the target remote control mode based on the trigger position corresponding to the remote control trigger operation, the current default mode, and the current mouse state.

[0025] The remote control trigger operation can be clicking the remote control button on the controlled device or launching the remote control program. The current default mode can be the system's default remote control mode at the current moment. The current default mode can include mouse control mode or touchscreen control mode, and can be the initial mode set by the remote control system or a historical mode used in the last remote control. The current mouse status can indicate whether the virtual mouse used for remote control is displayed and / or its display format at the current moment. The target remote control mode can be the control mode ultimately used to remotely control the controlled device. The target remote control mode can be either mouse control mode or touchscreen control mode.

[0026] In one embodiment, upon detecting a remote control trigger operation, in response to the remote control trigger operation, the current default mode and current mouse state of the remote control system are obtained. Based on the current mouse state, it is determined whether the virtual mouse is displayed. If the trigger position corresponding to the remote control trigger operation coincides with the current virtual mouse display position, it is determined that a mouse control mode has been triggered, and the target remote control mode is determined to be the mouse control mode. If the trigger position corresponding to the remote control trigger operation does not coincide with the current virtual mouse display position, or if the virtual mouse is not displayed, it is determined that a mouse control mode has not been triggered, and the target remote control mode is determined to be the current default mode.

[0027] S10211, when the target remote control mode is touch screen control mode, determine the touch screen operation data according to the remote control trigger operation, and calculate the target page scroll offset based on the touch screen operation data.

[0028] The touchscreen operation data can be data describing the characteristics of operations such as clicking, long-pressing, and swiping on the screen. Examples include click location coordinates, long-press duration, and swipe distance. The target page scroll offset can be the distance the page needs to be scrolled on the controlled device according to the touchscreen operation data.

[0029] In one embodiment, when the target remote control mode is a touchscreen control mode, the operation purpose is determined based on changes in operation states such as pressing, releasing, swiping, and dragging during the remote control trigger operation; the operation distance is determined based on changes in the operation position during the remote control trigger operation; and touchscreen operation data is determined based on the operation purpose and operation distance. The distance to drag or swipe the controlled device page according to the touchscreen operation data is calculated to obtain the target page scroll offset.

[0030] In one embodiment, determining touch screen operation data based on a remote control trigger operation and calculating the target page scroll offset based on the touch screen operation data includes: determining multiple operation states corresponding to the remote control trigger operation; determining the touch screen operation start point coordinates and touch screen operation end point coordinates based on the multiple operation states; and calculating the target page scroll offset based on the touch screen operation start point coordinates and touch screen operation end point coordinates.

[0031] The operation states can include pressing, releasing, dragging, and swiping. The starting coordinates of a touch operation can be the coordinates of the touch operation's position on the current page at the moment the operation begins. The ending coordinates of a touch operation can be the coordinates of the touch operation's position on the current page at the moment the operation ends.

[0032] In one embodiment, multiple operation states in a remote control triggered operation can be identified. Based on these multiple operation states, the start and end positions of the same touchscreen operation can be determined, resulting in the coordinates of the touchscreen operation's start and end points. For example, a "press" operation typically marks the start of a touchscreen operation, and a "release" operation typically marks the end. The start and end point coordinates are determined based on the receiving positions of the start and end operations. The sliding distance is then calculated using these coordinates to obtain the target page scroll offset corresponding to the touchscreen operation.

[0033] This solution determines the starting and ending coordinates of the touch screen operation by using multiple operation states corresponding to the remote control trigger operation. Based on the starting and ending coordinates of the touch screen operation, the target page scroll offset is calculated, which can improve the accuracy of the page scroll offset calculation results and thus improve the precision of remote control.

[0034] In one embodiment, determining multiple operation states corresponding to a remote control triggering operation, and determining the touch screen operation start point coordinates and touch screen operation end point coordinates based on the multiple operation states, includes: identifying the operation timestamp corresponding to each operation state; determining the target touch screen operation corresponding to the remote control triggering operation based on the operation state and the operation timestamp; and determining the touch screen operation start point coordinates and touch screen operation end point coordinates of the target touch screen operation based on the operation position coordinates corresponding to each operation state.

[0035] The operation timestamp can be the time when the operation corresponding to each operation state is received. The target touch screen operation can be the final touch screen operation used to control the controlled device.

[0036] In one embodiment, the operation timestamps corresponding to each operation state can be identified. The start timestamp of the operation with the same latest operation timestamp is determined based on the termination operation state corresponding to the latest operation timestamp. The target touchscreen operation corresponding to the remote control trigger operation is determined based on the start operation timestamp and the termination operation state. The start and end coordinates of the target touchscreen operation are determined based on the operation position coordinates corresponding to each operation state and the start and end operation states of the target touchscreen operation.

[0037] This solution determines the target touch screen operation corresponding to the remote control trigger operation by using the operation status and operation timestamp. It also determines the start and end coordinates of the target touch screen operation based on the operation position coordinates corresponding to each operation status, thereby improving the accuracy of the touch screen operation position coordinate determination results.

[0038] In one embodiment, the current mouse state includes the mouse button display state; before comparing the page scroll offset threshold corresponding to the touch control mode with the target page scroll offset, the method further includes: obtaining the actual size of the page content and the preset touch screen visible size, determining the initial page scroll offset threshold corresponding to the touch control mode based on the actual size of the page content and the preset touch screen visible size; determining the page black border coverage size corresponding to the mouse button display state, and determining the final page scroll offset threshold corresponding to the touch control mode based on the initial page scroll offset threshold and the page black border coverage size.

[0039] The mouse button display status describes whether the virtual mouse is displayed as a full button. This solution, to achieve efficient switching between touchscreen control mode and mouse control mode, displays a virtual mouse on the page when the target remote control mode is touchscreen control mode, allowing for immediate invocation of mouse control mode. Furthermore, when the virtual mouse is displayed as a full button, the black border size of the page is set to prevent excessive scrolling that could cause the virtual mouse buttons to slide off the page in touchscreen control mode, thus preventing mouse operation from being unavailable. The actual page content size can be the actual length or width of the displayed content. The preset touchscreen visible size can be the maximum length or width of the visible area of ​​the main control device's screen. The initial page scroll offset threshold can be the dimensional deviation between the actual page content size and the preset touchscreen visible size. The final page scroll offset threshold can be the sum of the initial page scroll offset threshold and the black border size.

[0040] In one embodiment, the actual size of the page content and the preset visible touch screen size can be read according to the page content parameters and touch screen configuration parameters, respectively. The width difference and length difference between the actual size of the page content and the preset visible touch screen size are calculated, and the width difference and length difference are used as the initial page scroll offset threshold corresponding to the touch screen control mode. When the mouse button display state is a virtual mouse displayed in full button form, the sum of the initial page scroll offset threshold and the page black border area coverage length, and the sum of the initial page scroll offset threshold and the page black border area coverage width are calculated according to the page black border area coverage length and the page black border area coverage width corresponding to the preset page black border coverage size, to obtain the final page scroll offset threshold corresponding to the touch screen control mode. When the mouse button display state is not a virtual mouse displayed in full button form, the page black border coverage size is determined to be 0.

[0041] This solution determines the initial page scroll offset threshold corresponding to the touch screen control mode by using the actual size of the page content and the preset visible size of the touch screen. It then determines the final page scroll offset threshold corresponding to the touch screen control mode based on the initial page scroll offset threshold and the size of the black border area covering the page. This avoids the problem of the mouse control mode not being triggered when the mouse scrolls outside the page, thus improving the flexibility of remote control.

[0042] S10212, compare the page scroll offset threshold corresponding to the touch screen control mode with the target page scroll offset, determine the final page scroll offset based on the comparison result, and generate touch screen remote control commands based on the final page scroll offset.

[0043] The page scroll offset threshold can be the maximum distance the page on the controlling device is allowed to scroll. The final page scroll offset can be the final distance at which the page on the controlled device can be scrolled. Touchscreen remote control commands can be commands that perform control operations on the controlled device according to the touchscreen control mode. Touchscreen remote control commands are used to remotely control another mobile device or touchscreen device. Users can directly perform touch operations on the screen, and the operation logic is consistent with the habits of daily mobile phone use. The touchscreen control mode has advantages such as intuitive and consistent operation, seamless support for gesture operation, higher operation efficiency, lower cognitive load, and better gaming experience.

[0044] In one embodiment, the page scroll offset threshold corresponding to the touch screen control mode can be compared with the target page scroll offset. If the page scroll offset threshold is greater than or equal to the target page scroll offset, it indicates that the target page scroll offset is within the allowed page scroll offset range of the main control device, and in this case, the target page scroll offset can be directly used as the final page scroll offset. If the page scroll offset threshold is less than the target page scroll offset, it indicates that the target page scroll offset exceeds the allowed page scroll offset range of the main control device. In this case, to avoid blank pages on the main control device, the page scroll offset threshold can be directly used as the final page scroll offset. The final page scroll offset is then added to the remote control command to generate the touch screen remote control command.

[0045] S1022, when the target remote control mode is mouse control mode, determine the mouse operation data according to the remote control trigger operation, and generate mouse remote control instructions based on the mouse operation data.

[0046] Mouse operation data can represent actions such as clicking and sliding on the displayed virtual mouse. Remote mouse control commands can be instructions to control the controlled device according to mouse control modes. The mouse control mode simulates desktop computer mouse operations; sliding the user's mouse on the controlling device's screen is equivalent to moving the desktop computer's mouse pointer, clicking is equivalent to the left mouse button, and holding down is equivalent to the right mouse button. Remote mouse control commands are suitable for remotely controlling desktop operating systems such as Windows, macOS, and Linux, as well as scenarios requiring precise operations using the right-click menu and scroll wheel (such as graphic design, programming, and fine-tuning window size).

[0047] In one embodiment, when the target remote control mode is mouse control mode, the operation button for the virtual mouse is determined according to the operation position in the remote control trigger operation, the operation purpose is determined according to the function of the operation button, the mouse operation data is determined according to the operation purpose, and the mouse operation data is added to the remote control command to generate the mouse remote control command.

[0048] The technical solution provided in this application, in response to a remote control trigger operation, obtains the current default mode and the current mouse state, and determines the target remote control mode based on the trigger position corresponding to the remote control trigger operation, the current default mode, and the current mouse state; if the target remote control mode is a touch screen control mode, determines touch screen operation data based on the remote control trigger operation, and calculates the target page scroll offset based on the touch screen operation data; compares the page scroll offset threshold corresponding to the touch screen control mode with the target page scroll offset, determines the final page scroll offset based on the comparison result, and generates a touch screen remote control command based on the final page scroll offset; if the target remote control mode is a mouse control mode, determines mouse operation data based on the remote control trigger operation, and generates a mouse remote control command based on the mouse operation data. The above-mentioned remote control method solves the problems of inflexible remote control and low efficiency in existing technologies. By determining the target remote control mode based on the trigger position, current default mode, and current mouse state, and by calculating the target page scroll offset based on touch screen operation data to generate touch screen remote control commands and mouse remote control commands based on mouse operation data, the goal of seamless switching of control modes can be achieved, thus improving the flexibility and efficiency of remote control.

[0049] Figure 2 This is a flowchart of another device remote control method provided in an embodiment of this application. For example... Figure 2 As shown, the specific steps include the following: S201, in response to a remote control trigger operation, obtain the current default mode and the current mouse state. If the current default mode is a non-mouse control mode, determine the current page mouse display area based on the mouse display state and mouse display mode in the current mouse state.

[0050] The mouse display status indicates whether the virtual mouse is displayed on the main control device screen. The mouse display mode indicates the display format of the virtual mouse. The mouse display mode can include the mouse cursor indicator mode and the full button display mode. The current page mouse display area can be the display area of ​​the virtual mouse icon or the full mouse button on the current page.

[0051] In one embodiment, in response to a remote control trigger operation, the current default mode and current mouse state of the remote control system can be obtained, and it can be identified whether the current default mode is a mouse control mode. If the current default mode is not a mouse control mode, it can be determined whether there is a mouse display area on the current page based on the mouse display state in the current mouse state, and the mouse display area on the current page can be determined based on the coverage of the mouse display area corresponding to the mouse display mode.

[0052] S202, determine the relationship between the trigger position corresponding to the remote control trigger operation and the position of the current page mouse display area, and determine the target remote control mode based on the position relationship.

[0053] In one embodiment, the trigger position coordinates of the remote control trigger operation on the current page and the area coordinate range of the mouse display area on the current page can be identified respectively. Based on the trigger position coordinates and the area coordinate range, it is determined whether the trigger position corresponding to the remote control trigger operation is inside the mouse display area of ​​the current page. If the trigger position is inside the mouse display area of ​​the current page, the target remote control mode is determined to be the mouse control mode. If the trigger position is outside the mouse display area of ​​the current page, the target remote control mode is determined to be the touch screen control mode.

[0054] S20311, when the target remote control mode is touch screen control mode, determine the touch screen operation data according to the remote control trigger operation, and calculate the target page scroll offset based on the touch screen operation data.

[0055] S20312, compare the page scroll offset threshold corresponding to the touch screen control mode with the target page scroll offset, determine the final page scroll offset based on the comparison result, and generate touch screen remote control commands based on the final page scroll offset. S2032, when the target remote control mode is mouse control mode, determine the mouse operation data according to the remote control trigger operation, and generate mouse remote control instructions based on the mouse operation data.

[0056] The technical solution provided in this application determines the current page mouse display area by the mouse display state and mouse display mode in the current mouse state, judges the positional relationship between the trigger position corresponding to the remote control trigger operation and the current page mouse display area, and determines the target remote control mode. This can achieve the purpose of determining the target remote control mode based on the mouse display area and the trigger operation position, further improving the accuracy of the target control mode determination result, realizing seamless switching between touch screen control mode and mouse control mode, and improving the flexibility of remote control.

[0057] Figure 3 This is a flowchart illustrating the generation of remote mouse control commands provided in an embodiment of this application. For example... Figure 3 As shown, the current page's mouse display area includes multiple mouse button areas, and the mouse operation data includes operation position coordinates and operation type parameters, specifically including the following steps: S301, determine the target mouse button area among multiple mouse button areas based on the operation position coordinates, and determine the target operation state corresponding to the target mouse button area based on the operation type parameter.

[0058] The mouse button area can be any area that the user can interact with, corresponding to the specific functions of each mouse button. The operation type parameter can be a parameter describing the corresponding gesture category, such as single-click, double-click, long-press, and drag. The operation state can be a parameter describing the working state of the button, such as pressed-triggered, released-triggered, continuously triggered, and continuously active. The target mouse button area can be the mouse button area where the operation position coordinates are located.

[0059] Figure 4 This is a schematic diagram of a virtual mouse page display provided in an embodiment of this application. For example... Figure 4 As shown in the diagram, the current page display area M of the main control device, the black border area F, and the virtual mouse K. The virtual mouse K includes a left-click area A, a middle-click area B, and a right-click area C. The target mouse button area is determined by identifying the coordinates of the operation position and the corresponding coordinate range of each mouse button area, and then the operation purpose is determined based on the corresponding function of the target mouse button area.

[0060] In one embodiment, a target mouse button area can be determined among multiple mouse button areas based on the operation position coordinates and the area boundary coordinates of each mouse button area, and the operation to be performed on the target mouse button area can be determined based on the operation type parameter to obtain the target operation state corresponding to the target mouse button area.

[0061] S302, determine the target operation event corresponding to the target operation state, and generate remote mouse control instructions based on the target operation event.

[0062] The target operation event can be a mouse operation event that controls the controlled device. For example, double-clicking the left mouse button to open a target file or target program.

[0063] In one embodiment, the operation purpose can be determined based on the target operation state, the target operation event for controlling the controlled device can be determined based on the operation purpose and the function corresponding to the target mouse button area, and the target operation event can be added to the remote control command to generate the mouse remote control command.

[0064] In one embodiment, the current page mouse display area includes a mouse button display area or a mouse icon display area, and the multiple mouse button areas include a left button area, a middle button area, and a right button area.

[0065] The mouse button display area can be the area covered by the complete mouse buttons on the page. The mouse icon display area can be the area covered by the mouse cursor on the page. The left-click area is used to receive actions performed on the virtual left mouse button. The middle-click area is used to receive actions performed on the virtual middle mouse button. The right-click area is used to receive actions performed on the virtual right mouse button.

[0066] This solution improves the flexibility of virtual mouse display by setting a mouse button display area or a mouse icon display area in the current page's mouse display area. During remote control, users can trigger mouse control mode by clicking on the mouse button display area or the mouse icon display area, achieving seamless switching between touch control mode and mouse control mode. Furthermore, by setting left-click display area, middle-click display area, and right-click display area, the solution can achieve the goal of full mouse functionality and precise mouse control, further improving the coordination and accuracy of remote control.

[0067] The technical solution provided in this application determines the target mouse button area by operating position coordinates, determines the target operation event corresponding to the target mouse button area according to operation type parameters, and generates remote mouse control commands based on the target operation event. This enables precise operation of the controlled device by the virtual mouse, improving the efficiency and accuracy of remote control.

[0068] Figure 5 This is a structural block diagram of a remote control device for an equipment provided in an embodiment of this application. For example... Figure 5 As shown, it specifically includes the following: The control mode determination module 501 is used to respond to the remote control trigger operation, obtain the current default mode and the current mouse state, and determine the target remote control mode according to the trigger position, the current default mode and the current mouse state corresponding to the remote control trigger operation. The offset calculation module 502 is used to determine the touch screen operation data based on the remote control trigger operation when the target remote control mode is the touch screen control mode, and to calculate the target page scroll offset based on the touch screen operation data. The touch screen control command generation module 503 is used to compare the size relationship between the page scroll offset threshold corresponding to the touch screen control mode and the target page scroll offset, determine the final page scroll offset based on the size relationship comparison result, and generate touch screen remote control commands based on the final page scroll offset. The mouse control command generation module 504 is used to determine mouse operation data based on the remote control trigger operation when the target remote control mode is mouse control mode, and generate mouse remote control commands based on the mouse operation data.

[0069] Furthermore, the offset calculation module 502 is specifically used for: Determine multiple operation states corresponding to the remote control trigger operation, and determine the starting position coordinates and ending position coordinates of the touch screen operation based on the multiple operation states; Calculate the target page scroll offset based on the coordinates of the starting and ending positions of the touch operation.

[0070] Furthermore, the offset calculation module 502 is specifically used for: Identify the operation timestamp corresponding to each operation state, and determine the target touch screen operation corresponding to the remote control trigger operation based on the operation state and operation timestamp; The starting point and ending point coordinates of the target touch screen operation are determined based on the operation position coordinates corresponding to each operation state.

[0071] Furthermore, the current mouse state includes the mouse button display state; The device also includes: The initial scroll offset threshold determination module is used to obtain the actual size of the page content and the preset touch screen visible size, and determine the initial page scroll offset threshold corresponding to the touch screen control mode based on the actual size of the page content and the preset touch screen visible size. The final scroll offset threshold determination module is used to determine the page black border coverage size corresponding to the mouse button display state, and to determine the final page scroll offset threshold corresponding to the touch screen control mode based on the initial page scroll offset threshold and the page black border coverage size.

[0072] Furthermore, the control mode determination module 501 is specifically used for: When the current default mode is non-mouse control mode, the mouse display area of ​​the current page is determined according to the mouse display state and mouse display mode in the current mouse state. Determine the relationship between the trigger position corresponding to the remote control trigger operation and the position of the current page mouse display area, and determine the target remote control mode based on the position relationship.

[0073] Furthermore, the current page's mouse display area includes multiple mouse button areas, and the mouse operation data includes operation position coordinates and operation type parameters; Mouse control command generation module 504 is specifically used for: The target mouse button area is determined from multiple mouse button areas based on the operation position coordinates, and the target operation state corresponding to the target mouse button area is determined based on the operation type parameter. Identify the target operation event corresponding to the target operation state, and generate remote mouse control commands based on the target operation event.

[0074] Furthermore, the current page mouse display area includes a mouse button display area or a mouse icon display area, and multiple mouse button areas include a left-click area, a middle-click area, and a right-click area.

[0075] The technical solution provided in this application includes a control mode determination module, used to respond to a remote control trigger operation, obtain the current default mode and the current mouse state, and determine the target remote control mode based on the trigger position corresponding to the remote control trigger operation, the current default mode, and the current mouse state; an offset calculation module, used to determine touch screen operation data based on the remote control trigger operation when the target remote control mode is a touch screen control mode, and calculate the target page scroll offset based on the touch screen operation data; a touch screen control command generation module, used to compare the page scroll offset threshold corresponding to the touch screen control mode with the target page scroll offset, determine the final page scroll offset based on the comparison result, and generate a touch screen remote control command based on the final page scroll offset; and a mouse control command generation module, used to determine mouse operation data based on the remote control trigger operation when the target remote control mode is a mouse control mode, and generate a mouse remote control command based on the mouse operation data. The aforementioned device remote control system solves the problems of inflexible remote control methods and low control efficiency in existing technologies. By determining the target remote control mode based on the trigger position, current default mode, and current mouse state corresponding to the remote control trigger operation, and generating touch screen remote control commands based on touch screen operation data and mouse remote control commands based on mouse operation data, the system can achieve seamless switching of control modes, thereby improving the flexibility and efficiency of remote control methods.

[0076] The device remote control system described in this application embodiment can be configured in a device, or in a component, integrated circuit, or chip within a terminal. The system can be configured in mobile electronic devices or non-mobile electronic devices. For example, mobile electronic devices can be mobile phones, tablets, laptops, PDAs, in-vehicle electronic devices, wearable devices, ultra-mobile personal computers (UMPCs), netbooks, or personal digital assistants (PDAs), etc., while non-mobile electronic devices can be servers, network-attached storage (NAS), personal computers (PCs), televisions (TVs), ATMs, or self-service machines, etc. This application embodiment does not impose specific limitations.

[0077] The device remote control system in this application embodiment can be an operating system. This operating system can be Android, iOS, or other possible operating systems; this application embodiment does not specifically limit it.

[0078] The device remote control system provided in this application can realize the various processes implemented in the above method embodiments. To avoid repetition, it will not be described again here.

[0079] like Figure 6 As shown, this application embodiment also provides an electronic device 600, including a processor 601, a memory 602, and a program or instructions stored in the memory 602 and executable on the processor 601. When the program or instructions are executed by the processor 601, they implement the various processes of the above-described embodiment of a device remote control method and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0080] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0081] This application also provides a readable storage medium storing a program or instructions. When the program or instructions are executed by a processor, they implement the various processes of the above-described embodiment of a remote device control method and achieve the same technical effect. To avoid repetition, they will not be described again here.

[0082] The processor is the processor in the electronic device described in the above embodiments. The readable storage medium includes computer-readable storage media, such as computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0083] This application also provides a program product including program code. When the program product is run on a computer device, the program code causes the computer device to perform the steps of the methods described above according to various exemplary embodiments of this application. For example, the computer device can execute a device remote control method described in the embodiments of this application. The program product can be implemented using any combination of one or more readable media.

[0084] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element. Furthermore, it should be noted that the scope of the methods and systems in the embodiments of this application is not limited to performing functions in the order shown or discussed, but may also include performing functions substantially simultaneously or in the reverse order, depending on the functions involved. For example, the described methods may be performed in a different order than described, and various steps may be added, omitted, or combined. Additionally, features described with reference to certain examples may be combined in other examples.

[0085] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, in essence, or the part that contributes to the prior art, can be embodied in the form of a computer software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions to cause a terminal (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of this application.

[0086] The embodiments of this application have been described above with reference to the accompanying drawings. However, this application is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of this application without departing from the spirit and scope of the claims, and all of these forms are within the protection scope of this application.

[0087] The above description is merely a preferred embodiment and the technical principles employed in this application. This application is not limited to the specific embodiments described herein, and various obvious changes, readjustments, and substitutions that can be made by those skilled in the art will not depart from the scope of protection of this application. Therefore, although this application has been described in detail through the above embodiments, this application is not limited to the above embodiments, and may include more other equivalent embodiments without departing from the concept of this application, the scope of which is determined by the scope of the claims.

Claims

1. A method for remote control of equipment, characterized in that, The method includes: In response to a remote control trigger operation, the current default mode and the current mouse state are obtained, and the target remote control mode is determined based on the trigger position corresponding to the remote control trigger operation, the current default mode, and the current mouse state; When the target remote control mode is a touch screen control mode, touch screen operation data is determined according to the remote control trigger operation, and the target page scroll offset is calculated based on the touch screen operation data; Compare the page scroll offset threshold corresponding to the touch screen control mode with the target page scroll offset, determine the final page scroll offset based on the comparison result, and generate a touch screen remote control command based on the final page scroll offset. When the target remote control mode is mouse control mode, mouse operation data is determined according to the remote control trigger operation, and mouse remote control instructions are generated based on the mouse operation data.

2. The remote control method for equipment according to claim 1, characterized in that, The step of determining touch screen operation data based on the remote control trigger operation and calculating the target page scroll offset based on the touch screen operation data includes: Determine multiple operation states corresponding to the remote control trigger operation, and determine the touch screen operation start position coordinates and touch screen operation end position coordinates based on the multiple operation states; The target page scroll offset is calculated based on the coordinates of the starting and ending positions of the touch operation.

3. The remote control method for equipment according to claim 2, characterized in that, The process of determining multiple operation states corresponding to the remote control trigger operation, and determining the touch screen operation start point coordinates and touch screen operation end point coordinates based on the multiple operation states, includes: Identify the operation timestamps corresponding to each operation state, and determine the target touch screen operation corresponding to the remote control trigger operation based on the operation state and the operation timestamps; The starting point coordinates and ending point coordinates of the target touch screen operation are determined based on the operation position coordinates corresponding to each operation state.

4. The remote control method for equipment according to claim 1, characterized in that, The current mouse state includes the mouse button display state; Before comparing the page scroll offset threshold corresponding to the touch screen control mode with the target page scroll offset, the method further includes: Obtain the actual size of the page content and the preset touch screen visible size, and determine the initial page scroll offset threshold corresponding to the touch screen control mode based on the actual size of the page content and the preset touch screen visible size; Determine the page black border coverage size corresponding to the mouse button display state, and determine the final page scroll offset threshold corresponding to the touch screen control mode based on the initial page scroll offset threshold and the page black border coverage size.

5. The remote control method for equipment according to claim 1, characterized in that, The step of determining the target remote control mode based on the trigger position corresponding to the remote control trigger operation, the current default mode, and the current mouse state includes: When the current default mode is non-mouse control mode, the current page mouse display area is determined according to the mouse display state and mouse display mode in the current mouse state; Determine the relationship between the trigger position corresponding to the remote control trigger operation and the position of the current page mouse display area, and determine the target remote control mode based on the position relationship.

6. The remote control method for equipment according to claim 5, characterized in that, The current page mouse display area includes multiple mouse button areas, and the mouse operation data includes operation position coordinates and operation type parameters; The process of generating remote mouse control commands based on the mouse operation data includes: The target mouse button area is determined based on the operation position coordinates, and the target operation state corresponding to the target mouse button area is determined based on the operation type parameter. Identify the target operation event corresponding to the target operation state, and generate remote mouse control commands based on the target operation event.

7. The remote control method for equipment according to claim 6, characterized in that, The current page mouse display area includes a mouse button display area or a mouse icon display area, and the multiple mouse button areas include a left button area, a middle button area, and a right button area.

8. A remote control device for equipment, characterized in that, The device includes: The control mode determination module is used to respond to a remote control trigger operation, obtain the current default mode and the current mouse state, and determine the target remote control mode based on the trigger position corresponding to the remote control trigger operation, the current default mode and the current mouse state; The offset calculation module is used to determine touch screen operation data based on the remote control trigger operation when the target remote control mode is touch screen control mode, and to calculate the target page scroll offset based on the touch screen operation data. The touch screen control command generation module is used to compare the size relationship between the page scroll offset threshold corresponding to the touch screen control mode and the target page scroll offset, determine the final page scroll offset based on the size relationship comparison result, and generate touch screen remote control commands based on the final page scroll offset. A mouse control command generation module is used to determine mouse operation data based on the remote control trigger operation when the target remote control mode is mouse control mode, and generate mouse remote control commands based on the mouse operation data.

9. An electronic device, characterized in that, It includes a processor, a memory, and a program or instructions stored in the memory and running on the processor, wherein the program or instructions, when executed by the processor, implement the steps of the device remote control method as described in any one of claims 1-7.

10. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the device remote control method as described in any one of claims 1-7.

Images