Touch screen pointer control method and device

Abstract

The present invention discloses a control method and device for a touch screen pointer, the method comprising: obtaining in real time the pointer dragging operation data performed by a user on an air operation device; displaying a pointer moving in response to the pointer dragging operation on the touch screen; the pointer dragging operation data including the pointer dragging movement amount, pointer dragging direction and pointer dragging time interval of each frame in the dragging operation process recorded frame by frame; after the dragging is released, the pointer dragging movement amount is reversely displaced based on the pointer dragging time interval and pointer dragging direction for each frame of data in the order or reverse order of the frame records in the pointer dragging operation data. The present invention is used to improve the control efficiency of the touch screen and improve the user experience.

Description

Technical Field

[0001] The present invention relates to the field of mobile terminal touch technology, and in particular to a control method and device for a touch screen pointer. Background Art

[0002] This section is intended to provide a background or context for the presented embodiments of the invention. No description herein is admitted to be prior art by virtue of its inclusion in this section.

[0003] With the development of computer technology, the user interface interaction mode has evolved from the "pointer" system of the PC era to the "touch" system of the mobile Internet era. Traditional computer operating systems are designed based on the pointer system, with the mouse as the main control carrier, providing a high-precision control experience. However, after entering the mobile Internet era, general mobile phone operating systems are optimized for touch control. Their characteristics are that interface elements (such as icons and fonts) occupy a large proportion of the screen, and most operations can be completed directly with fingers.

[0004] In recent years, more and more mobile screen devices (such as mobile phones, tablets, car screens, educational tablets, etc.) use mobile operating systems and appear in scenarios where users cannot directly touch the screen. In these cases, it is usually necessary to use air operation devices (such as touchpads, air mice, etc.) for remote control. The operation mode of such devices is different from traditional touch operations, especially in terms of sliding and dragging operations. Users need to frequently move the pointer from one position to another, and then manually move the pointer back to the starting position, which significantly reduces the operation efficiency and affects the user experience.

[0005] The main problem currently exists is that when using the air-operated device without a touch screen, the user needs to frequently manually move the pointer from the end point back to the starting point when operating the touch-optimized interface through the pointer control system.

[0006] 1. Low operational efficiency: When users need to slide or drag from point A to point B multiple times, they need to manually move the pointer from point B back to point A after each operation, which increases the user's repetitive actions and reduces operational efficiency.

[0007] 2. Poor user experience: Since additional manual operations are required to return the pointer, the experience of operating the device in the air is not as intuitive and natural as direct touch.

[0008] 3. Insufficient cross-platform compatibility: Existing solutions may not be applicable to all operating systems and devices, limiting the widespread use of touch screen control solutions. Summary of the invention

[0009] An embodiment of the present invention provides a method for controlling a touch screen pointer, which is used to solve the problem of low efficiency in manually moving the pointer back, improve the control efficiency of the touch screen, improve the user experience, and improve cross-platform compatibility. The method includes:

[0010] Real-time acquisition of pointer dragging operation data performed by a user on an air operation device; the touch screen displays a pointer moving in response to the pointer dragging operation; the pointer dragging operation data includes the pointer dragging movement amount, pointer dragging direction and pointer dragging time interval recorded frame by frame during the dragging operation;

[0011] After the drag is released, the data of each frame is reversely displaced by the pointer drag movement amount based on the pointer drag time interval and the pointer drag direction in the order or reverse order of the frame records in the pointer drag operation data.

[0012] The embodiment of the present invention further provides a control device for a touch screen pointer, which is used to solve the problem of low efficiency in manually moving the pointer back, improve the control efficiency of the touch screen, improve the user experience, and improve cross-platform compatibility. The device includes:

[0013] The pointer drag operation data acquisition module is used to acquire the pointer drag operation data performed by the user on the air operation device in real time; the touch screen displays the pointer moving in response to the pointer drag operation; the pointer drag operation data includes the pointer drag movement amount, pointer drag direction and pointer drag time interval of each frame in the drag operation process recorded frame by frame;

[0014] The reverse displacement module is used to reversely displace the pointer drag movement amount for each frame of data based on the pointer drag time interval and the pointer drag direction in the order or reverse order of the frame records in the pointer drag operation data after the drag is released.

[0015] An embodiment of the present invention further provides a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the above-mentioned touch screen pointer control method when executing the computer program.

[0016] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned touch screen pointer control method is implemented.

[0017] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the above-mentioned touch screen pointer control method is implemented.

[0018] In an embodiment of the present invention, pointer dragging operation data performed by a user on an aerial operating device is acquired in real time; the touch screen displays a pointer that moves in response to the pointer dragging operation; the pointer dragging operation data includes the pointer dragging movement amount, pointer dragging direction, and pointer dragging time interval of each frame in the dragging operation process recorded frame by frame; after the drag is released, the pointer dragging movement amount is reversely displaced based on the pointer dragging time interval and pointer dragging direction for each frame of data in the order or reverse order of the frame records in the pointer dragging operation data. The embodiment of the present invention can completely and accurately record the user's operation trajectory by acquiring detailed data of each frame when the user performs a pointer drag operation on the air operation device in real time, including the pointer drag movement amount, direction and time interval; when the user completes the drag operation, a reverse displacement operation is performed based on the recorded data, without the need for the user to manually return the pointer, thereby avoiding tedious manual return operations and directly improving the operation efficiency; the reverse displacement of the pointer drag movement amount is performed in the order or reverse order of the frame records in the pointer drag operation data, simulating a smooth experience similar to direct touch, improving the user's satisfaction with using the air operation device to control the touch screen in a non-touch screen scenario, and significantly improving the user experience; by focusing on the relative position change of the pointer during the operation process without relying on the absolute position information of the device, a unified pointer control logic can be implemented on different operating systems and various types of mobile screen devices, providing a cross-platform, highly compatible solution, and meeting the pointer control needs of diverse devices in non-touch operation scenarios. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the prior art descriptions. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work. In the drawings:

[0020] Figure 1 A schematic diagram of a flow chart of a method for controlling a touch screen pointer in an embodiment of the present invention;

[0021] Figure 2 A specific example diagram of a method for controlling a touch screen pointer in an embodiment of the present invention;

[0022] Figure 3 A specific example diagram of a method for controlling a touch screen pointer in an embodiment of the present invention;

[0023] Figure 4 A specific example diagram of a method for controlling a touch screen pointer in an embodiment of the present invention;

[0024] Figure 5 A specific example diagram of a method for controlling a touch screen pointer in an embodiment of the present invention;

[0025] Figure 6 A specific example diagram of a method for controlling a touch screen pointer in an embodiment of the present invention;

[0026] Figure 7 A specific example diagram of a sliding recording stage in a method for controlling a touch screen pointer in an embodiment of the present invention;

[0027] Figure 8 A specific example diagram of a data processing phase and a rebound execution phase in a method for controlling a touch screen pointer in an embodiment of the present invention;

[0028] Fig. 9 It is a schematic diagram of the structure of a control device for a touch screen pointer in an embodiment of the present invention;

[0029] Fig.10 Schematic diagram of a computer device for controlling a touch screen pointer in an embodiment of the present invention. DETAILED DESCRIPTION

[0030] To make the purpose, technical solution and advantages of the embodiments of the present invention more clear, the embodiments of the present invention are further described in detail below in conjunction with the accompanying drawings. Here, the exemplary embodiments of the present invention and their descriptions are used to explain the present invention, but are not intended to limit the present invention.

[0031] The term "and / or" herein only describes an association relationship, indicating that three relationships may exist. For example, A and / or B may represent the following three situations: A exists alone, A and B exist at the same time, and B exists alone. In addition, the term "at least one" herein represents any combination of at least two of any one or more of a plurality of. For example, including at least one of A, B, and C may represent including any one or more elements selected from the set consisting of A, B, and C.

[0032] In the description of this specification, the terms "include", "including", "have", "contain", etc. are all open terms, which mean including but not limited to. The descriptions with reference to the terms "one embodiment", "a specific embodiment", "some embodiments", "for example", etc. mean that the specific features, structures or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present application. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment or example. Moreover, the specific features, structures or characteristics described may be combined in a suitable manner in any one or more embodiments or examples. The order of steps involved in each embodiment is used to schematically illustrate the implementation of the present application, and the order of steps is not limited and can be appropriately adjusted as needed.

[0033] The acquisition, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of relevant laws and regulations. The information collected in this application is information and data authorized by the user or fully authorized by all parties, and the collection, storage, use, processing, transmission, provision, disclosure, and application of relevant data are in compliance with relevant laws, regulations, and standards, and necessary confidentiality measures are taken, which do not violate public order and good customs, and provide corresponding operation entrances for users to choose to authorize or refuse. In addition, this application provides users with corresponding operation entrances for users to choose to agree or refuse the results of automated decision-making. If the user chooses to refuse, they can enter the expert decision-making process.

[0034] It should be noted that in the embodiments of the present application, some existing solutions in the industry such as certain software, components, models, etc. may be mentioned. For example, some existing software tools, components, algorithm models or other well-known solutions in other technical fields may be cited. They should be considered as exemplary, and their purpose is only to illustrate the feasibility of the implementation of the technical solution of the present application. These references should be understood as typical examples, and their core purpose is to explain and verify the rationality and feasibility of the implementation of the technical solution proposed in the present application. However, it does not mean that the applicant has or will inevitably use the solution. Such references do not imply that the applicant has actually adopted these existing solutions, or will inevitably adopt these methods in the future during its technical implementation. In other words, these references only serve to illustrate the nature of the description, to help understand the connection and transcendence between the innovation of the present application and the prior art, and do not constitute an endorsement or reliance statement on a specific prior art product.

[0035] In the history of computer technology development, the user interface interaction method has evolved from the "pointer" system of the PC era to the "touch" system of the mobile Internet era. The computer operating system in the PC era is based on the pointer system design, with the mouse as the main control carrier to achieve high-precision control. The mobile phone operating system in the mobile Internet era is optimized for touch control, with the characteristics of interface elements (such as icons and fonts) occupying a large proportion of the screen, and most operations can be completed directly with fingers.

[0036] However, with the development of technology, more and more mobile screen devices (such as mobile phones, tablets, car screens, educational tablets, etc.) using mobile operating systems are appearing in scenarios where users cannot directly touch the screen. In this case, an air operation device (such as a touchpad, air mouse, etc.) is required for remote control. In this type of system, horizontal and vertical sliding are both high-frequency operations of users. In these scenarios, the existing pointer system has an obvious defect: when the user needs to perform sliding or dragging operations from point A to point B multiple times, the pointer needs to be manually moved from point B back to point A after each operation, which greatly reduces the operation efficiency.

[0037] In order to solve the above problems, an embodiment of the present invention provides a touch screen pointer control method to solve the problem of low efficiency of manually moving the pointer back, improve the control efficiency of the touch screen, improve the user experience, and enhance cross-platform compatibility. Figure 1 , Figure 1 The following is a flow chart of a method for controlling a touch screen pointer according to an embodiment of the present invention. The method may include:

[0038] Step 101: acquiring in real time the pointer dragging operation data performed by the user on the air operation device; the touch screen displays the pointer moving in response to the pointer dragging operation; the pointer dragging operation data includes the pointer dragging movement amount, pointer dragging direction and pointer dragging time interval of each frame in the dragging operation process recorded frame by frame;

[0039] Step 102: after the drag is released, the data of each frame is reversely displaced based on the pointer drag time interval and the pointer drag direction in the order or reverse order of the frame records in the pointer drag operation data.

[0040] In an embodiment of the present invention, pointer dragging operation data performed by a user on an aerial operating device is acquired in real time; the touch screen displays a pointer that moves in response to the pointer dragging operation; the pointer dragging operation data includes the pointer dragging movement amount, pointer dragging direction, and pointer dragging time interval of each frame in the dragging operation process recorded frame by frame; and the pointer dragging movement amount is reversely displaced for each frame of data based on the pointer dragging time interval and pointer dragging direction in the order or reverse order of the frame records in the pointer dragging operation data. The embodiment of the present invention can completely and accurately record the user's operation trajectory by acquiring detailed data of each frame when the user performs a pointer drag operation on the air operation device in real time, including the pointer drag movement amount, direction and time interval; when the user completes the drag operation, a reverse displacement operation is performed based on the recorded data, without the need for the user to manually return the pointer, thereby avoiding tedious manual return operations and directly improving the operation efficiency; the reverse displacement of the pointer drag movement amount is performed in the order or reverse order of the frame records in the pointer drag operation data, simulating a smooth experience similar to direct touch, improving the user's satisfaction with using the air operation device to control the touch screen in a non-touch screen scenario, and significantly improving the user experience; by focusing on the relative position change of the pointer during the operation process without relying on the absolute position information of the device, a unified pointer control logic can be implemented in different operating systems and various types of mobile screen devices, and automatic rebound control of the pointer can be implemented according to the set rebound mode association relationship, thereby providing a cross-platform, highly compatible solution and meeting the pointer control requirements of diverse devices in non-touch operation scenarios.

[0041] During the specific implementation, step 101 is first performed: the pointer dragging operation data performed by the user on the air operating device is acquired in real time; the pointer moving in response to the pointer dragging operation is displayed on the touch screen; the pointer dragging operation data includes the pointer dragging movement amount, pointer dragging direction and pointer dragging time interval of each frame in the dragging operation process recorded frame by frame.

[0042] In the embodiment, the interactive signals between the air operation device and the touch screen are continuously monitored. When it is detected that the user operates the air operation device to generate a signal that meets the starting characteristics of the pointer drag operation (such as a finger on the touch pad is pressed and starts to move, or a specific button on the air mouse is pressed and accompanied by movement, etc.), the data acquisition process is immediately started. At the same time, the relevant data recording module is initialized to prepare for recording subsequent operation data frame by frame.

[0043] At each time point in the operation process, the current time information is accurately obtained and marked as a timestamp. This timestamp is used to calculate the time interval during the subsequent pointer drag operation. Its recording accuracy must be able to accurately reflect the change in pointer operation time, such as accurate to the millisecond level, so as to provide a strong basis for accurately analyzing the pointer operation rhythm.

[0044] Synchronously obtain the corresponding position information of the pointer on the touch screen. By comparing the changes in the pointer position between the previous and next two frames, calculate the movement of the pointer in different directions. Assuming that the pointer position coordinates of the current frame are a set of values, and the pointer position coordinates of the previous frame are another set of values, then by subtracting the horizontal and vertical coordinate values ​​of the two, the movement of the pointer in the horizontal and vertical directions can be obtained. Based on the positive and negative conditions of these movement amounts, the dragging direction of the pointer can be determined. If the horizontal movement amount is a positive value, it indicates that the pointer moves to the right in the horizontal direction; if it is a negative value, the pointer moves to the left in the horizontal direction. Similarly, the positive and negative values ​​of the vertical movement amount can also clarify the vertical movement direction of the pointer.

[0045] The pointer movement amount (including horizontal and vertical directions), pointer drag direction (determined by the positive or negative movement amount) and corresponding timestamp obtained at each time point are integrated into a complete set of pointer drag operation data. Then, these data are stored in the system-specified cache area or special data storage structure in chronological order, retaining complete original data for subsequent data processing and analysis.

[0046] Keep an eye on the pointer operation status, and stop the data acquisition operation immediately when it detects that the user operates the air operation device to generate a signal that meets the characteristics of the pointer drag operation end (such as lifting a finger on the touchpad, or releasing a specific button on the air mouse, etc.). At this point, the frame-by-frame data of the user's pointer drag operation from the beginning to the end has been fully recorded.

[0047] The pointer display on the touch screen is closely related to the operation of the aerial operating device. When the aerial operating device issues a pointer movement command, the touch screen will update the position of the pointer on the screen according to the relevant parameters in the command after receiving the command signal. The movement of the pointer on the screen is achieved by adjusting the coordinate value in the screen coordinate system. For example, in a two-dimensional touch screen coordinate system, the initial position of the pointer has a specific coordinate value. When a command is received that there is a movement in the horizontal direction and no movement in the vertical direction, the coordinate value of the pointer in the horizontal direction will be updated accordingly according to the movement, while the coordinate value in the vertical direction remains unchanged, thereby realizing the horizontal movement of the pointer on the screen.

[0048] The display of the pointer on the touch screen provides users with intuitive operational feedback. By observing the change in the position of the pointer on the screen, users can clearly understand the effect of their operations on the air-operated device. The display properties of the pointer, such as shape and color, can be adjusted according to the system settings to adapt to different screen backgrounds and operation scenarios, ensuring that the pointer has good visibility in all situations. For example, when using the car screen in a bright outdoor environment, the pointer can use thicker lines and bright colors so that the driver can easily see the pointer position; when using the educational tablet indoors, the style of the pointer can be customized according to the teaching content and interface style to make it coordinated with the overall visual effect without affecting the user's observation and operational perception of the pointer movement. This close connection between the pointer display and the user's operation and the good visual feedback mechanism ensure that users can accurately and efficiently control the touch screen pointer through the air-operated device and achieve smooth interactive operations with the touch screen content.

[0049] In one embodiment, Figure 2 FIG. 1 is a specific example diagram of a method for controlling a touch screen pointer in an embodiment of the present invention. Figure 2 As shown, the pointer dragging operation data performed by the user on the air operation device is obtained in real time, including:

[0050] Step 201: When it is detected that the user starts to perform a pointer drag operation on the air operation device, a start timestamp is generated;

[0051] Step 202: Record the pointer drag movement amount and pointer drag time interval during the drag operation frame by frame until it is detected that the user stops the pointer drag operation on the air operation device; at the same time, add a corresponding timestamp to each frame of recorded data.

[0052] In the embodiment, the interaction between the air operation device and the touch screen is always monitored. Once the signal emitted by the air operation device is detected to match the characteristics of the user starting to perform a pointer drag operation, such as sensing a finger pressing in a specific way on the touchpad and generating a movement trend, or a specific button on the air mouse is pressed and its movement signal is detected, the time acquisition operation is immediately triggered to accurately obtain the time information of the current moment and define it as the starting timestamp. This starting timestamp serves as the time reference point for recording the entire pointer drag operation data, and provides a key reference starting value for the subsequent calculation of the pointer drag related time interval.

[0053] From the moment the pointer drag operation is detected, the pointer state is sampled and recorded at a fixed time interval (for example, a frame every several milliseconds). At each sampling moment, the corresponding position information of the pointer on the touch screen is obtained, and the movement of the pointer relative to the position of the previous frame is calculated. At the same time, the timestamp of the current sampling moment is recorded, compared with the timestamp recorded in the previous frame, and the time interval between the two is calculated. This time interval is the pointer drag time interval.

[0054] This frame-by-frame recording operation is continued, and the pointer movement amount and time interval data are continuously updated until the user stops dragging the pointer. This process ensures a complete record of the entire pointer dragging operation trajectory, providing detailed raw data for subsequent data processing and analysis.

[0055] When recording the movement amount and time interval data of each frame of pointer, a corresponding timestamp is added to the frame data. This timestamp is not only used to calculate the pointer drag time interval, but also serves as a time mark for the entire data record, so that each set of data can be accurately located on the time axis. The pointer drag movement amount, pointer drag time interval and corresponding timestamp are integrated into a complete data unit and stored in a pre-initialized data structure in chronological order.

[0056] In one embodiment, Figure 3 FIG. 1 is a specific example diagram of a method for controlling a touch screen pointer in an embodiment of the present invention. Figure 3 As shown, the pointer drag movement amount and pointer drag direction during the drag operation are recorded frame by frame, including:

[0057] Step 301: Recording relative coordinate information corresponding to the pointer during the drag operation frame by frame;

[0058] Step 302: The change value and the sign of the change value between the relative coordinate information of each frame and the relative coordinate information of the frame after the frame are respectively used as the pointer drag movement amount and the pointer drag direction corresponding to the frame.

[0059] In one embodiment, Figure 4 FIG. 1 is a specific example diagram of a method for controlling a touch screen pointer in an embodiment of the present invention. Figure 4 As shown, the pointer dragging time interval during the dragging operation is recorded frame by frame, including:

[0060] Step 401: Record the timestamp corresponding to the pointer during the drag operation frame by frame;

[0061] Step 402: The time interval between the timestamp of each frame and the timestamp of the frame after the frame is used as the pointer dragging time interval corresponding to the frame.

[0062] In the embodiment, in the process of recording the pointer drag operation frame by frame, at each sampling moment, the relative coordinate information of the pointer in the touch screen coordinate system is obtained by a special position detection means. This relative coordinate information is based on a specific fixed reference point on the screen (such as the top left corner of the screen) or the pointer position of the previous frame, and the coordinate offset of the pointer in the horizontal and vertical directions relative to the reference is calculated. For each sampling frame, the coordinate offset of the pointer in the horizontal and vertical directions is carefully recorded, thereby forming the complete relative coordinate information of the frame.

[0063] In order to ensure the consistency and accuracy of the relative coordinate information recorded during the entire pointer dragging operation, a specific sampling strategy and data processing flow are adopted. Even if the pointer moves quickly or there is a slight jitter, the sampling strategy can ensure that the pointer position is sampled at an appropriate frequency to ensure that accurate relative coordinate information is obtained for each frame. The data processing flow is responsible for organizing and storing the sampled data to prevent data loss or erroneous recording. In this way, the pointer position at each time point can be accurately captured to ensure the reliability of the relative coordinate information recording.

[0064] For the relative coordinate information recorded in each frame, the difference between the relative coordinates of the current frame and the relative coordinates of the next frame in the horizontal and vertical directions is calculated to determine the movement of the pointer in these two directions. Assuming that the relative coordinates of the pointer in the current frame are a set of values, and the relative coordinates of the pointer in the next frame are another set of values, the movement in the horizontal and vertical directions is obtained through corresponding calculations. These movement values ​​can accurately reflect the change in the position of the pointer between two adjacent frames.

[0065] The direction of the pointer drag is determined based on the positive and negative values ​​of the calculated movement. If the horizontal movement is positive, it means that the pointer is moving to the right in the horizontal direction; if it is negative, the pointer is moving to the left in the horizontal direction. Similarly, for the vertical movement, a positive value means that the pointer is moving downward in the vertical direction, and a negative value means that it is moving upward. This method of determining the direction by the positive and negative values ​​of the movement is simple and effective, and can accurately reflect the direction of movement of the pointer on a two-dimensional plane.

[0066] When recording the pointer dragging operation data frame by frame, the timing device is used to obtain the current time information at each sampling moment and mark it as a timestamp. The timing device has a certain accuracy, which can reach millisecond level or higher, to ensure accurate recording of the pointer operation time.

[0067] As the pointer drag operation continues, the timestamp of each frame is continuously recorded to form a timestamp sequence arranged in chronological order. This sequence fully records the time trajectory of the pointer during the entire operation process. Each timestamp is associated with the corresponding pointer position information, movement amount and other data, together forming a complete data record of the pointer drag operation.

[0068] Taking the large-screen power system monitoring pointer controlled by the power dispatch center as an example, the recorded timestamp sequence can clearly show the time sequence of the dispatcher's operations when monitoring the power operation status and adjusting the power parameters. Every time point from starting to pay attention to the power load in a certain area to making adjustment decisions is accurately recorded, which helps to review the operation process.

[0069] For the recorded timestamp sequence, the difference between adjacent timestamps is calculated to determine the pointer drag time interval. Let the timestamp of the current frame be a specific time value, and the timestamp of the next frame be another time value. The difference between the two is the pointer drag time interval. This time interval reflects the time span of the pointer between two adjacent frames, that is, the time rhythm of the pointer movement process.

[0070] The calculated pointer drag time interval data has many applications in subsequent data processing and analysis. For example, the time interval data is an important basis for judging whether the user's operation is paused or moved slowly. If the time interval is long and the pointer movement is small, it may mean that the user has a short stay at a certain location or is hesitant in operation.

[0071] In the specific implementation, after performing step 101: acquiring the pointer dragging operation data performed by the user on the air operating device in real time, perform step 102: after the drag is released, perform a reverse displacement of the pointer dragging movement amount on each frame of data based on the pointer dragging time interval and the pointer dragging direction in the order or reverse order of the frame records in the pointer dragging operation data.

[0072] In one embodiment, it further includes:

[0073] For each frame of recorded data, determining whether the frame of recorded data is a target frame; the pointer dragging time interval corresponding to the target frame is greater than a first threshold; modifying the pointer dragging time interval of the target frame to a preset time interval; replacing the target frame in the pointer dragging operation data with the modified target frame to obtain modified pointer dragging operation data;

[0074] In the order or reverse order of the frame records in the pointer drag operation data, the reverse displacement of the pointer drag movement amount is performed on each frame of data based on the pointer drag time interval and the pointer drag direction, including:

[0075] In the corrected order or reverse order of the frame records in the pointer drag operation data, the data of each frame is subjected to a reverse displacement of the pointer drag movement amount based on the pointer drag time interval and the pointer drag direction.

[0076] In an embodiment, after obtaining the complete pointer drag operation data, each frame of recorded data must be carefully analyzed and judged to determine whether the frame of recorded data belongs to the target frame. The condition for determining a frame as a target frame here is that its corresponding pointer drag time interval must be greater than a first threshold. For example, when operating a device in the air (such as a remote control controlling a smart TV screen pointer to switch channels), each pointer movement process will be recorded frame by frame, and some of the frames correspond to pointers that stay in a certain position for a long time, that is, the pointer drag time interval is relatively large. When this time interval exceeds the pre-set first threshold, then this frame meets the target frame judgment condition. This first threshold is a time standard that is pre-set based on the actual operation scenario and considerations of the pointer operation characteristics, and is intended to screen out those operation frames with specific time characteristics.

[0077] Once a frame is determined to be the target frame, the next step is to modify the pointer drag time interval and change it to a preset time interval. This preset time interval is also a fixed time value set in advance based on factors such as the overall system requirements and operational smoothness. After completing the modification of the pointer drag time interval of the target frame, use these modified target frames to replace the corresponding target frames in the original pointer drag operation data. Through such a replacement operation, the corrected pointer drag operation data can be obtained. In the complete data record of the entire pointer drag operation, after such processing, those data whose original time intervals do not conform to the convention or affect subsequent processing are optimized, and the corrected data can more accurately reflect the actual situation of the pointer operation.

[0078] When the reverse displacement of the pointer drag movement is performed in the order of frame records in the corrected pointer drag operation data, the operation starts from the first frame data and proceeds step by step backwards. First, for the first frame data, obtain its pointer drag time interval and pointer drag direction information, and determine the direction of the reverse displacement based on the pointer drag direction. For example, if the pointer drag direction is moving to the right in the horizontal direction, then its reverse displacement direction is moving to the left in the horizontal direction. Next, set the corresponding waiting time according to the pointer drag time interval recorded in the frame. After the waiting time is over, let the pointer move the corresponding pointer drag movement in the determined reverse displacement direction. Then, follow the same steps to process each subsequent frame of data, such as the second frame, the third frame, etc. For example, in a scenario where a smart watch is used to control the pointer on the phone screen to browse a collection of pictures, if the reverse displacement is performed in sequence, the pointer will move back from the position of the last browsed picture in the order in which the pictures were browsed previously. Each frame moves precisely the corresponding distance in the corresponding time interval and reverse direction, achieving a step-by-step backtracking effect and simulating the process of the pointer returning in reverse according to the order of operations. This helps to restore the operation path and can better meet user needs in some application scenarios that require operation trajectories.

[0079] If you choose to perform the reverse displacement of the pointer drag movement in the reverse order of the frame records in the corrected pointer drag operation data, the operation will be processed from the last frame of data forward. First, obtain the key information such as the pointer drag time interval and pointer drag direction of the last frame, and also determine the reverse displacement direction according to the pointer drag direction. For example, if the pointer drag direction is moving downward in the vertical direction, then the reverse displacement direction is moving upward in the vertical direction. Then set the waiting time according to the pointer drag time interval of this last frame. Once the waiting time is up, let the pointer move the corresponding pointer drag movement in the reverse displacement direction. After processing the last frame, process the second to last frame, and repeat the above steps of obtaining information, determining direction, waiting time, and moving the pointer. Repeat this cycle until the first frame of data is processed. For example, in a scenario where a long list of contents (such as a news list) is quickly browsed through the mobile phone's air gestures (air operation device) to control the screen pointer, the reverse displacement method is adopted, and the pointer can quickly move back from the end of the list to the beginning, skipping the intermediate operation process and returning to the initial state more efficiently. In scenarios where high requirements are placed on operation efficiency and less attention is paid to the restoration of specific operation paths, this method can reduce user waiting time and improve the convenience and smoothness of operation.

[0080] Whether the reverse displacement operation of the pointer drag movement is based on the sequential or reverse order, it is to achieve the pointer can reasonably return to its position after completing a drag operation, so that the operation of the entire touch screen pointer is more in line with the user's operation expectations, enhance the user's experience when using the air operation device to control the touch screen pointer, and ensure that the entire pointer operation process can achieve a good state in terms of logic and actual effect.

[0081] In the above embodiment, after the user completes the drag operation, a reverse displacement operation is performed based on the recorded data, and the user does not need to manually return the pointer, thereby avoiding tedious manual return operations and directly improving operation efficiency; in the process of determining the target frame and modifying its pointer drag time interval and replacing the data to obtain the corrected pointer drag operation data, for those target frames with small pointer movement and long time interval, by modifying their time interval to a preset time interval, the problem of subsequent rebound calculation being too complicated and time-consuming due to long pauses is avoided, the control speed and operation efficiency of the touch screen pointer are accelerated, so that the pointer can rebound more promptly and reduce user waiting time; the reverse displacement of the pointer drag movement is performed in the order or reverse order of the frame records in the corrected pointer drag operation data, simulating a smooth experience similar to direct touch, improving the user's satisfaction with using the air operation device to control the touch screen in a non-touch screen scenario, and significantly improving the user experience.

[0082] In one embodiment, for each frame of recorded data, determining whether the frame of recorded data is a target frame includes:

[0083] Determine whether the pointer dragging time interval corresponding to the frame record data is greater than a first threshold;

[0084] If yes, determining whether the pointer drag movement amount corresponding to the frame record data is less than or equal to a second threshold;

[0085] If so, the frame recorded data is determined to be the target frame.

[0086] In the embodiment, in a specific embodiment, for each frame of recorded data that has been acquired, to determine whether it is a target frame, a judgment operation based on the time interval of dragging the pointer needs to be performed first.

[0087] Check the key information of the pointer drag time interval recorded in each frame of recorded data, and then compare it with the pre-set first threshold. The recorded pointer drag time interval reflects the length of time the pointer stays or moves during each operation stage. If the pointer drag time interval in a certain frame of recorded data is long, for example, it exceeds the set first threshold, the first threshold here is a time standard set based on the consideration of longer pause time or special time nodes in daily operations. When this limit is reached, the first condition of becoming a target frame is met, and further subsequent judgment is required at this time.

[0088] When the pointer drag time interval corresponding to a frame of recorded data is greater than the first threshold, it is then determined whether the pointer drag movement amount corresponding to the frame of recorded data is less than or equal to the second threshold.

[0089] In this frame of recorded data, the pointer drag movement reflects the magnitude of the position change of the pointer in the horizontal and vertical directions during the corresponding operation stage. If the pointer stays for a long time at a certain moment (the time interval is greater than the first threshold), but its position on the screen also changes greatly (that is, the pointer drag movement exceeds the second threshold), it does not meet the requirements of the target frame. Only when the pointer drag movement is within a relatively small range, that is, less than or equal to the second threshold, on the basis of meeting the time interval condition, the second threshold here is also set based on the actual operation scenario and the definition of the pointer's slight movement. Only in this way can it be determined that the frame of recorded data meets the characteristics of the target frame and is further determined to be the target frame.

[0090] By first filtering based on the pointer dragging time interval and then making a secondary judgment based on the pointer dragging movement amount, it is possible to accurately find those target frames that truly meet specific requirements and have special operating characteristics from a large number of frame recording data, laying the foundation for subsequent data processing of these target frames and optimization of the entire touch screen pointer operation, making the entire pointer control process more in line with various needs in actual application scenarios.

[0091] In one embodiment, for each frame of recorded data, determining whether the frame of recorded data is a target frame includes:

[0092] In the process of acquiring the pointer dragging operation data performed by the user on the air operation device, determining in real time whether the data recorded in the current frame is the target frame;

[0093] Or, after detecting that the user stops the pointer dragging operation on the air operation device, for each frame of recorded data, determine whether the frame of recorded data is a target frame.

[0094] In the above embodiment, in the process of obtaining the pointer drag operation data performed by the user on the air operation device, the target frame judgment operation is performed immediately after each frame of data is recorded. By comparing the pointer drag movement amount of the frame with the preset first threshold, and the pointer drag time interval of the frame with the preset second threshold, it is determined whether the current frame is the target frame. If the pointer drag movement amount is less than or equal to the first threshold, and the pointer drag time interval is greater than the second threshold, the frame is determined to be the target frame. For example, in a scenario where a mobile device is projected onto a smart TV and the projection pointer is controlled by a remote control (air operation device) to browse a web page, if the first threshold is set to a smaller moving distance value and the second threshold is set to a relatively longer time value, when the user slowly moves the pointer on the remote control and the pause time is long, the recorded frame data may meet the target frame condition and be determined as a target frame in real time during the recording process. This real-time judgment method can promptly discover special frames that may exist and provide instant information for subsequent data processing.

[0095] Another time to determine the target frame is after detecting that the user stops dragging the pointer on the air operation device. At this time, the pointer drag operation data of each frame that has been completely recorded is judged in turn. The pointer drag movement of each frame is compared with the first threshold, and the pointer drag time interval is compared with the second threshold one by one. Taking the example of drawing a graphic on an electronic whiteboard with an electronic pen (air operation device) and completing the drawing operation, the target frame judgment is started for each frame of data recorded during the entire drawing process. If the pointer movement of a frame of data is extremely small and the pause time is long (meets the target frame condition), it is determined as the target frame at this stage. This post-judgment method can analyze and determine the target frame more comprehensively from an overall perspective after obtaining all the operation data, avoid misjudgment caused by real-time judgment, and ensure the accuracy of the target frame judgment.

[0096] In one embodiment, it further includes:

[0097] Merging the pointer dragging movement amounts in the recorded data of the consecutive target frames to obtain a merged frame;

[0098] Modify the target frame's pointer dragging interval to a preset interval, including:

[0099] Modify the pointer dragging time interval of the merged frame to the preset time interval;

[0100] The target frame in the pointer drag operation data is replaced with the modified target frame to obtain the corrected pointer drag operation data, including:

[0101] The corresponding continuous target frames in the pointer drag operation data are replaced with the merged frames to obtain corrected pointer drag operation data.

[0102] In the embodiment, after the merged frame is obtained, the pointer dragging time interval needs to be modified. Specifically, the pointer dragging time interval of the merged frame is modified to a preset time interval. This preset time interval is a fixed time value pre-set according to various factors during the design process of the entire touch screen pointer control method. For example, in a car navigation system, when the driver operates the navigation screen pointer through the car controller to search for a destination or adjust the route, when there are continuous target frames (such as the pointer has multiple short pauses and small movements around a specific search result), after the previous steps are merged into a merged frame, the original pointer dragging time intervals of these continuous target frames may be uneven and complex. By modifying the pointer dragging time interval of the merged frame to a preset time interval, this part of the data can be made more regular and unified, which is convenient for subsequent analysis of the operation rhythm based on these data, and providing a unified time reference for related operations such as pointer rebound, avoiding unnecessary troubles caused by the differences and complexity of the original time intervals to the subsequent data processing and system function implementation, making the entire data processing process smoother and more efficient.

[0103] After completing the modification of the pointer drag time interval of the merged frame, the next step is to carry out a data replacement operation, that is, to replace the corresponding continuous target frames in the pointer drag operation data with the modified merged frame, thereby obtaining the corrected pointer drag operation data.

[0104] In the original pointer drag operation data, those continuous target frames each record the operation of the pointer at different small stages. Although they are related, the data is relatively scattered and complex. After the previous merging and time interval modification, replacing them with merged frames is equivalent to integrating and optimizing the operation information carried by these scattered and intrinsically related target frames. If there are continuous target frames that record multiple small movements and pauses of the pointer near a certain channel classification area, after the merged frame replacement, the corrected data will be more concise and clear when reflecting the pointer operation at this stage. It can more accurately reflect the overall operation characteristics of the pointer in the area, remove the redundant information that may be caused by the dispersion of multiple frames of data, so that subsequent operations such as analyzing the pointer operation trajectory and optimizing the system response based on the operation data can be carried out based on more accurate and effective data, further improving the smoothness and rationality of the entire touch screen pointer operation process, and more in line with the user's operation expectations during actual use, enhancing the user's operation experience.

[0105] In one embodiment, Figure 5 FIG. 1 is a specific schematic diagram of a method for controlling a touch screen pointer in an embodiment of the present invention. Figure 5As shown, the pointer drag movement in the recorded data of the continuous target frames is merged to obtain a merged frame, including:

[0106] Step 501: in the order of the timestamps of each frame recording data from earliest to latest, obtain the target frame with the earliest timestamp as the frame to be merged;

[0107] Step 502: Determine whether the next frame of the frame to be merged is the target frame;

[0108] Step 503: If yes, then merge the pointer dragging movement of the frame to be merged and the next frame to obtain intermediate data;

[0109] Step 504: Replace the frame to be merged with the intermediate data, and repeat the above steps of determining whether the next frame of the intermediate data is the target frame and merging, until the next frame of the intermediate data is not the target frame, or the intermediate data is greater than a third threshold; the obtained intermediate data is used as the merged frame.

[0110] In an embodiment, a large amount of frame record data will be generated during the entire touch screen pointer operation process, and for those data frames that meet specific conditions and are determined to be target frames, it is necessary to perform a merge operation according to certain rules to obtain a merged frame. First, based on the timestamp of each frame record data, it is sorted and checked in order from first to last. For example, in the scenario where the smart TV screen pointer is operated by a remote control to switch channels and browse related program information, each movement, pause and other operations of the pointer will generate frame record data with a timestamp. From these data, find the target frame with the earliest timestamp and determine it as the frame to be merged. This frame to be merged becomes the basis for the subsequent merging operation. It carries the operating characteristics of the pointer in a certain specific starting stage, such as the initial position of the pointer near a channel icon and the slight movement, etc., which provides an initial data reference for the subsequent merging operation with other continuous target frames.

[0111] After determining the frame to be merged, it is necessary to check whether the next frame of the frame to be merged is also the target frame. This step is crucial because only when the next frame is also the target frame, the prerequisite for further merging the pointer drag movement is met. Continuing with the pointer operation on the TV screen as an example, if the frame to be merged corresponds to the stage where the pointer briefly stays on a certain program details page and the movement is small, then when checking the next frame, it is necessary to determine whether the pointer operation recorded in this frame also meets the judgment criteria of the target frame, that is, to consider whether the relevant factors such as the pointer drag time interval and the pointer drag movement are within the preset range that meets the requirements of becoming a target frame. Only when the next frame also meets the target frame conditions can the next merging operation be carried out to ensure the rationality of the merging and the relevance of the data, avoid merging irrelevant frame data incorrectly, and affect the subsequent accurate analysis and processing of the overall situation of the pointer operation.

[0112] Once it is determined that the next frame after the frame to be merged is also the target frame, then the pointer drag movement of the two frames can be merged to obtain the intermediate data. For example, the pointer drag movement of the frame to be merged in the horizontal direction is a certain value, and there is also a corresponding movement in the vertical direction, and the next frame also has corresponding movement values ​​in the horizontal and vertical directions. At this time, the movement of the two frames is added in the horizontal direction to obtain the merged horizontal pointer drag movement. The same is true for the vertical direction. Through such calculation operations, a new intermediate data containing the merged pointer drag movement is formed. This intermediate data combines the situation of the pointer position changes of the two frames, and can more comprehensively reflect the operation characteristics of the pointer in this continuous small stage, laying the foundation for the subsequent continuous merging operation and the formation of the final merged frame, so that the integration of continuous target frames is more in line with the consistency and integrity of the actual operation of the pointer.

[0113] After obtaining the intermediate data, the intermediate data is used to replace the frame to be merged, so that the frame to be merged is updated to the state represented by the intermediate data. Then, the steps of determining whether the next frame of the intermediate data is the target frame and merging are repeated again. If the frame to be merged initially determined is updated after a merge operation, then the next frame of the "frame" corresponding to the new intermediate data is checked to determine whether it is still the target frame. If it is, continue to merge its pointer drag movement, continuously update the intermediate data, and continue this cycle. However, this merge operation will not proceed indefinitely, but has corresponding stop conditions, that is, until the next frame of the intermediate data is not the target frame, that is, a frame that does not meet the target frame judgment condition is encountered, or when the pointer drag movement corresponding to the intermediate data generated during the merge process exceeds a pre-set third threshold (this third threshold is a boundary determined based on the reasonable movement amplitude of the pointer, the operation logic and other factors in the actual operation, to avoid the unreasonable large or small situation of the merged data affecting the subsequent judgment), the merge operation is stopped.

[0114] In one embodiment, the reverse displacement of the pointer drag movement amount is performed on each frame of data based on the pointer drag time interval and the pointer drag direction in the order or reverse order of the frame records in the pointer drag operation data, including:

[0115] When it is detected that the user stops the pointer dragging operation on the air operating device, an instruction is triggered to perform a reverse displacement of the pointer dragging movement amount for each frame of data based on the pointer dragging time interval and the pointer dragging direction in the corrected order or reverse order of the frame records in the pointer dragging operation data.

[0116] In an embodiment, when it is accurately detected that the user stops the pointer dragging operation on the air operation device, this key event becomes the starting signal for triggering a series of subsequent operations. For example, after using a tablet computer to complete a content dragging operation in document editing through a stylus (air operation device), the stylus is lifted up and the signal of stopping the operation is immediately captured. At this time, the preparation for the pointer rebound operation is quickly started, and the reverse displacement operation of the pointer drag movement amount is performed on each frame of data based on the pointer drag time interval and the pointer drag direction according to the order or reverse order of the frame records in the corrected pointer drag operation data. This trigger mechanism ensures that the reverse displacement operation is started in a timely and accurate manner after the user completes a complete drag operation, which conforms to the logical order and expected effect of the user's operation.

[0117] In one embodiment, the reverse shift operation based on the sequence includes:

[0118] If you choose to perform the reverse displacement operation in the order of frame records in the corrected pointer drag operation data, start processing from the first frame of data. First, obtain the pointer drag time interval and pointer drag direction information of the first frame, as well as the pointer drag movement amount corresponding to the frame. Determine the direction of the reverse displacement according to the pointer drag direction. If the pointer drag direction is to the right (assuming the horizontal direction), the reverse displacement direction is to the left. Then, set a waiting time according to the pointer drag time interval of the frame. After the waiting time is over, move the pointer in the reverse displacement direction by the corresponding pointer drag movement amount. For example, in a scenario where a smart TV remote control (air operation device) controls the screen pointer to browse the program list, if the pointer in the first frame moves a certain distance to the right, during the reverse displacement, after waiting for the corresponding time, the pointer will move the same distance to the left.

[0119] After processing the first frame, the second frame is processed in sequence. Repeat the above steps of obtaining information, determining the reverse displacement direction, waiting time and moving the pointer. And so on, perform this operation for each frame of data until all frames are processed. In this process, the pointer will gradually trace back in the opposite order and movement amount of the previous drag operation to achieve reverse displacement from the end point to the starting point. This sequence-based reverse displacement operation can more intuitively simulate the pointer rebounding in the reverse process of the operation sequence, which is suitable for some scenarios that have strict requirements on the operation sequence or need to gradually restore the operation path.

[0120] In one embodiment, the reverse bit shift operation based on the reverse order includes:

[0121] When choosing to perform a reverse displacement operation in the reverse order of the frame records in the corrected pointer drag operation data, the operation starts from the last frame of data. First, obtain the pointer drag time interval, pointer drag direction, and pointer drag movement amount of the last frame. Similarly, determine the reverse displacement direction based on the pointer drag direction. If the pointer drag direction is upward (assuming a vertical direction), the reverse displacement direction is downward. Then, set the waiting time according to the pointer drag time interval of the frame. After the waiting time is over, move the pointer in the reverse displacement direction by the corresponding pointer drag movement amount. For example, in a scenario where a mobile phone is used to control the screen pointer to select a picture through air gestures (air operation device), if the pointer moves a certain distance upward in the last frame, the pointer will move the same distance downward after waiting for the corresponding time during the reverse displacement.

[0122] After processing the last frame, the penultimate frame is processed next. Repeat the steps of obtaining information, determining the reverse displacement direction, waiting time, and moving the pointer, and operate on each frame of data in order from back to front. In this process, the pointer will start from the last operated position, gradually trace back in the direction opposite to the operation order, and move quickly to the starting position. This reverse displacement operation based on reverse order can return the pointer to the starting position faster in some cases. It is suitable for scenarios with high requirements for operation efficiency and less attention to the details of the operation path, such as the operation of returning to the top of a long list after quickly browsing the list. The reverse displacement can quickly move the pointer back to the starting position, reducing user waiting time and improving the operation experience.

[0123] Regardless of whether the reverse displacement operation is based on sequential or reverse order, it is necessary to process the data of each frame during the entire process, and perform waiting and moving operations according to the pointer dragging time interval to ensure that the pointer's reverse displacement process is smooth and accurate, and can be well coordinated with other functions of the system (such as the processing of user interrupt operations, etc.) to achieve efficient and smooth pointer control effects and meet the needs of users in various scenarios of controlling the touch screen pointer in aerial operation devices.

[0124] In one embodiment, Figure 6 FIG. 1 is a specific example diagram of a method for controlling a touch screen pointer in an embodiment of the present invention. Figure 6 As shown, it also includes:

[0125] Step 601: in the process of performing reverse displacement of the pointer drag movement amount based on the pointer drag time interval for each frame of data, if it is detected that the user performs the pointer drag operation again, then after the reverse displacement process is completed, the pointer drag operation data performed again by the user is obtained; the starting time of recording the pointer drag operation data performed again is the time when the reverse displacement process is completed;

[0126] Step 602: Based on the pointer dragging operation data performed again, reverse displacement is performed again.

[0127] In the embodiment, when performing a reverse displacement operation of the pointer drag movement amount based on the pointer drag time interval for each frame of data according to the established rules, the entire process needs to keep monitoring the user's operation status at all times. For example, after using a tablet computer to control the screen pointer through a stylus (as an air operation device) to perform a text movement operation in document editing, when the pointer begins to perform a reverse displacement based on the previously recorded data, that is, in the process of moving back to the initial position, if it is detected at this time that the user performs a pointer drag operation on the screen again through the stylus, the current reverse displacement operation will not be interrupted immediately, but will be allowed to continue until the current entire reverse displacement process is completed.

[0128] The reason for this is to ensure the integrity of each operation and the continuity of data recording. Only when the entire reverse displacement process is completed, will the user's pointer dragging operation data be obtained again. And this pointer dragging operation data has a clear starting recording time requirement, and its starting recording time is the moment when the above-mentioned reverse displacement process is completed. This means that from this specific moment, the new user operation is regarded as an independent and complete operation stage, and all subsequent data records related to it will revolve around this starting point, so as to clearly and accurately capture the user's new operation intentions and various details in the operation process, such as the direction and amount of movement of the pointer, and the time interval corresponding to each operation, etc., to provide an accurate data basis for subsequent further data processing and corresponding function realization.

[0129] After successfully obtaining the pointer dragging operation data of the user again, the next step is to perform the reverse displacement operation again based on the new data. The purpose of this step is to allow the pointer to return to its original position reasonably according to the corresponding rules after the new operation phase ends, ensuring the continuity of the entire pointer operation process at different stages and meeting the user's operating habits and expectations.

[0130] Based on the sorted data of the pointer drag operation, the reverse displacement operation is actually performed. Starting from the first frame of the new data, the pointer drag time interval and pointer drag direction information corresponding to the frame are first obtained, and the direction of the reverse displacement is determined according to the pointer drag direction. For example, if the pointer drag direction is moving to the right in the horizontal direction, then the reverse displacement direction is moving to the left in the horizontal direction. Then, the corresponding waiting time is set according to the pointer drag time interval recorded in the frame. When the waiting time is over, the pointer is moved according to the determined reverse displacement direction by the corresponding pointer drag movement amount.

[0131] Then, the second frame, the third frame, and each subsequent frame of data are processed in the same steps, so that the pointer can be gradually retraced and moved back from the last operation position to the corresponding starting position. Through such a complete reverse displacement operation, no matter how many times the user drags the pointer at different stages, it can be guaranteed that the pointer can eventually return to its proper position, making the entire touch screen pointer control process more flexible and reliable, further improving the user's operating experience when using the air operation device to control the touch screen pointer, and meeting a variety of practical application needs.

[0132] In one embodiment, it further includes:

[0133] During the process of performing reverse displacement of the pointer dragging movement amount based on the pointer dragging time interval for each frame of data, if it is detected that the user is performing an operation of moving the pointer, the process of performing reverse displacement of the pointer dragging movement amount based on the pointer dragging time interval is stopped, and the pointer is controlled to respond to the pointer movement operation.

[0134] In the embodiment, when performing a reverse displacement operation of the pointer drag movement amount based on the pointer drag time interval for each frame of data according to a predetermined process, the entire process requires constant close attention and accurate monitoring of the user's operation status.

[0135] When the pointer starts to move in the reverse direction according to the previously recorded frames of data, that is, in the process of moving back according to the corresponding time interval and in the reverse direction, it is detected in real time whether the user has made a new operation. If during this reverse displacement process, it is detected that the user has moved the pointer, such as the user pressing the corresponding button again on the remote control and moving the pointer, intending to move the pointer to a new position or change its original movement trajectory, at this time, it will respond immediately and stop the reverse displacement process of the pointer dragging movement amount based on the pointer dragging time interval.

[0136] This is because the user's newly initiated pointer movement operation represents his latest operation intention. In order to ensure the immediacy of the operation and meet the user's expectations, the new operation is given priority to be effectively executed. Therefore, the state will be quickly switched to control the pointer to respond to the new pointer movement operation.

[0137] Specifically, based on the specific instructions issued by the user through the air operating device, such as the direction and speed of movement (this information is accurately captured through the interaction mechanism between the operating device and the screen), the position and movement status of the pointer on the screen will be adjusted in real time, so that the pointer moves on the screen in the direction and rhythm expected by the user, as if the reverse displacement operation that was previously in progress had never occurred. The user's new operating instructions are the main guide to ensure that the pointer can move accurately and smoothly on the screen according to the new operating intention, thereby providing users with a coherent and convenient operating experience, meeting the user's diverse requirements for touch screen pointer control at different times and different needs, and ensuring that the entire touch screen pointer operation process can flexibly respond to various situations without being disturbed by previously unfinished operations.

[0138] A specific embodiment is given below to illustrate the specific application of the method of the present invention.

[0139] In a specific embodiment, a solution for implementing a pointer automatic rebound mechanism based on relative coordinates is embodied, and the solution mainly includes the following steps:

[0140] one, Figure 7 FIG. 1 is a specific example diagram of a sliding recording stage in a method for controlling a touch screen pointer in an embodiment of the present invention. Figure 7 As shown, the sliding recording stage involves the following operations:

[0141] 1. Start operation detection: Continuously monitor the interactive operation between the user's air operation device (such as touchpad, air mouse, etc.) and the touch screen. When it is detected that the user starts to drag the pointer on the touch screen through the air operation device, this point is determined as the starting point A. During this process, the movement of the pointer is tracked and recorded throughout the process.

[0142] 2. Movement recording: When the pointer starts to drag from starting point A, accurately record the movement of the pointer in the horizontal direction (x-axis) and vertical direction (y-axis) Δx and Δy each time a drag command is sent. These movement data reflect the change in the position of the pointer on the screen and are important basic data for subsequent calculations and operations.

[0143] 3. Time interval recording: At the same time, the time interval Δt between each instruction is also recorded. This time interval data can reflect the rhythm and speed of user operations, which is of great significance for judging the characteristics of user operation behavior and subsequent rebound operation optimization.

[0144] 4. Continue recording until the operation is completed: Continue to perform the above recording operation until the user releases the operating device at the end point B, completing the pointer dragging operation, to ensure that all data in the entire operation process are fully recorded.

[0145] Among them, when the user uses the air operation device to drag the pointer on the touch screen, each drag command sent is to move the pointer in the direction and distance expected by the user. For example, when using the touchpad to control the pointer on the computer screen to move the file icon, each command sent corresponding to the drag action made by the finger on the touchpad is a clear manifestation of the operation intention, and its purpose is to change the pointer position. And "each frame of recorded data" is also a detailed disassembly and recording of the entire pointer drag operation process, which is divided according to fixed time intervals or operation stages, and the operation process is recorded one by one. Each frame of data recorded actually corresponds to the operation issued by the user at that moment through the air operation device to make the pointer change accordingly. This is essentially consistent with the behavior of sending a drag command to drive the pointer to move, and both reflect an immediate action and intention of the user during the operation.

[0146] Each time a drag command is sent, it is a discrete operation point on the timeline, and they are arranged in sequence to form a complete time sequence of the entire drag operation. For example, when the remote control is used to operate the pointer on the smart TV screen to browse the program list, each time the user presses the remote control button and moves the pointer, a drag command is sent, and these commands are distributed on the timeline of the operation in sequence. Each frame of recorded data is also recorded in chronological order, and each recorded frame has a corresponding timestamp, marking its time position in the entire operation process. Therefore, from the time dimension, both are the embodiment of the operation process at different time nodes, and have inherent consistency.

[0147] "Each time a drag command is sent" will inevitably cause the pointer to change state accordingly, such as the position and speed of the pointer. "Each frame of recorded data" contains information such as the amount of pointer drag movement, the direction of pointer drag, and the time interval of pointer drag. This information is a detailed record of the pointer state changes after each drag command is sent. For example, when a drag command is sent to move the pointer from the left side of the screen to the right side, "Each frame of recorded data" can record how much the pointer moves in each small time interval (pointer drag movement), in which direction it moves (pointer drag direction), and how long the interval is between two adjacent frames (pointer drag time interval). Through these recorded data, the effect of each drag command can be fully restored. The two are closely corresponding in terms of recording the pointer state and reflecting the operation results.

[0148] two, Figure 8 FIG. 1 is a specific example diagram of a data processing phase and a rebound execution phase in a method for controlling a touch screen pointer in an embodiment of the present invention. Figure 8 As shown, the data processing stage (including optimization processing) involves the following operations:

[0149] 1. Frame-by-frame analysis and dwell determination: The pointer dragging time interval corresponding to the target frame in this specific embodiment is greater than the first threshold, and the corresponding pointer dragging movement amount is less than or equal to the second threshold; the recorded data is calculated and analyzed frame by frame. If the pointer movement distances in the horizontal direction (Δx) and the vertical direction (Δy) between two recording points are both less than the preset values, and the time interval Δt between the two recording points exceeds the preset value (such as 50ms), it is determined that the user has stayed here for a long time. In order to optimize data processing and subsequent rebound operations, the dwell time can be set to a fixed time T1 (such as 30ms) at this time. This processing method can not only reduce unnecessary time calculations, but also ensure accurate determination of the mouse stop state, and ensure that when processing subsequent frame data, the pointer movement will not be accelerated due to the dwell state, making data processing more accurate and efficient.

[0150] 2. Continuous dwell processing: If there is a continuous dwell time, that is, the moving distance between multiple consecutive recording points is less than the preset distance, the time of the consecutive frames is accumulated and classified as one frame for processing. At this time, the moving distance is the cumulative distance of several consecutive frames, and the dwell time is also set to T1. In this way, continuous dwell situations can be handled more reasonably, data redundancy can be reduced, the data processing process can be further optimized, and more accurate data support can be provided for subsequent rebound operations.

[0151] 3. Optimization of dragging time that is too long (extreme cases): During the data processing process, if it is detected that the user drags for too long, or even in extreme cases where the pointer is dragged back and forth between AB, corresponding measures will be taken to streamline the rebound time. In this case, if it is processed according to the conventional algorithm, the pointer will move back and forth in the opposite direction for a long time, seriously affecting the operating efficiency and user experience. Therefore, the recorded data is analyzed and processed through a specific algorithm. For example, after recording all the movement data, before moving in the opposite direction, check whether the pointer has moved in the opposite direction. If it moves in the opposite direction, the reverse movement distance and the forward movement will be offset in the algorithm. Then, within a certain time range, move the cursor so that the cursor roughly returns to its original position, avoiding unnecessary long-term reverse movement and improving operating efficiency.

[0152] three, Figure 8 FIG. 1 is a specific example diagram of a data processing phase and a rebound execution phase in a method for controlling a touch screen pointer in an embodiment of the present invention. Figure 8 As shown, the rebound execution phase involves the following operations:

[0153] 1. Delay setting: After the user releases the operating device, a short delay of T1 or greater than T1 (such as 50ms) will be set. The main purpose of this delay setting is to reduce the inaccurate rebound problem caused by the cursor speed gain. During the dragging process of the pointer, the cursor may produce a certain gain effect due to the speed change. If the rebound operation is performed immediately, the rebound position may be inaccurate. Through appropriate delay, there is enough time to stabilize the cursor state and improve the accuracy of the rebound operation.

[0154] 2. Reverse movement execution: After the delay is over, the reverse movement operation is performed in sequence according to the previously recorded instructions and the original instruction intervals. That is, for the recorded horizontal movement amount Δx, the reverse movement -Δx is performed; for the vertical movement amount Δy, the reverse movement -Δy is performed. In this way, the pointer can be gradually moved from the end point B in the opposite direction of the previous drag, so that the pointer gradually approaches the starting point A.

[0155] 3. Rebound completion judgment: After the reverse movement instruction is executed, the pointer should be able to roughly return to the starting point A. Here, "roughly" means that there may be certain errors in the processing process, but through the above-mentioned recording, processing and execution mechanism, it can ensure that the pointer returns to the vicinity of the starting position within a reasonable error range, realizing the automatic rebound function of the pointer, so that the user does not need to manually move the pointer back to the starting point, thereby improving operation efficiency.

[0156] 4. Optimization measures:

[0157] 1. Optimization of long dragging time (repeated): As mentioned above, during the data processing stage, if the user drags for too long (including dragging back and forth), the rebound time is shortened through optimization algorithms to ensure that the pointer can return to the approximate starting position within a reasonable time, avoiding the impact of too long rebound process on operation efficiency and user experience.

[0158] 2. User interrupt processing:

[0159] (1) Processing of new drag gestures: If a new drag gesture is detected during the rebound process, the current rebound operation will be paused and the new drag operation will be treated as a priority. After the current rebound operation is completed, the target rebound operation mode will be re-determined and a new pointer rebound operation will be executed based on the new pointer drag operation data and the above-mentioned sliding record, data processing and rebound execution process. This ensures that the user's new operation can be responded to in a timely manner, while avoiding conflicts between new and old operations, ensuring the continuity and accuracy of operations.

[0160] (2) Pointer movement processing: If it is detected during the rebound process that the user is moving the pointer (rather than starting a new drag gesture), the rebound process will be stopped immediately and the pointer movement operation will be responded to first. At this time, the pointer position will be updated according to the user's new pointer movement instruction, and after the pointer movement operation is completed, it will be determined whether the rebound operation needs to be re-executed or processed according to the new operation process based on the pointer state and operation status at that time. This flexible interrupt processing mechanism can better adapt to various changes in the user's actual operation process, improving the response speed and user experience.

[0161] In this specific embodiment, when the pointer is dragged, if it slides back and forth for a long time multiple times, the conventional rebound operation will cause the pointer to move in the opposite direction for a long time multiple times when it rebounds, which will inevitably reduce the operation efficiency and have a negative impact on the user experience, so special algorithm optimization is required.

[0162] During the specific operation, the pointer movement data is fully recorded throughout the process. All information from the starting position to each intermediate position and the final end position is recorded, including the change in the horizontal and vertical movement amount and the corresponding time sequence. Before the pointer rebound operation is about to be performed, the optimization algorithm process is started. First, a comprehensive check is carried out on all the recorded movement data, focusing on determining whether the pointer has reverse movement during the dragging period. The determination of this reverse movement is based on the precise analysis of the movement direction, and is determined by comparing the change in the pointer movement direction between adjacent data points.

[0163] Once the pointer is detected to be moving in the opposite direction, special processing is implemented at the algorithm level to offset the distance of the opposite movement with the distance of the positive movement. For example, if the pointer moves 100 pixels in the positive direction of the x-axis first, and then moves 30 pixels in the negative direction of the x-axis, then in the optimization algorithm, the effective positive movement distance is recorded as 70 pixels (100-30). In this way, the redundant movement calculations caused by sliding back and forth can be effectively reduced, and the amount of data that needs to be processed during the rebound process can be streamlined.

[0164] After completing the offset processing of the reverse movement distance and the forward movement distance, move the cursor within a certain reasonable time range. The setting of this time range comprehensively considers factors such as device performance, real-time requirements of user operations, and stability of pointer movement. Based on the remaining valid movement data, control the movement of the pointer in a relatively stable and efficient manner to move it toward the starting position. Although the cursor may not be able to accurately return to the original starting position at this time due to interference from the offset processing and various factors in actual operation, with the help of the efficiency of the optimization algorithm, the cursor can roughly return to the vicinity of the starting position, which is sufficient to meet the user's needs to continue subsequent operations. At the same time, it avoids the long and complex reverse movement process caused by the traditional rebound method, significantly improves operating efficiency and user experience, and makes pointer control more intelligent and efficient in complex drag scenarios.

[0165] This specific embodiment has the following technical effects:

[0166] 1. Improved operational efficiency

[0167] The automatic pointer rebound mechanism of the present invention is implemented in relative coordinates. This feature allows the user to move the pointer back to the starting point without manual intervention after performing a pointer operation. For example, in a scenario where the pointer is frequently dragged from point A to point B, such as when browsing a long video list on a smart TV, after the user selects a video to play, the pointer will automatically rebound to the initial position without manual operation by the user, thereby greatly reducing repetitive operations. This automated rebound process saves user operation time, significantly improves work efficiency, and allows users to focus more on the operation task itself, rather than spending time adjusting the pointer position.

[0168] 2. User experience optimization

[0169] The design of the automatic rebound mechanism simulates the feeling of touch operation. When an air-operated device (such as a touchpad, air mouse, etc.) interacts with the touch screen, the automatic rebound behavior of the pointer is similar to the feeling of the interface elements automatically resetting after the finger leaves the screen in direct touch operation. Taking in-vehicle infotainment as an example, when the driver uses air gestures to control the screen to select a navigation destination or switch music tracks, the process of the pointer automatically rebounding to the starting position makes the operation more intuitive and natural, as if the touch operation is performed directly on the screen, reducing the discomfort caused by differences in operation methods, improving the driver's operating convenience and comfort during driving, and enhancing the user's satisfaction with the entire interaction process.

[0170] 3. Cross-platform compatibility achieved

[0171] The implementation based on relative coordinates brings excellent cross-platform compatibility to the mechanism. Regardless of the operating system or different types of devices (such as mobile phones, tablets, car screens, smart TVs, educational tablets, augmented reality and virtual reality devices, etc.), the pointer automatic rebound mechanism can work effectively. On different platforms and devices, relative coordinates focus on the relative movement of the pointer within the screen, rather than relying on the absolute position information of a specific device. This eliminates the need for developers to perform a lot of customized development for different operations and devices, reduces development costs and complexity, ensures that users can enjoy a consistent pointer control experience on different devices and platforms, and improves the versatility and scalability of the technical solution.

[0172] 4. Enhanced flexibility and controllability

[0173] By providing optimization measures such as threshold setting and speed control, the rebound mechanism shows a high degree of flexibility and controllability. In terms of threshold setting, for example, the thresholds for the duration and distance of the pointer drag can be set. When the user operates within different threshold ranges, different rebound modes can be automatically selected. In terms of speed control, the speed of pointer rebound can be adjusted according to different application scenarios and user operation habits. For example, on an educational tablet, when teachers are giving explanations and demonstrations, they may want the pointer to rebound at a moderate speed so that students can clearly observe the movement trajectory of the pointer; when users quickly browse long lists of content, such as searching for programs on smart TVs, the pointer rebound speed can be appropriately increased to improve operation efficiency. This flexibility enables the rebound mechanism to better adapt to diverse user needs and operation scenarios, and improves the intelligence and customizability of the mechanism.

[0174] 5. Wide adaptability

[0175] The automatic pointer rebound mechanism is particularly suitable for application scenarios that require frequent sliding operations. When browsing long lists, such as product lists or news information lists on e-commerce platforms, the pointer automatically rebounds after the user continuously slides and browses, allowing the user to quickly return to the top of the list for new browsing operations; in page turning operations, whether it is e-book reading or document viewing, the pointer rebounds to the starting position to facilitate the user to turn to the next page; when adjusting the volume / brightness and other operations, such as on smart TVs or in cars, the pointer automatically rebounds after the user adjusts it, avoiding interference with subsequent operations due to changes in the pointer position. This wide adaptability enables this technical solution to play an important role in many common interactive scenarios, improving the convenience and fluency of user operations.

[0176] 6. Simulate finger sliding effect

[0177] The present invention achieves an operation effect similar to direct touch, effectively enhancing the user's natural interaction experience. During the operation, the movement and rebound behavior of the pointer are similar to the feeling of sliding a finger on the screen. Taking augmented reality and virtual reality devices as an example, when the user controls the pointer through the air operation device in the virtual space, the automatic rebound method of the pointer simulates the operation logic of the finger on the virtual interface, making it easier for the user to understand and adapt to the interaction method in the virtual environment, as if directly interacting with the virtual elements with their fingers, which improves the user's immersion in the virtual scene and the naturalness of the operation, and shortens the interaction distance between the user and the virtual interface.

[0178] This specific embodiment has the following innovative features:

[0179] 1. Relative coordinates realize automatic rebound advantage

[0180] The use of relative coordinates to achieve automatic rebound is an important innovation of the present invention. Relative coordinates focus on the relative position change of the pointer in the screen coordinate system, and compared with traditional absolute coordinates, they overcome many of its limitations. In absolute coordinates, the position of the pointer depends on the absolute physical position information of the device, which makes the control logic of the pointer complicated and difficult to unify on different devices or in different operating environments. For example, in some mobile devices, changes in screen orientation or differences in device hardware may cause deviations in the calculation of the pointer position under absolute coordinates, affecting the accuracy and consistency of user operations. The relative coordinates of the present invention can stably implement the automatic rebound function of the pointer under different devices and operating scenarios by recording the movement of the pointer relative to the starting point (Δx and Δy) and the operation time interval (Δt), ensuring the accuracy and reliability of the operation, and providing users with a more consistent and convenient pointer control experience.

[0181] 2. Record and replay instructions to achieve accurate rebound

[0182] The present invention achieves accurate rebound by recording and replaying movement instructions. This innovative method does not rely on the absolute position information of the device. During the pointer operation process, the movement amount (Δx and Δy) and the instruction time interval (Δt) of each drag instruction are accurately recorded. These recorded data become the key basis for the pointer rebound. When the user releases the operating device, according to the recorded instructions, the reverse movement (-Δx and -Δy) is performed in sequence according to the original instruction interval, so as to achieve accurate rebound of the pointer from the end point to the starting point. This method avoids the problem of inaccurate rebound caused by differences in device hardware or different understandings of absolute position by operators. For example, on different models of mobile phones or tablets, although the hardware parameters such as the screen size and resolution of the device may be different, the rebound mechanism based on recording and replaying movement instructions can ensure that the pointer can rebound to the vicinity of the starting position according to the expected trajectory on various devices, improve the accuracy and reliability of the rebound, and make the pointer control more accurate and stable.

[0183] 3. Delay mechanism improves rebound accuracy

[0184] The introduction of a delay mechanism is another innovation of the present invention, which effectively reduces the problem of inaccurate rebound caused by the cursor speed gain. During the pointer dragging process, changes in the cursor speed may cause a speed gain effect, which makes it difficult to accurately control the initial speed and trajectory of the cursor during rebound, thereby affecting the accuracy of the rebound. The present invention sets a short delay (such as T1 or greater than T1, such as 50ms) after the user releases the operating device. During the delay period, the cursor speed and state can be stabilized to eliminate the influence of the speed gain, so as to prepare for subsequent precise rebound operations. Through this delay mechanism, the pointer rebound process can be started in a more stable and accurate state, ensuring that the pointer returns to the starting position according to the predetermined path and speed, thereby improving the accuracy and reliability of the rebound operation, allowing users to more accurately anticipate the pointer's rebound behavior during operation, and improving the overall operating experience.

[0185] 4. Optimization measures to enhance resilience intelligence and adaptability

[0186] A series of optimization measures, such as rebound threshold and speed control, are designed to significantly improve the intelligence and adaptability of the rebound mechanism. The setting of the rebound threshold, such as the threshold of the pointer dragging time and distance, can automatically select the appropriate rebound mode according to the characteristics of the user's operation. When the user performs fast and small-range pointer operations, a fast and simple rebound mode can be selected through threshold judgment; for long-distance and complex pointer operations, a more suitable rebound mode is selected according to the threshold. In terms of speed control, the pointer rebound speed can be dynamically adjusted according to different application scenarios and user operation habits. In scenarios that require fast response, such as game operations or fast browsing interfaces, the rebound speed is increased; in scenarios that require precise operations, such as drawing or fine-tuning settings, the rebound speed is reduced. These optimization measures enable the rebound mechanism to intelligently adapt to the needs of different user operations and application scenarios, provide more personalized and efficient pointer control services, and enhance the practicality and flexibility of the technical solution.

[0187] 5. Cross-platform implementation ensures consistent experience

[0188] Providing a cross-platform implementation solution is one of the key innovations of the present invention, which ensures a consistent experience on different operations and devices. Based on relative coordinates and unified pointer control logic, the present invention can implement the same automatic pointer rebound function on different mobile phone operating systems and various devices such as mobile phones, tablets, car screens, smart TVs, educational tablets, augmented reality and virtual reality devices. Regardless of the platform and device, users can experience similar operating methods and interactive experiences, reducing the learning cost of users when switching between different devices. For example, when users use in-car infotainment and mobile phones, their expectations for pointer operation and rebound are consistent, which improves users' acceptance and satisfaction with the entire technical solution, promotes the widespread application of technology in different fields and devices, and enhances the versatility and compatibility of the technical solution.

[0189] This specific embodiment has the following potential application scenarios:

[0190] 1. In-vehicle infotainment applications

[0191] In in-vehicle infotainment, when the driver controls the screen through air gestures, the automatic rebound mechanism of the pointer of the present invention plays an important role. Since the driver needs to concentrate on the road conditions during driving, it is neither convenient nor safe to manually operate the screen. Through an air operation device (such as a touchpad similar to the one on the vehicle center console or a controller with gesture recognition function), the driver can easily control the pointer to move on the screen, select the navigation destination, switch music tracks, adjust the radio channel or operate other in-vehicle application functions. The automatic rebound of the pointer to the starting position enables the driver to quickly perform the next operation after completing an operation, without the need to be distracted to adjust the pointer position, thereby improving the operation efficiency and driving safety. For example, in the navigation operation, the pointer automatically rebounds after the driver selects the destination, which facilitates the driver to immediately plan the route or view other navigation information; when the music is playing, the rebound of the pointer after adjusting the volume or switching songs makes the operation smoother, reduces the interference caused by inconvenient operation to driving, and improves the driver's interactive experience in the car.

[0192] 2. Application on smart TV or large-screen devices

[0193] For smart TVs or large-screen devices, users usually use remote controls or air mice to operate them. When browsing long lists of content (such as video program lists, application lists, or setting option lists, etc.), the pointer automatically rebounds to the top or starting position of the list after frequent sliding operations, allowing users to quickly return and make new selections. For example, when users are watching online video platforms, after browsing many video programs, the pointer rebounds so that they can easily return to the top of the list to select new categories or filtering conditions; when setting smart TV parameters, the pointer automatically resets after adjusting each setting, making it easier to perform the next setting operation. This automatic pointer rebound mechanism greatly improves the user's convenience when using smart TVs or large-screen devices, allowing users to interact with the device more efficiently and enjoy rich multimedia content and convenient device control.

[0194] 3. Educational tablet applications

[0195] In the field of education, educational tablets provide teachers with the convenience of controlling screen content from a distance. Teachers can control the display of teaching courseware on the tablet, mark key content, switch pages, or start different teaching applications through air operation devices (such as wireless stylus or remote control paired with the tablet) at different locations in the classroom. The automatic rebound of the pointer to the starting position helps teachers maintain the consistency and efficiency of operations. For example, when explaining electronic courseware, the pointer rebounds after the teacher marks a knowledge point, which is convenient for them to immediately mark the next knowledge point or switch to the next page of courseware; when using teaching software for interactive teaching, such as deducing mathematical formulas or drawing graphics, the pointer is reset to facilitate subsequent operations. This not only improves the teaching efficiency of teachers, but also makes the teaching process smoother, enhances students' understanding and attention to teaching content, and provides a more efficient and convenient interactive method for modern education and teaching.

[0196] 4. Augmented reality (AR) and virtual reality (VR) device applications

[0197] In augmented reality and virtual reality devices, the automatic pointer rebound mechanism of the present invention provides a more natural way of interacting with virtual interfaces. When a user interacts with a virtual interface in a virtual space through a head-mounted device and a matching controller (as an air operation device), the automatic rebound of the pointer simulates the feeling of interaction between fingers and objects in the real world. For example, in an AR game, after the player controls the pointer to select game props or perform operations through a controller, the pointer rebounds to make the operation more natural and smooth, enhancing the immersion of the game; in VR architectural design software, the designer uses the pointer to select architectural elements and adjust the layout, and the pointer automatically resets to facilitate the next round of design operations, improving design efficiency and the naturalness of interaction. This automatic pointer rebound mechanism in a virtual environment helps to narrow the distance between the user and the virtual interface, allowing the user to integrate into the virtual world more naturally, and improving the user's interactive experience and work efficiency in AR and VR applications.

[0198] Of course, it is understandable that the above detailed process may have other variations, and the relevant variations should all fall within the protection scope of the present invention.

[0199] In an embodiment of the present invention, pointer dragging operation data performed by a user on an aerial operating device is acquired in real time; the touch screen displays a pointer that moves in response to the pointer dragging operation; the pointer dragging operation data includes the pointer dragging movement amount, pointer dragging direction, and pointer dragging time interval of each frame in the dragging operation process recorded frame by frame; after the drag is released, the pointer dragging movement amount is reversely displaced based on the pointer dragging time interval and pointer dragging direction for each frame of data in the order or reverse order of the frame records in the pointer dragging operation data. The embodiment of the present invention can completely and accurately record the user's operation trajectory by acquiring detailed data of each frame when the user performs a pointer drag operation on the air operation device in real time, including the pointer drag movement amount, direction and time interval; when the user completes the drag operation, a reverse displacement operation is performed based on the recorded data, without the need for the user to manually return the pointer, thereby avoiding tedious manual return operations and directly improving the operation efficiency; the reverse displacement of the pointer drag movement amount is performed in the order or reverse order of the frame records in the pointer drag operation data, simulating a smooth experience similar to direct touch, improving the user's satisfaction with using the air operation device to control the touch screen in a non-touch screen scenario, and significantly improving the user experience; by focusing on the relative position change of the pointer during the operation process without relying on the absolute position information of the device, a unified pointer control logic can be implemented on different operating systems and various types of mobile screen devices, providing a cross-platform, highly compatible solution, and meeting the pointer control needs of diverse devices in non-touch operation scenarios.

[0200] As mentioned above, the present invention focuses on addressing the significant defects in the prior art, that is, when the user leaves the touch screen and uses the pointer to control the touch interface, the frequent manual operation of returning the pointer from the end point to the starting point brings great inconvenience to the user. Based on this, the present invention establishes the following specific objectives:

[0201] 1. Significantly improve user operation efficiency: We are committed to reducing repetitive actions to the greatest extent possible, and realize the automatic rebound function of the pointer through innovative technical means. For example, in application scenarios such as smart TVs, when users frequently switch between different function menus using air operation devices (such as remote controls), the pointer can automatically rebound to the initial position after completing each selection, thereby effectively avoiding the extra time and energy spent by users manually adjusting the pointer position, greatly speeding up the operation process, allowing users to complete more operation tasks per unit time, and significantly improving overall operation efficiency.

[0202] 2. Deeply improve user experience: Make every effort to create a user experience that is highly similar to the direct touch experience. In the process of interaction between the air-operated device and the touch screen, the movement characteristics and rebound mechanism of the pointer are carefully designed to make it similar to direct touch operation in terms of operation logic and visual feedback. For example, in in-vehicle infotainment, when the driver uses air gestures to control the screen to set the navigation destination or control music playback, the automatic rebound process of the pointer is smooth and natural, giving the driver an intuitive feeling of directly touching the screen, reducing the discomfort caused by differences in operation methods, and thus comprehensively improving the user's satisfaction and comfort when using air-operated devices.

[0203] 3. Build a cross-platform, highly compatible solution: Based on relative coordinates, carefully build a universal technical framework to ensure that it can be seamlessly adapted to various operating systems and diverse device types (including mobile phones, tablets, car screens, smart TVs, educational tablets, augmented reality and virtual reality devices, etc.). In this way, regardless of the operating platform or device hardware configuration, users can enjoy a consistent and stable pointer control and rebound experience, without the need for cumbersome customized settings or adjustments for different devices, effectively reducing development costs and maintenance difficulties, and greatly expanding the scope of application and market potential of the invention.

[0204] 4. Accurately achieve the effect of simulating finger sliding: Through fine optimization of the pointer movement trajectory, speed change, rebound mode and other aspects, we strive to achieve an operation effect similar to sliding a finger directly on the touch screen. In the interactive scene of augmented reality or virtual reality devices, when the user uses the matching controller to operate the virtual interface, the pointer can move and rebound in a very natural way that conforms to human operating habits, as if the user's finger is directly interacting with the virtual elements, effectively enhancing the naturalness and immersion of the operation, allowing users to more deeply integrate into the virtual interactive environment, and improving the realism and fluency of the interactive experience.

[0205] The present invention also provides a touch screen pointer control device, as described in the following embodiments. Since the principle of the device to solve the problem is similar to the touch screen pointer control method, the implementation of the device can refer to the implementation of the touch screen pointer control method, and the repeated parts will not be repeated.

[0206] The embodiment of the present invention also provides a touch screen pointer control device to solve the problem of low efficiency of manually moving the pointer back, improve the control efficiency of the touch screen, improve the user experience, and enhance cross-platform compatibility. Fig. 9 FIG. 1 is a schematic diagram of a structure of a control device for a touch screen pointer according to an embodiment of the present invention. Fig. 9 As shown, the device comprises:

[0207] The pointer drag operation data acquisition module 901 is used to acquire the pointer drag operation data performed by the user on the air operation device in real time; the touch screen displays the pointer moving in response to the pointer drag operation; the pointer drag operation data includes the pointer drag movement amount, pointer drag direction and pointer drag time interval of each frame in the drag operation process recorded frame by frame;

[0208] The reverse displacement module 902 is used to reversely displace the pointer drag movement amount for each frame of data based on the pointer drag time interval and the pointer drag direction in the order or reverse order of the frame records in the pointer drag operation data after the drag is released.

[0209] In one embodiment, it further includes:

[0210] For each frame of recorded data, determining whether the frame of recorded data is a target frame; the pointer dragging time interval corresponding to the target frame is greater than a first threshold; modifying the pointer dragging time interval of the target frame to a preset time interval; replacing the target frame in the pointer dragging operation data with the modified target frame to obtain modified pointer dragging operation data;

[0211] In the order or reverse order of the frame records in the pointer drag operation data, the reverse displacement of the pointer drag movement amount is performed on each frame of data based on the pointer drag time interval and the pointer drag direction, including:

[0212] In the corrected order or reverse order of the frame records in the pointer drag operation data, the data of each frame is subjected to a reverse displacement of the pointer drag movement amount based on the pointer drag time interval and the pointer drag direction.

[0213] In one embodiment, for each frame of recorded data, determining whether the frame of recorded data is a target frame includes:

[0214] Determine whether the pointer dragging time interval corresponding to the frame record data is greater than a first threshold;

[0215] If yes, determining whether the pointer drag movement amount corresponding to the frame record data is less than or equal to a second threshold;

[0216] If so, the frame recorded data is determined to be the target frame.

[0217] In one embodiment, for each frame of recorded data, determining whether the frame of recorded data is a target frame includes:

[0218] In the process of acquiring the pointer dragging operation data performed by the user on the air operation device, determining in real time whether the data recorded in the current frame is the target frame;

[0219] Or, after detecting that the user stops the pointer dragging operation on the air operation device, for each frame of recorded data, determine whether the frame of recorded data is a target frame.

[0220] In one embodiment, it further includes:

[0221] Merging the pointer dragging movement amounts in the recorded data of the consecutive target frames to obtain a merged frame;

[0222] Modify the target frame's pointer dragging interval to a preset interval, including:

[0223] Modify the pointer dragging time interval of the merged frame to the preset time interval;

[0224] The target frame in the pointer drag operation data is replaced with the modified target frame to obtain the corrected pointer drag operation data, including:

[0225] The corresponding continuous target frames in the pointer drag operation data are replaced with the merged frames to obtain corrected pointer drag operation data.

[0226] In one embodiment, merging the pointer dragging movement amounts in the recorded data of consecutive target frames to obtain a merged frame includes:

[0227] In the order of the timestamps of the data recorded in each frame from earliest to latest, the target frame with the earliest timestamp is obtained as the frame to be merged;

[0228] Determine whether the next frame of the frame to be merged is the target frame;

[0229] If yes, then merge the pointer drag movement of the frame to be merged and the next frame to obtain intermediate data;

[0230] The frame to be merged is replaced with the intermediate data, and the above steps of determining whether the next frame of the intermediate data is the target frame and merging are repeated until the next frame of the intermediate data is not the target frame or the intermediate data is greater than a third threshold; the obtained intermediate data is used as the merged frame.

[0231] In one embodiment, it further includes:

[0232] In the process of performing reverse displacement of the pointer drag movement amount based on the pointer drag time interval for each frame of data, if it is detected that the user performs the pointer drag operation again, then after the process of completing the reverse displacement is completed, the pointer drag operation data performed again by the user is obtained; the starting time of recording the pointer drag operation data performed again is the time when the process of completing the reverse displacement is completed;

[0233] Based on the pointer drag operation data performed again, the reverse displacement is performed again.

[0234] In one embodiment, it further includes:

[0235] During the process of performing reverse displacement of the pointer dragging movement amount based on the pointer dragging time interval for each frame of data, if it is detected that the user is performing an operation of moving the pointer, the process of performing reverse displacement of the pointer dragging movement amount based on the pointer dragging time interval is stopped, and the pointer is controlled to respond to the pointer movement operation.

[0236] An embodiment of the present invention provides an embodiment of a computer device for implementing all or part of the content of the above-mentioned touch screen pointer control method. The computer device specifically includes the following content:

[0237] Processor, memory, communication interface and bus; wherein the processor, memory and communication interface communicate with each other through the bus; the communication interface is used to realize information transmission between related devices; the computer device can be a desktop computer, a tablet computer and a mobile terminal, etc., but the present embodiment is not limited thereto. In the present embodiment, the computer device can be implemented with reference to the embodiment of the control method for implementing the touch screen pointer and the embodiment of the control device for implementing the touch screen pointer, and the contents thereof are incorporated herein and the repeated parts are not repeated.

[0238] Fig.10 FIG. 1 is a schematic block diagram of the system structure of the computer device 1000 according to an embodiment of the present application. Fig.10 As shown, the computer device 1000 may include a central processor 1001 and a memory 1002; the memory 1002 is coupled to the central processor 1001. It is worth noting that Fig.10 is exemplary; other types of structures may also be used to supplement or replace this structure to implement telecommunication functions or other functions.

[0239] In one embodiment, the control function of the touch screen pointer may be integrated into the central processing unit 1001. The central processing unit 1001 may be configured to perform the following control:

[0240] Real-time acquisition of pointer dragging operation data performed by a user on an air operation device; the touch screen displays a pointer moving in response to the pointer dragging operation; the pointer dragging operation data includes the pointer dragging movement amount, pointer dragging direction and pointer dragging time interval recorded frame by frame during the dragging operation;

[0241] For each frame of recorded data, determining whether the frame of recorded data is a target frame; the pointer drag movement amount corresponding to the target frame is less than or equal to a first threshold, and the pointer drag time interval is greater than a second threshold; modifying the pointer drag time interval of the target frame to a preset time interval; replacing the target frame in the pointer drag operation data with the modified target frame to obtain modified pointer drag operation data;

[0242] In the corrected order or reverse order of the frame records in the pointer drag operation data, the data of each frame is subjected to a reverse displacement of the pointer drag movement amount based on the pointer drag time interval and the pointer drag direction.

[0243] In another embodiment, the control device of the touch screen pointer can be configured separately from the central processing unit 1001. For example, the control device of the touch screen pointer can be configured as a chip connected to the central processing unit 1001, and the control function of the touch screen pointer is realized through the control of the central processing unit.

[0244] like Fig.10 As shown, the computer device 1000 may also include: a communication module 1003, an input unit 1004, an audio processor 1005, a display 1006, and a power supply 1007. It is worth noting that the computer device 1000 does not necessarily have to include Fig.10 In addition, the computer device 1000 may also include Fig.10 For components not shown, reference may be made to the prior art.

[0245] like Fig.10 As shown, the central processor 1001 is sometimes also referred to as a controller or an operation control, and may include a microprocessor or other processor devices and / or logic devices. The central processor 1001 receives inputs and controls the operations of various components of the computer device 1000 .

[0246] The memory 1002 may be, for example, a cache, a flash memory, a hard drive, a removable medium, a volatile memory, a non-volatile memory or other suitable devices, or one or more thereof. The memory may store the above-mentioned information related to the device, and may also store a program for executing the related information. The CPU 1001 may execute the program stored in the memory 1002 to implement information storage or processing, etc.

[0247] The input unit 1004 provides input to the CPU 1001. The input unit 1004 is, for example, a key or a touch input device. The power supply 1007 is used to provide power to the computer device 1000. The display 1006 is used to display display objects such as images and text. The display may be, for example, an LCD display, but is not limited thereto.

[0248] The memory 1002 may be a solid-state memory, such as a read-only memory (ROM), a random access memory (RAM), a SIM card, etc. It may also be a memory that saves information even when the power is off, can be selectively erased, and is provided with more data, examples of which are sometimes referred to as EPROMs, etc. The memory 1002 may also be some other type of device. The memory 1002 includes a buffer memory 1021 (sometimes referred to as a buffer). The memory 1002 may include an application / function storage unit 1022, which is used to store application programs and function programs or processes for executing the operation of the computer device 1000 through the central processor 1001.

[0249] The memory 1002 may also include a data storage unit 1023 for storing data, such as contacts, digital data, pictures, sounds, and / or any other data used by the computer device. The driver storage unit 1024 of the memory 1002 may include various drivers for the computer device for communication functions and / or for executing other functions of the computer device (such as messaging applications, address book applications, etc.).

[0250] The communication module 1003 is a transmitter / receiver that sends and receives signals via the antenna 1008. The communication module (transmitter / receiver) 1003 is coupled to the central processor 1001 to provide input signals and receive output signals, which may be the same as the case of a conventional mobile communication terminal.

[0251] Based on different communication technologies, multiple communication modules 1003 may be provided in the same computer device, such as a cellular network module, a Bluetooth module and / or a wireless local area network module, etc. The communication module (transmitter / receiver) 1003 is also coupled to a speaker 1009 and a microphone 1010 via an audio processor 1005 to provide an audio output via the speaker 1009 and receive an audio input from the microphone 1010, thereby realizing a common telecommunication function. The audio processor 1005 may include any suitable buffer, decoder, amplifier, etc. In addition, the audio processor 1005 is also coupled to the central processor 1001, so that the sound can be recorded on the local machine through the microphone 1010, and the sound stored on the local machine can be played through the speaker 1009.

[0252] An embodiment of the present invention further provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned touch screen pointer control method is implemented.

[0253] An embodiment of the present invention further provides a computer program product, which includes a computer program. When the computer program is executed by a processor, the above-mentioned touch screen pointer control method is implemented.

[0254] In an embodiment of the present invention, pointer dragging operation data performed by a user on an aerial operating device is acquired in real time; the touch screen displays a pointer that moves in response to the pointer dragging operation; the pointer dragging operation data includes the pointer dragging movement amount, pointer dragging direction, and pointer dragging time interval of each frame in the dragging operation process recorded frame by frame; after the drag is released, the pointer dragging movement amount is reversely displaced based on the pointer dragging time interval and pointer dragging direction for each frame of data in the order or reverse order of the frame records in the pointer dragging operation data. The embodiment of the present invention can completely and accurately record the user's operation trajectory by acquiring detailed data of each frame when the user performs a pointer drag operation on the air operation device in real time, including the pointer drag movement amount, direction and time interval; when the user completes the drag operation, a reverse displacement operation is performed based on the recorded data, without the need for the user to manually return the pointer, thereby avoiding tedious manual return operations and directly improving the operation efficiency; the reverse displacement of the pointer drag movement amount is performed in the order or reverse order of the frame records in the pointer drag operation data, simulating a smooth experience similar to direct touch, improving the user's satisfaction with using the air operation device to control the touch screen in a non-touch screen scenario, and significantly improving the user experience; by focusing on the relative position change of the pointer during the operation process without relying on the absolute position information of the device, a unified pointer control logic can be implemented on different operating systems and various types of mobile screen devices, providing a cross-platform, highly compatible solution, and meeting the pointer control needs of diverse devices in non-touch operation scenarios.

[0255] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0256] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0257] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0258] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0259] The specific embodiments described above further illustrate the objectives, technical solutions and beneficial effects of the present invention in detail. It should be understood that the above description is only a specific embodiment of the present invention and is not intended to limit the scope of protection of the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for controlling a touch screen pointer, characterized in that: include: Real-time acquisition of pointer dragging operation data performed by the user on the air operation device; The touch screen displays a pointer that moves in response to a pointer drag operation; The pointer dragging operation data includes the pointer dragging movement amount, pointer dragging direction and pointer dragging time interval of each frame in the dragging operation process recorded frame by frame; After the drag is released, the data of each frame is reversely displaced according to the pointer drag time interval and the pointer drag direction in the order or reverse order of the frame records in the pointer drag operation data; The reverse displacement of the pointer drag movement is performed in the order of the frame records in the pointer drag operation data, including: According to the time sequence of the frame records in the pointer drag operation data, for each frame of data, determine the corresponding delay time based on the pointer drag time interval of the frame, and adjust the pointer position in the opposite direction of the pointer drag direction of the frame to move the pointer drag movement amount; The reverse displacement of the pointer drag movement is performed in the reverse order of the frame records in the pointer drag operation data, including: According to the reverse order of the time sequence of the frame records in the pointer drag operation data, for each frame of data, the corresponding delay time is determined based on the pointer drag time interval of the frame, and the pointer position is adjusted in the opposite direction of the pointer drag direction of the frame to move the pointer drag movement amount.

2. The method according to claim 1, characterized in that Also includes: For each frame of recorded data, determining whether the frame of recorded data is a target frame; the pointer dragging time interval corresponding to the target frame is greater than a first threshold; Modify the time interval of dragging the pointer of the target frame to the preset time interval; Replacing the target frame in the pointer drag operation data with the modified target frame to obtain the corrected pointer drag operation data; In the order or reverse order of the frame records in the pointer drag operation data, the reverse displacement of the pointer drag movement amount is performed on each frame of data based on the pointer drag time interval and the pointer drag direction, including: In the corrected order or reverse order of the frame records in the pointer drag operation data, the data of each frame is subjected to a reverse displacement of the pointer drag movement amount based on the pointer drag time interval and the pointer drag direction.

3. The method according to claim 2, characterized in that For each frame of recorded data, determining whether the frame of recorded data is a target frame includes: Determine whether the pointer dragging time interval corresponding to the frame record data is greater than a first threshold; If yes, determining whether the pointer drag movement amount corresponding to the frame record data is less than or equal to a second threshold; If so, the frame recorded data is determined to be the target frame.

4. The method according to claim 2 or 3, characterized in that For each frame of recorded data, determining whether the frame of recorded data is a target frame includes: In the process of acquiring the pointer dragging operation data performed by the user on the air operation device, determining in real time whether the data recorded in the current frame is the target frame; Or, after detecting that the user stops the pointer dragging operation on the air operation device, for each frame of recorded data, determine whether the frame of recorded data is a target frame.

5. The method according to claim 2 or 3, characterized in that: Also includes: Merging the pointer dragging movement amounts in the recorded data of the consecutive target frames to obtain a merged frame; Modify the target frame's pointer dragging interval to a preset interval, including: Modify the pointer dragging time interval of the merged frame to the preset time interval; The target frame in the pointer drag operation data is replaced with the modified target frame to obtain the corrected pointer drag operation data, including: The corresponding continuous target frames in the pointer drag operation data are replaced with the merged frames to obtain corrected pointer drag operation data.

6. The method according to claim 5, characterized in that The pointer drag movement in the recorded data of the continuous target frames is merged to obtain a merged frame, including: In the order of the timestamps of the data recorded in each frame from earliest to latest, the target frame with the earliest timestamp is obtained as the frame to be merged; Determine whether the next frame of the frame to be merged is the target frame; If yes, then merge the pointer drag movement of the frame to be merged and the next frame to obtain intermediate data; The frame to be merged is replaced with the intermediate data, and the above steps of determining whether the next frame of the intermediate data is the target frame and merging are repeated until the next frame of the intermediate data is not the target frame or the intermediate data is greater than a third threshold; the obtained intermediate data is used as the merged frame.

7. The method according to claim 1, characterized in that Also includes: In the process of performing reverse displacement of the pointer drag movement amount based on the pointer drag time interval for each frame of data, if it is detected that the user performs the pointer drag operation again, then after the process of completing the reverse displacement is completed, the pointer drag operation data performed again by the user is obtained; the starting time of recording the pointer drag operation data performed again is the time when the process of completing the reverse displacement is completed; Based on the pointer drag operation data performed again, the reverse displacement is performed again.

8. The method according to claim 1, characterized in that Also includes: During the process of performing reverse displacement of the pointer dragging movement amount based on the pointer dragging time interval for each frame of data, if it is detected that the user is performing an operation of moving the pointer, the process of performing reverse displacement of the pointer dragging movement amount based on the pointer dragging time interval is stopped, and the pointer is controlled to respond to the pointer movement operation.

9. A control device for a touch screen pointer, characterized in that: include: The pointer drag operation data acquisition module is used to acquire the pointer drag operation data performed by the user on the air operation device in real time; The touch screen displays a pointer moving in response to a pointer drag operation; the pointer drag operation data includes a pointer drag movement amount, a pointer drag direction, and a pointer drag time interval recorded frame by frame during the drag operation; A reverse displacement module, for performing reverse displacement of the pointer drag movement amount on each frame of data based on the pointer drag time interval and the pointer drag direction in the order or reverse order of the frame records in the pointer drag operation data after the drag is released; The reverse displacement module is specifically used to: perform reverse displacement of the pointer drag movement amount in the order of frame records in the pointer drag operation data, including: according to the time sequence of the frame records in the pointer drag operation data, determine the corresponding delay time based on the pointer drag time interval of the frame for each frame of data in turn, and adjust the pointer position in the opposite direction of the pointer drag direction of the frame to move the pointer drag movement amount; perform reverse displacement of the pointer drag movement amount in the reverse order of the frame records in the pointer drag operation data, including: according to the reverse order of the time sequence of the frame records in the pointer drag operation data, determine the corresponding delay time based on the pointer drag time interval of the frame for each frame of data in turn, and adjust the pointer position in the opposite direction of the pointer drag direction of the frame to move the pointer drag movement amount.

10. A computer device comprising a memory, a processor and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method according to any one of claims 1 to 8 is implemented.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

12. A computer program product, characterized in that The computer program product comprises a computer program, and when the computer program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images