Implementation method, device and equipment for floating touch, medium and product

Abstract

The invention provides a floating touch implementation method and device, equipment, a medium and a product, and the method comprises the steps: building a reference coordinate system and a touch coordinate system, enabling an original point of the reference coordinate system to be located on a display plane of projection equipment, enabling an original point of the touch coordinate system to be located on touch equipment worn by a user, or a fingertip position when the user executes the suspension touch operation by using the touch equipment; determining suspension touch data related to the suspension touch operation to determine a suspension touch position, and mapping the suspension touch position to a three-dimensional image suspension display space corresponding to the reference coordinate system; identifying the type of the floating touch operation according to the floating touch data; and based on the suspension touch position and type, matching with a target display position in the three-dimensional image suspension display space to realize suspension touch interaction. Therefore, correlation between touch control and display can be achieved through calculation of the relative poses of the two coordinate systems, an efficient, flexible and accurate suspension touch control mode is provided, and the interaction experience of a user is improved.

Description

Technical Field

[0001] The embodiments of the present application relate to the technical field of floating projection display, and in particular, to a method, device, equipment, medium and product for realizing floating touch control. Background Art

[0002] Currently, in the field of touch control technology, current traditional touch control solutions mainly include technologies such as capacitive, resistive, infrared and vision-based. Among them, capacitive and resistive are mainly used for flat display and touch control, but there are obvious defects in the field of three-dimensional floating touch control. Although the infrared type can achieve floating touch control by arranging infrared sensors, its three-dimensional structure design results in a large volume of the device, and its compatibility between flat display and floating touch control is poor. Although the vision detection solution supports flat and floating touch control, its required computing power is too high, making it unsuitable for portable display devices. In addition, the solution for obtaining the touch position by a single pose sensor is based on the world coordinate system and cannot adjust the touch position accordingly when the floating projection position changes, resulting in an impact on the accuracy of touch operations.

[0003] In summary, there are still problems in the current floating touch control field, such as low accuracy of floating touch control operations, too high computing power required for floating touch control, and being unsuitable for portable display devices. Summary of the Invention

[0004] The embodiments of the present application provide a method, device, equipment, medium and product for realizing floating touch control to solve the problems in the current floating touch control field, such as low accuracy of floating touch control operations, too high computing power required for floating touch control, and being unsuitable for portable display devices.

[0005] To solve the above technical problems, the present application is implemented as follows:

[0006] In a first aspect, the embodiments of the present application provide a method for realizing floating touch control, the method including:

[0007] Establish a reference coordinate system and a touch coordinate system, wherein the origin of the reference coordinate system is located on the display plane of the projection device, and the origin of the touch coordinate system is located on the touch device worn by the user or the fingertip position when the user performs a floating touch control operation using the touch device;

[0008] Determine floating touch control data related to the floating touch control operation, wherein the floating touch control data includes: the three-axis offset angles of the touch coordinate system relative to the reference coordinate system, and the position coordinates of the origin of the touch coordinate system relative to the origin of the reference coordinate system;

[0009] Determine the hovering touch position according to the hovering touch data, and map the hovering touch position to the hovering display space of the three-dimensional image corresponding to the reference coordinate system;

[0010] Identify the type of the hovering touch operation according to the hovering touch data;

[0011] Match with the target display position in the hovering display space of the three-dimensional image based on the hovering touch position and the identified type of the hovering touch operation, so as to realize hovering touch interaction.

[0012] Optionally, the projection device is built-in with a first electronic gyroscope, and the touch device is built-in with a second electronic gyroscope;

[0013] Determining the hovering touch data related to the hovering touch operation includes:

[0014] Determine the first initial pose of the reference coordinate system and the first current pose corresponding to the hovering touch operation based on the first electronic gyroscope;

[0015] Determine the second initial pose of the touch coordinate system and the second current pose corresponding to the hovering touch operation based on the second electronic gyroscope;

[0016] Determine the hovering touch data based on the first initial pose, the first current pose, the second initial pose, and the second current pose.

[0017] Optionally, the touch device is built-in with a second electronic gyroscope. Identifying the type of the hovering touch operation according to the hovering touch data includes:

[0018] Collect the accelerations of the X-axis, Y-axis, and Z-axis of the touch coordinate system based on the second electronic gyroscope;

[0019] Perform vector synthesis on the accelerations of the X-axis, Y-axis, and Z-axis to obtain the total acceleration of the hovering touch operation;

[0020] Identify the type of the hovering touch operation based on the hovering touch data and the total acceleration of the hovering touch operation.

[0021] Optionally, the type includes at least one of the following: click operation, slide operation, long press operation. Identifying the type of the hovering touch operation based on the hovering touch data and the total acceleration of the hovering touch operation includes:

[0022] Perform time series analysis on the total acceleration to extract the peak value, duration, and change trend of the total acceleration;

[0023] If the peak value of the total acceleration exceeds a first threshold and the duration is less than a second threshold, then the type is identified as the click operation;

[0024] If the change trend is continuous fluctuation and the change range of the position coordinates in the hovering touch data exceeds a third threshold, then the type is identified as the swipe operation;

[0025] If the change trend is stable and the change range of the position coordinates in the hovering touch data is less than a fourth threshold, then the type is identified as the long - press operation.

[0026] Optionally, determining the hovering touch data related to the hovering touch operation includes:

[0027] During the process of the user performing the hovering touch operation, determining the hovering touch data in real - time, or determining the hovering touch data once every preset time.

[0028] In a second aspect, an embodiment of the present application provides an implementation device for hovering touch, and the device includes:

[0029] A coordinate system establishment module, configured to establish a reference coordinate system and a touch coordinate system, wherein the origin of the reference coordinate system is located on the display plane of the projection device, and the origin of the touch coordinate system is located on the touch device worn by the user or the fingertip position when the user performs the hovering touch operation using the touch device;

[0030] An execution module, configured to determine the hovering touch data related to the hovering touch operation, wherein the hovering touch data includes: the three - axis offset angles of the touch coordinate system relative to the reference coordinate system, and the position coordinates of the origin of the touch coordinate system relative to the origin of the reference coordinate system;

[0031] Determine the hovering touch position according to the hovering touch data, and map the hovering touch position to a three - dimensional image hovering display space corresponding to the reference coordinate system;

[0032] Identify the type of the hovering touch operation according to the hovering touch data;

[0033] Based on the hovering touch position and the identified type of the hovering touch operation, match with a target display position in the three - dimensional image hovering display space to implement hovering touch interaction.

[0034] Optionally, the projection device is built - in with a first electronic gyroscope, and the touch device is built - in with a second electronic gyroscope;

[0035] The execution module is further configured to determine a first initial pose of the reference coordinate system and a first current pose corresponding to the hovering touch operation based on the first electronic gyroscope;

[0036] determine a second initial pose of the touch coordinate system and a second current pose corresponding to the hovering touch operation based on the second electronic gyroscope;

[0037] determine the hovering touch data based on the first initial pose, the first current pose, the second initial pose, and the second current pose.

[0038] Optionally, the touch device is built with a second electronic gyroscope, and the execution module is further configured to collect accelerations of the X-axis, Y-axis, and Z-axis of the touch coordinate system based on the second electronic gyroscope;

[0039] perform vector synthesis on the accelerations of the X-axis, Y-axis, and Z-axis to obtain a total acceleration of the hovering touch operation;

[0040] identify the type of the hovering touch operation based on the hovering touch data and the total acceleration of the hovering touch operation.

[0041] Optionally, the type includes at least one of the following: click operation, slide operation, long press operation. The execution module is further configured to perform time series analysis on the total acceleration to extract the peak value, duration, and change trend of the total acceleration;

[0042] If the peak value of the total acceleration exceeds a first threshold and the duration is less than a second threshold, then identify the type as the click operation;

[0043] If the change trend is continuous fluctuation and the change range of the position coordinates in the hovering touch data exceeds a third threshold, then identify the type as the slide operation;

[0044] If the change trend is stable and the change range of the position coordinates in the hovering touch data is less than a fourth threshold, then identify the type as the long press operation.

[0045] Optionally, the execution module is further configured to determine the hovering touch data in real time during the process of the user performing the hovering touch operation, or determine the hovering touch data once every preset time.

[0046] In a third aspect, an embodiment of the present application provides a network device, including: a processor, a memory, and a program stored on the memory and executable on the processor. When the program is executed by the processor, the steps of a method for implementing a hovering touch as described in the first aspect are implemented.

[0047] Fourth aspect, an embodiment of the present application provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of an implementation method of floating touch as described in the first aspect are realized.

[0048] Fifth aspect, an embodiment of the present application provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, the steps of an implementation method of floating touch as described in the first aspect are realized.

[0049] In the embodiment of the present application, the floating touch data includes: the three-axis offset angles of the touch coordinate system relative to the reference coordinate system, and the position coordinates of the origin of the touch coordinate system relative to the origin of the reference coordinate system. Thus, the association between touch and display can be realized through the relative pose calculation of the two coordinate systems, so as to realize floating touch. Compared with the visual detection scheme, the method shown in the embodiment of the present application does not require too high computing power and is applicable to portable display devices; compared with the scheme of obtaining the touch position by a single pose sensor, which results in low accuracy of touch operations, the method shown in the embodiment of the present application provides an efficient, flexible and accurate touch method, improving the user's interaction experience. Description of the Drawings

[0050] By reading the following detailed description of the preferred embodiments, various other advantages and benefits will become clear to those of ordinary skill in the art. The drawings are only for the purpose of showing the preferred embodiments and are not considered to be a limitation of the present application. Moreover, throughout the drawings, the same reference numerals are used to represent the same components. In the drawings:

[0051] Figure 1 is a flowchart of an implementation method of floating touch provided by an embodiment of the present application;

[0052] Figure 2 is a schematic diagram of a reference coordinate system and a touch coordinate system provided by an embodiment of the present application;

[0053] Figure 3 is a schematic diagram of the pose change of a reference coordinate system provided by an embodiment of the present application;

[0054] Figure 4 is a flowchart of an implementation method of floating touch provided by an embodiment of the present application;

[0055] Figure 5 is a flowchart of an implementation method of floating touch provided by an embodiment of the present application;

[0056] Figure 6A is a schematic diagram of a reference coordinate system and a touch coordinate system during the initialization stage provided by an embodiment of the present application;

[0057] Figure 6B Schematic diagrams of a reference coordinate system and a touch coordinate system when the pose of the touch coordinate system provided in the embodiment of the present application changes;

[0058] Figure 6C Schematic diagrams of a reference coordinate system and a touch coordinate system when the poses of both the touch coordinate system and the reference coordinate system provided in the embodiment of the present application change;

[0059] Figure 7 Block diagram of a structure of an implementation device for floating touch provided in the embodiment of the present application;

[0060] Figure 8 Block diagram of a structure of a network device provided in the embodiment of the present application. Detailed implementation manners

[0061] Next, the technical solutions in the embodiments of the present application will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present application. Obviously, the described embodiments are some, but not all, of the embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without making creative efforts shall fall within the protection scope of the present application.

[0062] Figure 1 Illustrates an implementation method for floating touch provided in the embodiment of the present application, as Figure 1 shown, the method includes:

[0063] Step S101, establish a reference coordinate system and a touch coordinate system;

[0064] Among them, the origin of the reference coordinate system is located on the display plane of the projection device, and the origin of the touch coordinate system is located on the touch device worn by the user, or the fingertip position when the user performs a floating touch operation using the touch device;

[0065] Step S102, determine floating touch data related to the floating touch operation, where the floating touch data includes: the three-axis offset angles of the touch coordinate system relative to the reference coordinate system, and the position coordinates of the origin of the touch coordinate system relative to the origin of the reference coordinate system;

[0066] Step S103, determine the floating touch position according to the floating touch data, and map the floating touch position to a three-dimensional image floating display space corresponding to the reference coordinate system;

[0067] Step S104, identify the type of the floating touch operation according to the floating touch data;

[0068] Step S105: Based on the floating touch position and the type of the recognized floating touch operation, match with the target display position in the three-dimensional image floating display space to achieve floating touch interaction.

[0069] In step S101, as Figure 2 shown, first define two coordinate systems: a reference coordinate system and a touch coordinate system. The origin of the reference coordinate system is set on the display plane of the projection device to ensure that the positioning of all floating display contents is based on this point. Exemplarily, the origin can be located at the geometric center, the upper left corner, the lower left corner, etc. of the display plane of the projection device. And the origin of the touch coordinate system is located on the touch device worn by the user or at the fingertip position when the user performs a floating touch operation. The touch device includes at least one of the following: a wearable ring, a glove, or a handheld controller. Thereby, the relationship between the user's touch operation and the projection display content can be clearly distinguished, providing a basis for the relative pose calculation (calculation of floating touch data) between the two coordinate systems.

[0070] And it should be noted that the reference coordinate system represents the pose of the planar display or the floating projection interface, and the change of the pose data of the reference coordinate system represents the change of the position and pose of the projection object (as Figure 3 shown), that is, if the touch position remains unchanged and the pose of the touch reference plane, or the display plane, or the floating display interface changes, the position touched on the floating display interface will also change accordingly.

[0071] In step S102, it is necessary to obtain data related to the floating touch operation, mainly including two aspects: the three-axis offset angles of the touch coordinate system relative to the reference coordinate system (i.e., pose information, which can include: pitch angle, yaw angle, roll angle) and the position coordinates of the origin of the touch coordinate system relative to the origin of the reference coordinate system. These data can reflect the position and direction of the touch device in the three-dimensional space, providing necessary information for the calculation of the floating touch position, and by obtaining detailed floating touch data, accurate recognition of the user's touch action can be achieved, enhancing the flexibility and accuracy of the touch.

[0072] And in a possible implementation manner, determining the floating touch data related to the floating touch operation includes: during the process of the user performing the floating touch operation, determining the floating touch data in real time, or determining the floating touch data every preset time. That is to say, not only can the floating touch data be determined in real time, but also the floating touch data can be determined at a fixed time, a fixed frequency, or a fixed number of acquisitions. For example, it is collected once every 0.1 second, for example, 10 times per second (10 Hz), for example, 5 consecutive floating touch data are collected as a group of samples.

[0073] In a possible implementation, the projection device is built-in with a first electronic gyroscope, and the touch device is built-in with a second electronic gyroscope; Step S102, determining the hovering touch data related to the hovering touch operation includes: determining the first initial pose of the reference coordinate system and the first current pose corresponding to the hovering touch operation based on the first electronic gyroscope; determining the second initial pose of the touch coordinate system and the second current pose corresponding to the hovering touch operation based on the second electronic gyroscope; determining the hovering touch data based on the first initial pose, the first current pose, the second initial pose, and the second current pose.

[0074] Among them, an electronic gyroscope is a device that uses an electronic physical effect to measure angular velocity and can obtain its attitude position data through a calculus algorithm. In the method shown in the embodiments of the present application, two electronic gyroscopes are used, which are respectively used for attitude measurement of the reference coordinate system and the touch coordinate system. And the application of the electronic gyroscope can improve touch accuracy.

[0075] First, the first electronic gyroscope is used to measure the pose of the reference coordinate system. When the display device is started, the first electronic gyroscope records the initial pose of the reference coordinate system, which is usually carried out when the device is in a stationary state to ensure the accuracy of the data. When the user performs a hovering touch operation, the first electronic gyroscope real-time monitors the current pose of the reference coordinate system, reflecting any rotation, tilt, or position movement that may occur during the operation (the change of the reference coordinate system after the position movement is as Figure 3 shown).

[0076] The second electronic gyroscope is used to measure the pose of the touch coordinate system. After the user wears the touch device, or wears and turns on the touch device, the second electronic gyroscope records the initial pose of the touch coordinate system, which is also carried out in a stationary state to ensure accuracy. When the user performs a hovering touch operation, the second electronic gyroscope real-time monitors the current pose of the touch coordinate system, reflecting the user's gesture changes. Through the above steps, the following four key data points can be obtained: the first initial pose, the first current pose, the second initial pose, and the second current pose. Then, the hovering touch data can be calculated through the four key data points. Specifically, by comparing the initial pose and the current pose, the pose changes of the reference coordinate system and the touch coordinate system can be calculated. These pose changes are usually represented by three-axis offset angles (such as rotation angles around the X, Y, and Z axes). At the same time, it is also necessary to calculate the position coordinates of the origin of the touch coordinate system relative to the origin of the reference coordinate system. Finally, the hovering touch data can be generated according to the above calculation results. The hovering touch data provides a solid foundation for subsequent touch position calculation, operation recognition, and interaction matching, ensuring that the user's hovering touch operation can obtain timely and accurate feedback and response, thereby enhancing the user's overall experience.

[0077] In step S103, the hovering touch data obtained in the previous step can be used to calculate the hovering touch position (hovering touch point), and map it to the three-dimensional image hovering display space corresponding to the reference coordinate system. Thus, it can be ensured that the actual touch position of the user can be correctly reflected on the hovering display content (for example: clicking a button on the hovering display content), improving the intuitiveness and response speed of the interaction.

[0078] In step S104, the hovering touch data needs to be analyzed to identify the type of hovering touch operation performed by the user, such as clicking, swiping, zooming, etc. By identifying the hovering touch data, the system can understand the user's intention and thus make corresponding feedback to enhance the user experience (the subsequent implementation method will specifically describe this step).

[0079] In step S105, it is necessary to match the user's hovering touch position with the target display position in the three-dimensional image hovering display space. First, it is necessary to identify and obtain all possible target display positions in the three-dimensional image hovering display space. These positions include buttons that the user hopes to click, objects to be selected, or elements that require other interactions. Each target display position has its fixed coordinates in the reference coordinate system. In the previous steps, the hovering touch position of the user has been calculated through the hovering touch data. This position is determined based on the position of the user's gesture or fingertip in the touch coordinate system and is mapped and converted to the reference coordinate system. Thus, both the target display position and the hovering touch position are based on the reference coordinate system, enabling the matching of the two and performing corresponding interaction operations. These operations may include: if the matched target is a button, triggering the click event of the button; if the matched target is a selectable object, updating the state of the object (such as highlighting, selecting, etc.); if the user's touch intention is to execute a certain command (such as rotating, zooming, etc.), corresponding processing will be performed according to the touch position and the operation type. And to enhance the user experience, immediate visual or tactile feedback can also be provided after the matching is successful. For example, after the user touches the target, the target can change color, size, or produce a sound prompt to confirm that the operation has been recognized and executed.

[0080] In summary, the method shown in the embodiment of the present application realizes precise positioning, flexible recognition, and efficient interaction of hovering touch. When the user performs a hovering touch operation, real-time and accurate feedback can be obtained, greatly improving the interaction experience. Moreover, the solution provided by the embodiment of the present application is not only applicable to flat displays, but also can effectively support three-dimensional hovering displays, and is also applicable to portable display devices, overcoming the limitations of traditional touch solutions and providing an efficient and portable hovering touch solution.

[0081] In a possible implementation manner, the touch device is built-in with a second electronic gyroscope, such as Figure 4As shown, step S104. Identifying the type of the floating touch operation according to the floating touch data includes:

[0082] Step S1041. Based on the second electronic gyroscope, collect the accelerations of the X-axis, Y-axis, and Z-axis of the touch coordinate system.

[0083] Step S1042. Perform vector synthesis on the accelerations of the X-axis, Y-axis, and Z-axis to obtain the total acceleration of the floating touch operation.

[0084] Step S1043. Identify the type of the floating touch operation based on the floating touch data and the total acceleration of the floating touch operation.

[0085] It should be noted that the second electronic gyroscope can not only measure the angular velocity, but also integrate an acceleration sensor for detecting the acceleration in the touch coordinate system. When the user performs a floating touch operation, it can collect the acceleration data of the X-axis, Y-axis, and Z-axis of the touch coordinate system in real time or at intervals, and perform vector synthesis on the accelerations of the X-axis, Y-axis, and Z-axis to obtain the total acceleration of the floating touch operation. Through the synthesized total acceleration, a comprehensive acceleration index can be obtained, and this index can reflect the user's overall touch force and motion state. That is to say, the magnitude of the total acceleration can help determine whether the user's touch operation is a light touch, a slide, or a rapid movement, etc. Then, the floating touch data (such as attitude information, position coordinates) can be combined with the total acceleration calculated in the previous step to identify the type of the floating touch operation.

[0086] In a possible implementation manner, the type includes at least one of the following: click operation, slide operation, long-press operation. Step S1043. Identifying the type of the floating touch operation based on the floating touch data and the total acceleration of the floating touch operation includes: performing time series analysis on the total acceleration to extract the peak value, duration, and change trend of the total acceleration; if the peak value of the total acceleration exceeds the first threshold and the duration is less than the second threshold, then identify the type as a click operation; if the change trend is continuous fluctuation and the change range of the position coordinates in the floating touch data exceeds the third threshold, then identify the type as a slide operation; if the change trend is stable and the change range of the position coordinates in the floating touch data is less than the fourth threshold, then identify the type as a long-press operation.

[0087] It should be noted that, first of all, the total acceleration will be analyzed to extract key features such as peak value, duration, and change trend. The peak value represents the maximum force applied by the user, the duration reflects the length of time the force is maintained, and the change trend shows the fluctuation pattern of the acceleration. When identifying the operation type, the system sets some thresholds to judge the user's intention. If the peak value of the total acceleration exceeds the first threshold and the duration is less than the second threshold, the system will identify it as a "click operation", which usually means that the user quickly and clearly triggers a certain function; on the contrary, if the total acceleration shows a continuous fluctuating change trend and the change range of the position coordinates in the floating touch data exceeds the third threshold, the system will identify it as a "swipe operation", which indicates that the user is performing continuous gesture movements; and if the change trend of the total acceleration is stable and the change range of the position coordinates is less than the fourth threshold, the system will identify it as a "long press operation", which means that the user stays still at a certain position, possibly to call up more options. Through this comprehensive analysis, the floating touch operation type of the user can be accurately identified, so as to achieve a fast and accurate response and improve the user's interaction experience.

[0088] In summary, the embodiment of the present application uses two electronic gyroscopes to implement floating touch, which can be compatible with plane and floating displays. By obtaining the poses of the reference coordinate system and the touch coordinate system, the corresponding touch position and action can be identified; and as the pose of the reference coordinate system changes, the touch position and touch action can still be obtained through relative position calculation and comparison; in addition, the acceleration change of the touch position can be detected by the electronic gyroscope, and the type of touch action can be determined accordingly. Compared with the visual detection scheme, the method shown in the embodiment of the present application does not require high computing power and can be applied to portable display devices; compared with the scheme of obtaining the touch position by a single pose sensor, which results in low accuracy of touch operations, the method shown in the embodiment of the present application provides an efficient, flexible and accurate touch method, improving the user's interaction experience.

[0089] Generally describe the floating touch solution provided by the embodiment of the present application. The solution shown in the embodiment of the present application provides two electronic gyroscopes. Thus, firstly, the triaxial space data can be detected by the electronic gyroscope in the touch device worn on the user's hand, so that the touch action of the user and the touch position relative to the spatial image can be analyzed, and the role of the other gyroscope is to locate the display base (projection device) to the reference spatial position. As the display base moves (such as Figure 3As shown in the figure, the operations of the user can also be matched with the spatially displayed image in real time. Second, compared with a single gyroscope, the touch control method is optimized. The touch control solution based on a single gyroscope only simply collects the spatial data of the gyroscope. However, in the method shown in the embodiments of the present application, the data of two gyroscopes can be collected in real time, and the spatial data collected by the gyroscope in the touch control device worn by the user is converted into relative coordinates with reference to the reference coordinate system to match the movement of the spatial image caused by the movement of the display base. Therefore, it can be applied to real-time touch control when future floating three-dimensional displays are used as portable or mobile products. If a single gyroscope is used, it is impossible to ensure accurate matching of the touch position after the portable product moves.

[0090] The specific touch control method can refer to Figure 5 , that is, collect the floating touch control data of two gyroscopes for comparison, so as to calculate the relative position and attitude between the two. Specifically, in the initialization stage (as Figure 6A shown), the poses of the reference coordinate system and the touch control coordinate system are fixed, and a coordinate pose initial value (the above-mentioned second initial pose) can be directly given to the touch control coordinate system, that is, the relative coordinate position of the touch control coordinate system with respect to the reference coordinate system. After the initialization is completed, the relative position of the two coordinate systems can be obtained through algorithm calculation. For example: the initial pose of the reference coordinate system is O(0, 0, 0), θ(0, 0, 0), and the initial pose of the touch control coordinate system is O'(a, b, c), θ'(0, 0, 0). Since the orientations of the three axes are the same during initialization, when the pose of the touch control coordinate system changes (as Figure 6B shown), the positions and poses of the two coordinate systems are O(0, 0, 0), θ(0, 0, 0); O'(a + x2, b + y2, C + z2), θ'(α2, β2, γ2). When the poses of both the reference coordinate system and the touch control coordinate system change (as Figure 6C shown, and Figure 6C in, a(xy), a(zy), and a(xz) are the accelerations of the floating touch control operations in the XY plane, ZY plane, and XZ plane respectively), the positions and poses of the two coordinate systems are O(x1, y1, z1), θ(α1, β1, γ1); O'(a + x2 - x1, b + y2 - y1, C + z2 - z1), θ'(α2 - α1, β2 - β1, γ2 - γ1).

[0091] Moreover, since the floating display is three-dimensional, the image of the three-dimensional floating display is also referenced to the reference coordinate system, which is the same as the reference of the touch position. Therefore, by comparing the pose data of the three-dimensional display image and the touch position, real-time matching of touch and three-dimensional floating display can be achieved. In addition, the acceleration of the floating touch operation can be collected by a dual-electronic gyroscope and synthesized, and then the type of the floating touch operation can be determined by combining the acceleration with the pose change.

[0092] In summary, the embodiment of the present application uses a dual-electronic gyroscope to implement floating touch, which can be compatible with planar and floating displays. By obtaining the poses of the reference coordinate system and the touch coordinate system, the corresponding touch position and action can be recognized. And as the pose of the reference coordinate system changes, the touch position and touch action can still be obtained through relative position calculation and comparison, as well as the follow-up of the touch action to the three-dimensional floating image. Thus, an efficient, flexible and accurate touch method is provided, improving the user's interaction experience.

[0093] Figure 7 Fig. 70 shows an implementation device 70 for floating touch provided according to an embodiment of the present application. The device 70 includes:

[0094] A coordinate system establishment module 701, configured to establish a reference coordinate system and a touch coordinate system, wherein the origin of the reference coordinate system is located on the display plane of the projection device, and the origin of the touch coordinate system is located on the touch device worn by the user or the fingertip position when the user performs a floating touch operation using the touch device;

[0095] An execution module 702, configured to determine floating touch data related to the floating touch operation, wherein the floating touch data includes: the three-axis offset angles of the touch coordinate system relative to the reference coordinate system, and the position coordinates of the origin of the touch coordinate system relative to the origin of the reference coordinate system;

[0096] Determine the floating touch position according to the floating touch data, and map the floating touch position to the three-dimensional image floating display space corresponding to the reference coordinate system;

[0097] Identify the type of the floating touch operation according to the floating touch data;

[0098] Match with the target display position in the three-dimensional image floating display space based on the floating touch position and the identified type of the floating touch operation to achieve floating touch interaction.

[0099] In a possible implementation manner, the projection device is built-in with a first electronic gyroscope, and the touch device is built-in with a second electronic gyroscope;

[0100] The execution module 702 is further configured to determine the first initial pose of the reference coordinate system and the first current pose corresponding to the floating touch operation based on the first electronic gyroscope;

[0101] Determine the second initial pose of the touch coordinate system and the second current pose corresponding to the hovering touch operation based on the second electronic gyroscope;

[0102] Determine the hovering touch data based on the first initial pose, the first current pose, the second initial pose, and the second current pose.

[0103] In a possible implementation, the touch device is built with a second electronic gyroscope, and the execution module 702 is further configured to collect the accelerations of the X-axis, Y-axis, and Z-axis of the touch coordinate system based on the second electronic gyroscope;

[0104] Perform vector synthesis on the accelerations of the X-axis, Y-axis, and Z-axis to obtain the total acceleration of the hovering touch operation;

[0105] Identify the type of the hovering touch operation based on the hovering touch data and the total acceleration of the hovering touch operation.

[0106] In a possible implementation, the type includes at least one of the following: click operation, slide operation, long press operation. The execution module 702 is further configured to perform time series analysis on the total acceleration to extract the peak value, duration, and change trend of the total acceleration;

[0107] If the peak value of the total acceleration exceeds the first threshold and the duration is less than the second threshold, then identify the type as a click operation;

[0108] If the change trend is continuous fluctuation and the change range of the position coordinates in the hovering touch data exceeds the third threshold, then identify the type as a slide operation;

[0109] If the change trend is stable and the change range of the position coordinates in the hovering touch data is less than the fourth threshold, then identify the type as a long press operation.

[0110] In a possible implementation, the execution module 702 is further configured to determine the hovering touch data in real time during the process of the user performing the hovering touch operation, or determine the hovering touch data once every preset time.

[0111] In summary, in the embodiments of the present application, the association between touch and display can be realized through the relative pose calculation of two coordinate systems, so as to realize hovering touch. Moreover, compared with the visual detection scheme, the method shown in the embodiments of the present application does not require high computing power and can be applied to portable display devices; compared with the scheme of obtaining the touch position by a single pose sensor, which results in low accuracy of touch operations, the method shown in the embodiments of the present application provides an efficient, flexible, and accurate touch method, improving the user's interaction experience.

[0112] The embodiments of the present application provide a network device 80, such asFigure 8 As shown in Figure 8 , the network device 80 includes: a processor 801, a memory 802, and a program stored on the memory 802 and executable on the processor 801. When the program is executed by the processor 801, it implements the steps of the method for realizing floating touch as shown in the above embodiments.

[0113] The embodiment of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the method for realizing floating touch as shown in the above embodiments, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here. Among them, the computer-readable storage medium is, for example, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disc, etc.

[0114] The embodiment of the present application further provides a computer program product, including computer instructions. When the computer instructions are executed by a processor, they implement the steps of the method for realizing floating touch as shown in the above embodiments, and can achieve the same technical effects. To avoid repetition, it will not be elaborated here.

[0115] It should be noted that in this article, the term "including", "comprising", or any other variant thereof is intended to cover non-exclusive inclusion, so that a process, method, article, or device including a series of elements not only includes those elements, but also includes other elements not expressly listed, or further includes elements inherent to such process, method, article, or device. Without further limitations, an element defined by the statement "including a..." does not exclude the existence of additional identical elements in the process, method, article, or device including that element.

[0116] Through the description of the above embodiments, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases, the former is a better implementation method. Based on such an understanding, the technical solution of the present application, in essence, or the part that contributes to the prior art, can be embodied in the form of a software product. The computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disc), and includes several instructions for causing a terminal (which can be a mobile phone, a computer, a server, an air conditioner, or a network device, etc.) to execute the methods described in the various embodiments of the present application.

[0117] The embodiments of the present application have been described above in conjunction with the accompanying drawings. However, the present application is not limited to the above specific embodiments. The above specific embodiments are merely illustrative rather than restrictive. Under the inspiration of the present application, those of ordinary skill in the art can also make many forms without departing from the purpose of the present application and the scope protected by the claims, and all of them fall within the protection scope of the present application.

Claims

1. A method for realizing floating touch control, characterized in that The method includes: Establishing a reference coordinate system and a touch coordinate system, wherein the origin of the reference coordinate system is located on the display plane of the projection device, and the origin of the touch coordinate system is located on the touch device worn by the user or at the fingertip position when the user performs a hovering touch operation using the touch device; Determining hovering touch data related to the hovering touch operation, wherein the hovering touch data includes: the three-axis offset angles of the touch coordinate system relative to the reference coordinate system, and the position coordinates of the origin of the touch coordinate system relative to the origin of the reference coordinate system; Determining a hovering touch position according to the hovering touch data and mapping the hovering touch position to a three-dimensional image hovering display space corresponding to the reference coordinate system; Identifying the type of the hovering touch operation according to the hovering touch data; Based on the hovering touch position and the identified type of the hovering touch operation, matching with a target display position in the three-dimensional image hovering display space to achieve hovering touch interaction.

2. The method according to claim 1, characterized in that, The projection device is built-in with a first electronic gyroscope, and the touch device is built-in with a second electronic gyroscope; Determining the hovering touch data related to the hovering touch operation includes: Determining a first initial pose of the reference coordinate system and a first current pose corresponding to the hovering touch operation based on the first electronic gyroscope; Determining a second initial pose of the touch coordinate system and a second current pose corresponding to the hovering touch operation based on the second electronic gyroscope; Determining the hovering touch data based on the first initial pose, the first current pose, the second initial pose, and the second current pose.

3. The method according to claim 1, characterized in that, The touch device is built-in with a second electronic gyroscope. Identifying the type of the hovering touch operation according to the hovering touch data includes: Collecting the accelerations of the X-axis, Y-axis, and Z-axis of the touch coordinate system based on the second electronic gyroscope; Performing vector synthesis on the accelerations of the X-axis, Y-axis, and Z-axis to obtain the total acceleration of the hovering touch operation; Identifying the type of the hovering touch operation based on the hovering touch data and the total acceleration of the hovering touch operation.

4. The method according to claim 3, characterized in that, The type includes at least one of the following: click operation, slide operation, long press operation. Identifying the type of the hovering touch operation based on the hovering touch data and the total acceleration of the hovering touch operation includes: Performing time series analysis on the total acceleration to extract the peak value, duration, and change trend of the total acceleration; If the peak value of the total acceleration exceeds a first threshold and the duration is less than a second threshold, then identifying the type as the click operation; If the change trend is continuous fluctuation and the change range of the position coordinates in the hovering touch data exceeds a third threshold, then identifying the type as the slide operation; If the change trend is stable and the change range of the position coordinates in the hovering touch data is less than a fourth threshold, then identifying the type as the long press operation.

5. The method according to any one of claims 1-4, characterized in that, Determining the hovering touch data related to the hovering touch operation includes: During the process of the user performing the hovering touch operation, the hovering touch data is determined in real time, or the hovering touch data is determined once every preset time.

6. An implementation device for floating touch control, characterized in that, The device includes: a coordinate system establishment module, configured to establish a reference coordinate system and a touch coordinate system, wherein the origin of the reference coordinate system is located on the display plane of the projection device, and the origin of the touch coordinate system is located on the touch device worn by the user, or the fingertip position when the user performs the hovering touch operation using the touch device; an execution module, configured to determine hovering touch data related to the hovering touch operation, wherein the hovering touch data includes: the three-axis offset angles of the touch coordinate system relative to the reference coordinate system, and the position coordinates of the origin of the touch coordinate system relative to the origin of the reference coordinate system; determine the hovering touch position according to the hovering touch data, and map the hovering touch position to a three-dimensional image hovering display space corresponding to the reference coordinate system; identify the type of the hovering touch operation according to the hovering touch data; match with a target display position in the three-dimensional image hovering display space based on the hovering touch position and the identified type of the hovering touch operation, so as to implement hovering touch interaction.

7. The device according to claim 6, characterized in that, The projection device is built-in with a first electronic gyroscope, and the touch device is built-in with a second electronic gyroscope; The execution module is further configured to determine a first initial pose of the reference coordinate system and a first current pose corresponding to the hovering touch operation based on the first electronic gyroscope; determine a second initial pose of the touch coordinate system and a second current pose corresponding to the hovering touch operation based on the second electronic gyroscope; determine the hovering touch data based on the first initial pose, the first current pose, the second initial pose, and the second current pose.

8. A network device, characterized in that, including: a processor, a memory, and a program stored on the memory and executable on the processor, and when the program is executed by the processor, the steps of a method for implementing hovering touch according to any one of claims 1-5 are implemented.

9. A computer-readable storage medium, characterized in that, A computer program is stored on the computer-readable storage medium, and when the computer program is executed by a processor, the steps of a method for implementing hovering touch according to any one of claims 1-5 are implemented.

10. A computer program product, characterized in that, including computer instructions, and when the computer instructions are executed by a processor, the steps of a method for implementing hovering touch according to any one of claims 1-5 are implemented.

Images