Input system

Abstract

An input system having a position input device configured to detect position input interactions with a surface thereof and a plurality of input devices configured to detect touch interactions. The controller is configured to: receive signals from the position input device and the input device; matching the location input interaction with the touch interaction by time-correlating the timing of the location input interaction and the touch interaction; and generating an input interaction signal for the computer device by spatially associating the assigned spatial order of the input device with the coordinates of the location input interaction to match any remaining unmatched interactions.

Description

Technical Field

[0001] The present invention generally relates to a computer input system. Background Art

[0002] The input system is used to interact with electronic devices with various input devices. These input devices can include a mouse pointer, keyboard, touch-sensitive pad, touch-sensitive screen overlay, etc.

[0003] US2022 / 0026998 A1 (APPLE INC.) filed on January 27, 2022 gives an example of a user input system that includes a stylus and an electronic device, wherein a user can manipulate the stylus on an input surface of the electronic device and movement can be detected using an axis-aligned electric field generated by the stylus.

[0004] According to US2022 / 0026998 A1, a stylus can be identified by means of a "ringing signal" detected through the surface of an electronic device. In this way, more than one stylus can interact with the electronic device at the same time.

[0005] The present invention seeks to provide an input system that will overcome or substantially ameliorate at least some of the deficiencies of the prior art, or the present invention seeks to at least provide an alternative.

[0006] It should be appreciated that, if any prior art information is referenced herein, such reference does not constitute an admission that the information forms part of the common general knowledge in the art, in Australia or any other country. Summary of the invention

[0007] An input system connected to a device via an interface is provided. The system includes a position input device configured to detect position input interactions with a surface thereof. Each position input interaction is assigned corresponding coordinates relative to the surface.

[0008] The system also includes a plurality of input devices configured to detect touch interactions.

[0009] Each touch interaction is assigned a corresponding input device ID. In addition, these devices are assigned a spatial order. In addition, the spatial order is configured relative to the axis of the surface.

[0010] The input devices may include finger wearable devices, wherein each input device may include an elongated capacitive sensitive pad located at each fingertip and connected to a central wrist-worn controller by wired means. The wrist-worn controller may receive capacitive signals from each capacitive sensitive pad to detect touch interactions and assign a touch ID to each touch interaction.

[0011] The system also includes a controller configured to receive signals from the position input device and the input device.

[0012] The controller is configured to match the position input interaction with the touch interaction by temporally correlating the timing of the position input interaction and the corresponding touch interaction.

[0013] However, for simultaneous touches (where, for example, a user simultaneously presses fingers on the surface of the position input device such that more than one touch interaction occurs within a sampling window of the position input device), the controller may also be configured to match any remaining unmatched interactions by spatially associating the assigned spatial order of the input devices and the coordinates of the position input interactions.

[0014] After matching the position input interactions and the unmatched interactions, the controller can generate input interaction signals for the device, each input interaction including corresponding coordinates and an ID of a corresponding input device.

[0015] In an embodiment, the controller may be configured for an application-specific environment requiring a specific hand positioning and input device configuration. The controller interface may be installed on an operating system of the device that allows an application to configure the controller for the application-specific environment of the application.

[0016] The application-specific environment may be used to determine the spatial order and orientation of the spatial axes. Thus, the controller is able to generate an input interaction signal by processing the interaction according to the provided application-specific environment.

[0017] In an embodiment, the controller may process interactions according to defined zones, which may be specific areas of the surface area of ​​the position input device.

[0018] Additionally, other aspects of the invention are disclosed. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Although there are other forms that fall within the scope of the invention, preferred embodiments of the invention are described herein by way of example only with reference to the accompanying drawings, in which:

[0020] Figure 1 A schematic diagram showing an input system for interfacing devices according to one embodiment is shown;

[0021] Figure 2 A physical example of an input system connecting devices through an interface is shown;

[0022] Figure 3A An example of four input devices used on a single position input device is shown;

[0023] Figure 4AAn example is shown in which a user uses a pair of input devices 104 in each hand, and the hand orientation angle is 90°;

[0024] Figure 5A An example is shown where the input device 104 is confined to a particular area 125;

[0025] FIG6 shows an example of non-orthogonal spatial order axes and non-rectangular regions;

[0026] FIG7 shows a conventional drawing application configured with the present input system;

[0027] Figure 8 The processing 128 performed by the controller to generate the input interaction 108 is shown;

[0028] Fig. 9 The process 134 performed by the controller 105 to match the position input interaction 106 with the touch interaction 107 is shown;

[0029] Fig.10 The time matching process 139 performed by the controller 105 is shown;

[0030] Fig.11 shows the spatial matching process performed by the controller; and

[0031] Fig.12 An application specific environment process 150 is shown. DETAILED DESCRIPTION

[0032] Figure 1 1 shows a schematic functional block diagram of an input system 101 connected to a computer device 102 via an interface. Figure 2 , the device 102 may include a digital display 115 configured to display a user interface 116 of the application 112. The user interface 116 may include an input pointer 117.

[0033] The system 101 comprises a position input device 103. The position input device 103 is configured to detect position input interactions with its surface. Each position input interaction is assigned a corresponding XY coordinate relative to the surface.

[0034] according to Figure 2 In the illustrated embodiment, the position input device 103 comprises a touch pad. However, in alternative embodiments, the position input device 103 may comprise a touch-sensitive layer overlying the display 115 of the device 102.

[0035] The system 101 may include a plurality of position input devices 103. For example, Figure 2 In the illustrated embodiment, the system 101 includes three position input devices 103 .

[0036] The position input device 103 can detect position input interactions with it in a capacitive manner. For example, if an object touches the surface 118 of the position input device 103, the capacitive sensor matrix below it detects the capacitance change at the position of the object, so that the position input device can generate a corresponding position input interaction including XY coordinates.

[0037] Each position input interaction may include a down state when the interaction is first detected (i.e., when the object presses down on surface 118), a tracking state (i.e., when the object moves across surface 118), and an up state (i.e., when the object leaves surface 118 and there is no longer any interaction).

[0038] The system 100 also includes a plurality of input devices 104. Each input device 104 is configured to detect a touch interaction. Each touch interaction is assigned a corresponding input device ID.

[0039] According to the preferred embodiment shown, the input device 104 is a fingertip device. Figure 2 In the example shown, four input devices 104 are worn on the index finger and thumb of the left and right hands, respectively. Each input device 104 will be assigned a corresponding device ID (such as a digital index, such as numbers 1 to 4 for each of the four input devices 104).

[0040] According to one embodiment, each input device 104 includes an elongated capacitive sensitive pad located at each fingertip and connected to a central wrist-worn controller by wire. The wrist-worn controller can receive analog capacitive signals from each pad to detect touch interactions and assign corresponding touch IDs accordingly. The wrist-worn controller can transmit the detected touch interactions 107 to the controller 107 wirelessly (e.g., via Bluetooth).

[0041] The input device 104 ID is assigned a spatial order relative to the axis of the surface 118. For example, referring to Figure 2 , which can be accessed along the X-axis of the position input device 103. The spatial order can be assigned by direction relative to the axis.

[0042] The axis may also include a Y axis. In an embodiment, the access may be non-orthogonal and its orientation configurable.

[0043] exist Figure 2 With the hand orientation shown and the corresponding positions of the hands, the index finger of the left hand would be assigned spatial order 1, the thumb of the left hand would be assigned spatial order 2, the thumb of the right hand would be assigned spatial order 3, and the index finger of the right hand would be assigned spatial order 4.

[0044] The controller 105 is configured to receive signals from the position input device 103 and the input device 104. As described above, the position input interaction signal 106 may include XY coordinates relative to the surface 118 of the position input device 103. Additionally, the touch interaction signal 107 may include a corresponding device ID.

[0045] The controller 105 is configured to match the position input interaction 106 with the touch interaction 107 by temporally correlating the timing of the position input interaction 106 and the touch interaction 107 .

[0046] The controller 105 is also configured to match any remaining unmatched interactions 106 , 107 by spatially associating the assigned spatial order of the input devices 104 and the coordinates of the positional input interactions 106 .

[0047] Then, the controller 105 generates an input interaction signal 108 for the device 102 according to the matched interactions 106, 107. Each input interaction 108 includes corresponding XY coordinates and an ID of the corresponding input device 104.

[0048] The controller 105 may interface with a human input device (HID) interface 110 of the device 102. For example, the controller 105 may interface with the HID interface 110 of the device 102 to control a mouse cursor 117.

[0049] Additionally or alternatively, the controller interface 111 may be installed in the operating system 113 of the device 102 . The controller interface 111 is configured to interact with the controller 105 to receive and implement the interaction signal 108 .

[0050] In the illustrated embodiment, the input system 101 is separate from the device 102 and connected to the device 102 via a wired or wireless interface. Alternatively, the input system 101 may be implemented partially or completely by software executed by the device 102.

[0051] The controller interface 111 may be used to provide an application-specific environment for the controller 105. For example, different applications 112 may require different configurations of the input device 104 and hand orientations.

[0052] For example, Fig. 7A An interface 122 of a conventional painting application 112 is shown controlled using a keyboard 119 and a mouse 120. The mouse 120 controls a mouse pointer 121 on the screen.

[0053] The interface 122 includes object selection controls 123, each of which can be selected using the pointer 121 to select a pencil or eraser. The interface 122 also includes layer selection controls 124, where the pointer 121 can be used to select a layer.

[0054] Figure 7B A painting application 112 is shown configured using the input system 101, wherein a first position input device 103A is used to interact with a first layer, and a second position input device 103B is used for user interaction with a second layer.

[0055] In addition, the first input device 104A is used for a pencil, and the second input device 104B is used for an eraser. In this way, the user can draw and erase on each layer using only one hand.

[0056] Therefore, according to this example, the currently executing painting application 112 will utilize the controller interface 111 to transfer the application-specific environment 109 to the controller 105 .

[0057] In this example, the application-specific environment is used to determine the spatial order down the Y-axis of the positional input device 103 (given the orientation of the hand), where the index finger input device 104A is assigned a spatial order of 1 and the thumb input device 104B is assigned a spatial order of 2.

[0058] In an embodiment, the controller 105 may interpret the position input interaction 106 and the touch interaction 107 according to the defined area 125. For example, referring to Figure 1 , the surface 118 of the position input device 103 can be divided into regions 125 .

[0059] For example, referring to the example of FIG. 7 , one pair of input devices 104 may be used on the left hand and another pair of input devices 104 may be used on the right hand.

[0060] The regions 125 may indicate the position input device ID and the corresponding individual device IDs. For example, a region 125A may specify that only the left-hand input device 104 may be used on the first position input device 103A, while another region 125B may specify that only the right-hand input device 104 may be used on the second position input device 103B. Each region 125 may also be assigned an area, a spatial order, and a spatial axis.

[0061] Figure 3A An example is given of using four input devices 104 on a single position input device 103. According to this example, both hands are oriented at 0° and the spatial order is relative to the X-axis.

[0062] An exemplary interaction 127 is shown with corresponding coordinates and device ID.

[0063] As described above, each interaction 127 may include a downward state (one sampling frame) when the input device 104 presses the surface 118 of the position input device 103, a tracking state (multiple sampling frames) when the input device 104 moves on the surface 118 of the position input device 103, and an upward state (one sampling frame) when the input device 104 leaves the surface 118 of the position input device 103.

[0064] The controller 105 may be configured to group the down states within a sampling window. The sampling window may be configured to be long enough so that the controller 105 can receive signals from the position input device 103 and the at least one input device 104 .

[0065] According to the time association, if only one position input interaction 106 and one touch interaction 107 are received within the sampling window, the interactions 106, 107 may match.

[0066] However, if more than one position input interaction 106 and / or more than one touch interaction 107 is received within the sampling window, the controller 105 performs spatial association. For example, the user may press all four fingers simultaneously on the surface 118 of the position input device 103. The spatial association may involve the down state of the unmatched interactions 106, 107, and the controller 115 may perform pairwise matching of the position input interaction 106 with the touch interaction 107 according to the spatial order and the spatial axis 126.

[0067] For example, in FIG3 , the spatial axis is the positive X axis for the area and input device 104. The input device 104 assigns spatial order 1 to the index finger of the left hand, spatial order 2 to the thumb of the left hand, spatial order 3 to the thumb of the right hand, and spatial order 4 to the index finger of the right hand. The coordinates generated on the position input device 103 are assigned from left to right, with the leftmost coordinate assigned spatial order 1 and the rightmost coordinate assigned spatial order 4. In addition, the input device 104 ID matches the assigned spatial order.

[0068] As such, the controller 105 will perform spatial association by pair-matching the leftmost coordinate with the spatial order 1 to determine that the left index finger with the spatial order 1 performed the interaction 127A.

[0069] The controller 105 will then match the next leftmost coordinate with the spatial order 2 to determine that the left thumb with the spatial order 2 performed the interaction 127B, and so on, until all interactions 127 are matched.

[0070] Figure 3B An example is given of invalid input due to hand swapping such that the positioning of the input device 104 no longer correlates to the assigned spatial order. Figure 3CExamples of invalid input due to changes in hand orientation are further given.

[0071] Figure 4A An example is given of a user using a pair of input devices 104 on each hand, with the hands oriented at 90°. Figure 4 further gives an example of assigning a spatial order to the respective regions 125 of the position input device 103. In the example shown, each region 125 is rectangular and can be defined by minimum and maximum XY coordinates.

[0072] The first region 125A may have a spatial order assigned to a downward vertical axis 126A, while the second region 125B will have a spatial order assigned to an upward vertical axis.

[0073] More specifically, the first region 125A may have a spatial order {1, 2, 3, 4} downward along the vertical axis, and the second region 125B may have a spatial order {1, 2, 3, 4} upward along the vertical axis.

[0074] Figure 4B An example of invalid input is given due to the right hand being inverted, where the index finger input device 104C with ID 4 is located above the thumb input device 104D with ID 3, thus being inconsistent with the allocated spatial order {1, 2, 3, 4}.

[0075] However, Figure 4C An example of valid input is given because the right hand is now turned over, so that the input device IDs are correlated with the allocated space order.

[0076] Figure 4D Another example of invalid input is given because the right-hand input device 104 is located above the left-hand input device 104 and is therefore inconsistent with the allocated spatial order.

[0077] Figure 5A An example is given where the input devices 104 are confined to a particular region 125. This is indicated by the list of input device IDs next to the spatial order of the region. The list of input device IDs is sorted according to the spatial order of the input devices. In the example shown, an index finger input device 104 and a thumb input device 104 are used on each hand, each hand is oriented at approximately 90°, and the hands are crossed. The resulting matches are {3, 4, 1, 2}, i.e., coordinate 1 matches device 3, coordinate 2 matches device 4, and so on.

[0078] The first region 125A has a spatial order allocated upward along the vertical Y-axis 126A, and the second region 125B has a spatial order allocated downward along the vertical Y-axis 126B.

[0079] In addition, according to Figure 5A , the first area 125A is limited to interacting with input devices 104 having ID 3 and ID 4, while the second area 125B is limited to interacting with input devices 104 having ID 1 and ID 2.

[0080] Figure 5B An example of a valid input is given because the first region 125A is limited to input device IDs {1, 2, 4} because coordinates 1, 2, 3 match {1, 2, 4}, and coordinate 4 matches 3 because ID 1 was used in a previous coordinate match. The resulting match is {1, 2, 4, 3}.

[0081] Figure 5C is an example of invalid input because when used with Figure 5B When comparing, the spatial order of the regions is reversed, so the coordinates are sorted differently (i.e., from the right region to the left region). In addition, coordinate 4 cannot be matched to the input device ID because the valid input device ID {1, 2, 4} has been used previously. The resulting match result is {1, 2, 4, ?}.

[0082] Figure 5D Another example of invalid input is given because coordinate 1 is incorrectly matched to input device 1 instead of input device 4. The resulting matching result is {1, 2, 4, 3}.

[0083] Figure 6 gives an example where the space sequence axis 126 is non-orthogonal and its orientation can be configured according to the application specific environment. In the example shown, the space sequence axis can be at about 45° relative to the orthogonal axis.

[0084] also, Fig. 6A An example of non-rectangular regions is given where the first region 125A is circular and the second region 125B is L-shaped.

[0085] Additionally, FIG. 6 shows an example where each region 125 is limited to certain device identifiers and the spatial order of the regions may be affected by the spatial order of the parent location input devices.

[0086] Although FIG. 6 shows regions 125 as being separate, in embodiments, regions 125 may overlap.

[0087] In an embodiment, the orientation and / or shape of region 125 may be reconfigured as required by an application.

[0088] Figure 8 Processing 128 performed by the controller to generate the input interaction 108 is shown.

[0089] In step 129, the controller 105 receives position input interactions 106, which may be stored in a buffer. In step 130, the controller 105 receives touch interactions 107, which may also be stored in a buffer.

[0090] In the case that the system 100 includes more than one position input device 103 , in step 131 , the controller 105 assigns each position input interaction 106 to a corresponding position input device 103 and subsequently removes the position input interaction from the buffer.

[0091] Furthermore, the controller 105 may assign each such interaction 107 to a corresponding input device 104 and subsequently remove the interaction from the buffer.

[0092] In step 132 , the controller 105 matches the position input interaction 107 with the touch interaction 107 to generate an input interaction 108 for the device 102 .

[0093] Fig. 9 The process 134 performed by the controller 105 to match the position input interaction 106 with the touch interaction 107 is shown (as a sub-process of step 132 ).

[0094] In step 135, the controller 105 obtains the unmatched position input interaction 106 and attempts to match it with the corresponding touch interaction 107 in step 136. As described above, a temporal match may be found where only one touch interaction 107 occurs within a sampling window associated with the position input interaction 106 (such as a sampling window starting from the down state of the position input interaction 106).

[0095] If the temporal match fails (eg, where more than one touch interaction 107 occurs within the same sampling window), the process 134 may begin spatial matching in step 137 .

[0096] For all interactions 106 , 107 in the cache, time matching and / or spatial matching is performed in steps 136 and 137 , and then the process 134 is exited in step 138 .

[0097] Fig.10 A time matching process 139 is shown performed by the controller 105 (as a sub-flow of step 136), where, in step 140, the controller 105 determines that the touch interaction 107 is within a sampling window (T) associated with the position input interaction 106 (e.g., a sampling window starting from the down state of the position input interaction 106). The controller 105 may determine whether the down state of the touch interaction 107 is within the window.

[0098] If only one touch interaction 107 is found within the sampling window in step 141, a matching assignment is performed in step 142; otherwise, the process 139 is exited in step 143, and the process proceeds to Fig.11 The spatial matching process step 144 is shown (as a sub-flow of step 137).

[0099] In step 145 and step 146, the controller 105 determines the position input interaction 106 and the touch interaction 107 within the sampling window. The selected sampling window may start from the downward state of the position input interaction 106.

[0100] In step 147 , the controller 105 may determine whether any areas are applicable based on the collected location input interactions.

[0101] In step 148 , the controller will determine the spatial order of each captured position input interaction 106 by utilizing the spatial order of the corresponding regions 125 and the spatial axis.

[0102] In step 149, the controller performs pairwise matching of the position input interactions with the input devices 104 in the sorted order. If the input device is valid for the region 125 of the position interaction and has not been used in a previous match, then the match is successful.

[0103] Fig.12 An application specific environment process 150 is shown.

[0104] In step 151, the application 112 may detect changes in the application-specific environment, such as when a Figure 7B In step 152 , the application 112 may utilize the controller interface 111 to transmit the application-specific environment 109 to the controller 105 to configure the input system 101 .

[0105] As described above, the dedicated environment 109 may be used to determine the spatial order and the orientation of the spatial order axis.

[0106] Additionally, the application-specific environment 109 may specify a region 125 in which a device ID is assigned to a region of the position input device 103 .

[0107] In step 153, the user may be notified of the change in the application-specific environment. Such notification may indicate to the user the correct positioning of the hand and input device 104, which the user is to follow in step 154.

[0108] Therefore, in step 155 , the input system 101 processes the position input interaction 106 and the touch interaction 107 according to the new application-specific environment to generate an input interaction signal in step 156 .

[0109] For the purpose of illustration, the above description uses specific terms to provide a comprehensive understanding of the present invention. However, it is obvious to those skilled in the art that the present invention can be practiced without specific details. Therefore, for the purpose of illustration and description, the above description of specific embodiments of the present invention is presented. These descriptions are not intended to be exhaustive or to limit the present invention to the exact form disclosed, because in view of the above teachings, there can obviously be many modifications and variations. These embodiments are selected and described in order to maximize the explanation of the principles of the present invention and its practical application, so that other technical personnel in the field can maximize the use of the present invention and various embodiments with various modifications suitable for the intended specific use. The following claims and their equivalents are intended to limit the scope of the present invention.

Claims

1. An input system connected to a device through an interface, include: a position input device configured to detect position input interactions with a surface thereof, each position input interaction being assigned a corresponding coordinate relative to said surface; a plurality of input devices configured to detect touch interactions, each touch interaction being assigned a corresponding input device ID, the input devices being assigned a spatial order, the spatial ordering being configured relative to a spatial axis of the surface; Controller, configured as: receiving signals from the position input device and the input device; matching the position input interaction with the touch interaction by temporally correlating the timing of the position input interaction and the touch interaction; matching any remaining unmatched interactions by spatially correlating the assigned spatial order of the input devices and the coordinates of the positional input interactions; as well as According to the matched interactions, an input interaction signal is generated for the device, and each input interaction includes corresponding coordinates and an ID of a corresponding input device.

2. The system of claim 1, wherein the spatial ordering is directional relative to the spatial axis.

3. The system of claim 1, wherein the orientation of the spatial axis is configurable.

4. The system of claim 1, wherein the surface of the position input device comprises a plurality of regions, and wherein a spatial order is assigned to each region.

5. The system of claim 1, wherein the system comprises a plurality of position input devices, and wherein a spatial order is assigned to each position input device.

6. The system of claim 1, wherein the controller is configurable with an application-specific environment, and wherein the application-specific environment determines the spatial order.

7. The system of claim 6, wherein the controller is configured to receive application-specific environment changes from the device.

8. The system of claim 7, wherein the operating system of the device is installed with a controller interface, and the controller interface transmits the application-specific environment changes to the controller.

9. The system of claim 1, wherein the controller is configured to process the interaction according to a defined area.

10. The system of claim 9, wherein the area includes a position input ID, a corresponding input device ID, a spatial order, and a spatial axis. The system of claim 10 , wherein the regions overlap.

12. The system of claim 10, wherein the controller is configured to discard touch interactions whose IDs do not match the input device IDs of the zones.

13. The system of claim 10, wherein at least one of a shape and an orientation of the region is configurable via an application.

14. The system of claim 1, wherein the controller is configured to time-match the touch interaction with the position input interaction only if the touch interaction is the only touch interaction occurring within a sampling window associated with the position input interaction.

15. The system of claim 14, wherein the sampling window starts from a down state of the position input interaction.

16. The system of claim 14, wherein the controller is configured to perform spatial correlation only if more than one touch interaction occurs within the sampling window.

Images