Optical stylus for optical position determining device

Abstract

The present invention relates to an optical stylus (10) for an optical position determination device (1) with a position-coding surface (50) having different position-coding patterns (52). The optical stylus (10) comprises a housing (12) including an image sensor (20) for capturing at least one image of any position-coding pattern (52) on the position-coding surface (50), an optical arrangement (15) with a dynamic optical element (16) configured to be electrically operated to adjust the focal length of the optical stylus (10) depending on a separation distance (D) between a reference point of the optical stylus (10) and the position of any position-coding pattern (52), and a control unit (24) for controlling the dynamic optical element (16), and further comprises a distance measuring sensor (32) for measuring the separation distance (D). The control unit (24) operates the dynamic optical element (16) to adjust the focal length of the optical stylus (10).

Description

[Technical Field]

[0001] The present invention relates to an optical stylus for an optical position determination device, and to an optical position determination device comprising an optical stylus and a position-coding surface for determining the precise two-dimensional and three-dimensional position of the optical stylus relative to the position-coding surface. The present invention also relates to a method for determining the three-dimensional coordinates of the optical stylus relative to the position-coding surface. [Background technology]

[0002] Among user input devices, touch-sensitive displays offer the most natural means of human-computer interaction, through simple gestures, signs, or even handwritten text. Using fingers as input devices generally produces inaccurate results. Styluses are widely used for more precise data entry.

[0003] However, independent of the technology used, most existing styluses have various limitations, requiring either physical contact or very little separation between the pen and the screen, and therefore their range of applications remains limited to roughly graphic input within a two-dimensional plane.

[0004] US Patent No. 5,949,999 discloses an electro-optical pen that can determine the distance between the pen and the writing surface using different imaging processing techniques such as triangulation, field of view overlap, dot cross-correlation or stereoscopic image processing techniques.

[0005] Patent Document 2 discloses a camera pen that can be switched between different operating modes. The pen can be used in a contact mode to read data from an object surface while the pen is in physical contact with the object surface. The pen can also be used in a hover mode, which allows the pen to be controlled to output three-dimensional position data, thereby enabling the pen to be used for 3D applications. The camera is equipped with a variable-focus lens, the focal length of which is controlled according to the object distance given by parameters derived from previously captured images. Therefore, the camera pen derives the object distance using an image processing approach.

[0006] Similarly, US Pat. No. 6,299,649 discloses a camera pen for 2D and 3D applications that uses an image processing approach to derive object distance for 3D applications.

[0007] It is an object of the present invention to provide a versatile optical stylus for 2D and 3D applications that uses an alternative approach to determining object distance for 3D applications.

[0008] This versatile optical stylus is advantageously adapted for applications such as 3D engineering design, technical sketching, artistic drawing or calligraphy. [Prior art documents] [Patent documents]

[0009] [Patent Document 1] U.S. Patent Application Publication No. 20200264710 [Patent Document 2] U.S. Patent Application Publication No. 2010 / 085471 [Patent Document 3] Chinese Patent No. 1086641553 Summary of the Invention [Means for solving the problem]

[0010] This object is achieved by an optical stylus for an optical position determination device having a position-coding surface with different position-coding patterns. The optical stylus comprises: an image sensor for capturing at least one image of any position-coding pattern on the position-coding surface; an optical arrangement including a dynamic optical element electrically operated to adjust the focal length of the optical stylus in response to a separation distance between a reference point of the optical stylus and the position of any position-coding pattern so as to have a substantially focused position-coding pattern corresponding to the position of the optical stylus; a control unit for controlling the dynamic optical element; and an aperture or window at the distal end of the optical stylus. The optical stylus is adapted to identify any substantially focused coding pattern on the position-coding surface independently of the separation distance for determining the position of the optical stylus relative to the surface. The optical stylus further comprises a distance measuring sensor for measuring the separation distance. The control unit is adapted to operate the dynamic optical element in response to an output signal of the distance measuring sensor to adjust the focal length of the optical stylus.

[0011] In one embodiment, the control unit has two following modes: a 3D mode in which an electrical signal is applied to the dynamic optical element, the power of the electrical signal varying with separation distance to vary the focal length of the optical stylus accordingly; and a 2D mode in which no electrical signal is applied to the dynamic optical element, so that when the distal tip of the optical stylus is in contact with or within up to 5 mm of the position-coding surface, the focal length of the optical stylus remains constant and any coding pattern is substantially in focus. The dynamic optical element is configured to control the dynamic optical element according to any one of the following:

[0012] In one embodiment, the control unit is configured to switch the optical stylus between 2D and 3D modes and vice versa depending on the output signal of the distance measuring sensor.

[0013] In one embodiment, the optical stylus further comprises a function button electrically connected to the control unit for switching the optical stylus between 2D mode and 3D mode and vice versa.

[0014] In one embodiment, the optical stylus further comprises a pressure sensor located at the tip of the stylus, and the control unit is configured to switch the optical stylus between the 2D mode and the 3D mode and vice versa in response to an output signal of the pressure sensor.

[0015] In one embodiment, the distance measurement sensor is a time-of-flight sensor.

[0016] In one embodiment, the dynamic optical element is a liquid or polymer lens that is electrically actuatable to adjust the focal length of the optical stylus.

[0017] In one embodiment, the dynamic optical element is a fixed focus lens attached to an actuator configured to shift the fixed focus lens along the optical path of the optical stylus to adjust its focal length.

[0018] In one embodiment, the actuator is a piezoelectric element.

[0019] In one embodiment, the optical stylus further comprises at least one light emitting unit adapted to illuminate any position-coding pattern.

[0020] In one embodiment, the optical stylus further comprises an inertial measurement unit.

[0021] In one embodiment, the optical stylus further comprises a battery and an electrical circuit including a control unit, an image sensor and a memory, the memory storing pattern recognition algorithms for image processing and / or determining the position of the optical stylus relative to the position-coded surface.

[0022] In one embodiment, the optical stylus further comprises a particular shape at its distal end for writing, painting, or calligraphy applications. The particular shape is in the form of a brush with a filament bundle, or another flexible tip, or a pen-like tip. An aperture or window is located at the distal end of the optical stylus to allow images of both the particular shape and the corresponding position-coding pattern to be captured together by the image sensor.

[0023] Another aspect of the invention relates to an optical position determination device comprising an optical stylus, a host device comprising a screen and a position-coding surface having a different position-coding pattern, the host device being configured to receive position-related data from the optical stylus to generate on the screen a trace following movement of the optical stylus relative to the position-coding surface.

[0024] In one embodiment, the position-coding surface is an integral part of the host device

[0025] In one embodiment, the position-coding surface is made from a flexible material that is applied to the top surface of the screen of the host device.

[0026] In one embodiment, the optical position determining device comprises a stand-alone position-coding surface.

[0027] In one embodiment, the different position-coding patterns of the position-coding surface are transparent in the visible range of the light spectrum.

[0028] In one embodiment, the host device or a stand-alone position-coding surface comprises one or more light sources configured to emit light outside the visible range of the light spectrum and positioned below the position-coding surface to back-illuminate each of the position-coding patterns.

[0029] Another aspect of the invention relates to a method for determining the position of an optical stylus relative to a position-coding surface including different position-coding patterns. The optical stylus comprises an image sensor for capturing at least one image of any position-coding pattern of the position-coding surface, an optical arrangement with a dynamic optical element configured to be electrically activated, a distance measuring sensor for measuring a separation distance between a reference point of the optical stylus and a position of any position-coding pattern, and a control unit for controlling the dynamic optical element. The method includes the steps of: a) holding the optical stylus in the direction of the position-coding surface; b) operating the dynamic optical element to adjust the focal length of the optical stylus in response to an output signal of the distance measuring sensor; c) acquiring by the image sensor an optical signal of an image formed by the optical arrangement of a substantially focused position-coding pattern of the position-coding surface; and d) processing the optical signal to determine the position of the optical stylus relative to the position-coding surface.

[0030] In one embodiment, the optical stylus further comprises a gyroscope and / or an accelerometer, and the position of the optical stylus is calculated based on both the optical signal acquired by the image sensor and the data acquired by the gyroscope and / or accelerometer, or based solely on the data acquired by the gyroscope and / or accelerometer.

[0031] In one embodiment, the orientation of the optical stylus is measured by a gyroscope and / or an accelerometer at least during the acquisition of the optical signal under step c.

[0032] In one embodiment, the tilt angle of the optical stylus is used, for example, to correct for perspective distortion of the captured coding pattern, providing more robust pattern recognition and position determination.

[0033] In one embodiment, the optical stylus further comprises a light emitting unit, the light intensity of which is controlled according to the output signal of the distance measuring sensor.

[0034] In one embodiment, data representing an image projection of any of the position-coding patterns captured by the image sensor is transmitted wirelessly to a host device comprising a screen and configured to run a computer vision algorithm to determine the position of the optical stylus after relative movement of the optical stylus with respect to the position-coding surface before displaying a trace on the screen.

[0035] In one embodiment, data representing an image projection of any of the position-coding patterns captured by the image sensor is processed by a processing unit of the optical stylus which executes computer vision algorithms to determine the position of the optical stylus after relative movement of the optical stylus with respect to the position-coding surface before wirelessly transmitting position-related data to a host device for displaying a trace on the screen.

[0036] In one embodiment, the hue, stroke width and / or shape of the trace are adjusted in response to any one or a combination of two or more of the following parameters of the optical stylus: velocity, acceleration, direction and tip pressure on the position-coded surface.

[0037] The invention will be better understood from the description of some embodiments given by way of example and illustrated by the drawings in which: [Brief explanation of the drawings]

[0038] [Figure 1] 1 shows a schematic elevational view with a partial cross section of an optical stylus according to one embodiment of the present invention; [Figure 2] Figure 2 shows an elevation view of the optical stylus of Figure 1 in a short focal length configuration with the distal tip of the optical stylus proximate to the position-coding surface, and Figure 2a is an image of the position-coding pattern captured by the image sensor of the optical stylus in the configuration shown in Figure 2. [Figure 3]Figure 3 shows an elevation view of the optical stylus of Figure 1 in a short focus configuration, along with a partial cross-section showing the adjustable liquid lens when the optical stylus is defocused and away from the position-coding surface. Figure 3a shows a cross-section of the adjustable liquid lens when a first tension is applied to the lens to form a convex curvature of the interface between the two liquids. Figure 3b shows an image of the position-coding pattern captured by the image sensor of the optical stylus in the configuration shown in Figure 3. [Figure 4] Figure 4 shows an elevation view of the optical stylus of Figure 1 in a focused configuration when the stylus is away from the position-coding surface, along with a partial cross-section showing the adjustable liquid lens. Figure 4a shows a cross-section of the adjustable liquid lens when a second tension force is applied to the lens to create a concave curvature of the interface between the two liquids. Figure 4b shows the position-coding pattern captured by the image sensor of the optical stylus in the configuration shown in Figure 4. [Figure 5] Figure 5 shows an elevation view of an optical stylus according to another embodiment of the invention, comprising a brush at its distal end comprising a bundle of filaments, and Figure 5a shows the position-coding pattern and shape of the brush as captured by the image sensor of the optical stylus in the configuration shown in Figure 5. [Figure 6] Figure 6a shows a distal end portion of an optical stylus interacting with a position-coding surface, where a light emitting unit is integrated inside the optical stylus, according to one embodiment of the invention, and Figure 6b shows a distal end portion of an optical stylus interacting with a position-coding surface, where a light emitting unit is embedded in the position-coding surface, according to another embodiment of the invention. [Figure 7] FIG. 1 is a block diagram of pattern recognition and position determination of an optical stylus relative to a position-coded surface. DETAILED DESCRIPTION OF THE INVENTION

[0039] The optical stylus 10 of Figure 1 is configured to determine its position relative to a position-encoded surface 50. Within the context of the present invention, the term "position" of the optical stylus shall be understood as any of the following (1) to (3) or a combination thereof: (1) the two-dimensional (2D) coordinates of a reference point of the optical stylus relative to the position-encoded surface, (2) the distance between the reference point of the optical stylus and the position-encoded surface, and (3) the orientation of the optical stylus relative to the position-encoded surface, where the orientation can be expressed in terms of Euler angles, yaw angles, pitch and roll angles, a quaternion, or a rotation matrix.

[0040] 1, optical stylus 10 comprises a housing 12 containing an optical arrangement 15 for shaping a light beam. Optical arrangement 15 includes a dynamic optical element 16 and other optical components 18 including, for example, lenses, optical filters, apertures, and mirrors. Dynamic optical element 16 is positioned in the optical path of optical stylus 10 along with other optical components 18 to optimally adjust the focal length of optical stylus 10 depending on a separation distance D separating a reference point on optical stylus 10, which may be the distal tip of the optical stylus, from a position-encoding surface 50 including a position-encoding pattern 52, as shown in FIG.

[0041] The position-coding pattern 52 may be in the form of, for example, a tag that encodes an XY two-dimensional coordinate grid, or in the form of a different pattern that can be decoded back into the coordinates of a particular position on the position-coding surface 50.

[0042] Alternatively, the position-coding pattern 52 may be in the form of a unique tag or signature that can be uniquely assigned to a specific location on the position-coding surface 50. A look-up table containing data relating to a series of corresponding unique tags or signatures and the location of each unique pattern or signature on the coding surface can be stored in a memory that can be incorporated into the optical stylus housing 12 or in a host device that communicates with the optical stylus to receive raw optical signals from the optical stylus corresponding to the unique tags or signatures. In one embodiment, the 2D position of the optical stylus (i.e., the 2D coordinates of the distal tip of the optical stylus relative to the position-coding surface) can be determined by comparing the tag or signature on the position-coding surface 50 with the series of corresponding unique tags or signatures contained in the look-up table. The separation distance D can be determined by estimating the relative size of the imaged position-coding pattern or by estimating the field of view, while the orientation of the optical stylus can be estimated by analyzing the perspective distortion of the imaged position-coding pattern. Other methods and / or additional sensors can be utilized to determine the position of the optical stylus.

[0043] Adjustment of the focal length of the optical arrangement 15 of the optical stylus 10 is performed so that the object plane shifts to have a substantially focused position-coding pattern 52 that corresponds to the position of the optical stylus 10 relative to the position-coding surface 50. Within the context of the present invention, the term "substantially focused position-coding pattern" should be interpreted such that the position-coding pattern may be slightly out of focus but still be identifiable by pattern recognition software as described below.

[0044] The object plane of the optical arrangement 15 may be parallel to, for example, within 5 mm of, the plane containing the position-coding pattern 52, so that the latter may be slightly out of focus and blurred, but still distinct enough to be identified by pattern recognition software. The object plane may also be tilted relative to the plane containing the position-coding pattern 52, so that only part of the image of the pattern 52 is in focus. The in-focus part of the image still allows the pattern to be identified by the pattern recognition software.

[0045] An image sensor 20 (FIG. 1), e.g., a CCD, CMOS sensor, or an array of photodiodes, is mounted inside the housing 12 in the stylus optical path to capture images of the position-coding pattern 52, e.g., as shown in FIGS. 2 and 2a. For 3D applications such as 3D engineering design, technical sketching, artistic drawing, or calligraphy, the focal length of the optical arrangement 15 of the optical stylus 10 must always be properly adjusted to avoid an out-of-focus position-coding pattern 52 that would not be identified by any pattern recognition software, as shown in FIG. 3b.

[0046] In one embodiment, the dynamic optical element 16 is an electrically tunable liquid lens. An electrical signal applied to the tunable liquid lens 16 determines the curvature of the interface between two liquids 16a, 16b with different refractive indices, as shown, for example, schematically in Figures 3a and 4a.

[0047] In another embodiment not shown, the dynamic optical element is an optical lens having a fixed focus, the optical lens being arranged to be shifted along the optical path of the optical stylus by an actuator, for example a piezoelectric element.

[0048] 4, 4a and 4b, the adjustable liquid lens 16 is electrically actuated in response to a separation distance D between a reference point on the optical stylus 10, for example the distal tip of the optical stylus, and the position of any position-coding pattern 52 on the position-coding surface 50, to provide a focused position-coding pattern 52 that corresponds to the position of the optical stylus 10. Adjusting the focus to provide a focused position-coding pattern 52 can be achieved according to an open-loop or a closed-loop configuration.

[0049] In an open-loop configuration, the housing 12 of the optical stylus 10 may include a distance measurement sensor 32, which may be, for example, a time-of-flight sensor, a laser sensor, a PDAF sensor, or an ultrasonic sensor, as shown in Figure 1. The distance measurement sensor 32 is configured to measure the distance between a reference point on the optical stylus 10 and the position of any position-encoding surface 52 that correlates to a separation distance D.

[0050] The housing 12 of the optical stylus 10 comprises a control or processing unit 24 configured to constantly control the electrical signal power applied to the adjustable lens 16 so as to change the focal length of the lens 16 based on the output of the distance measuring sensor 32, for example to change the curvature of the interface between the two liquids 16a, 16b.

[0051] In a closed-loop configuration, the dynamic optical element 16 is electrically activated in response to an autofocus algorithm executed by the processing unit 24, such as a contrast optimization autofocus algorithm or a phase detection autofocus algorithm. In the case of a contrast optimization algorithm, if the image captured by the image sensor 20 is blurry, as shown in FIG. 3b, an electrical signal is applied to the adjustable liquid lens 16 or piezoelectric element to change the focal length of the optical arrangement 15 of the optical stylus 10. If this improves the contrast of the image (positive gradient), a higher electrical signal is applied to the liquid lens 16 until an optimal point (point of zero gradient) is found. In an advantageous embodiment, the dynamic optical element 16 is activated in response to both the output of the signal from the distance measuring sensor 32 and parameters obtained from the autofocus algorithm to obtain a more robust position of the optical stylus.

[0052] The optical stylus 10 may advantageously be provided with a function button 19 configured to switch the optical stylus 10 from a 2D mode for a 2D application to a 3D mode for a 3D application and vice versa, as shown in Figures 2 to 4, so that the position of the optical stylus 10 relative to the position-coding surface 50 can be effectively determined in both the 2D mode and the 3D mode for a selected application.

[0053] In 2D mode, no electrical signals are applied to the dynamic optics 16, so that when the distal tip of the optical stylus is in contact with or within 5 mm of the position-coding surface 50, the focal length of the optical arrangement 15 of the optical stylus 10 remains constant and any coding pattern 52 is substantially in focus. Therefore, 2D mode is energy efficient because the focal length is not adjusted by applying a varying electrical signal to the dynamic optics 16. For dedicated 2D applications, the dynamic optics can be replaced by a fixed focus lens.

[0054] When the optical stylus 10 is in the 2D mode, positioning the optical stylus 10 further away from the position-coding surface 50 makes it more difficult or even impossible to detect the position-coding pattern 52. This is shown schematically in Figure 3, where a diverging light beam represents a defocused optical stylus 10, and one or more images 21 of any position-coding pattern 52 acquired by the image sensor 20 of the optical stylus 10 in this position reveal a blurred position-coding pattern 52 (Figure 3d). The 2D mode corresponds to prior art touch-sensitive optical position input systems, since, like conventional touch-sensitive technology, the optical stylus 10 can only operate in close proximity to the position-coding surface 50.

[0055] In 3D mode, an electrical signal is constantly applied to the dynamic optical element 16, the power of which varies depending on the separation distance D, and the focal length of the optical arrangement 15 of the optical stylus 10 varies accordingly. The working range between the distal end of the optical stylus 10 and the position-coding surface 50 at which the position-coding pattern 52 can still be detected can extend, for example, up to 15 cm from the position-coding surface 50.

[0056] In an alternative embodiment, the control or processing unit 24 is configured to automatically switch the optical stylus 10 from 2D mode to 3D mode and vice versa in response to the output signal of the distance measuring sensor 32. In another, not shown, embodiment, the control or processing unit is configured to switch the optical stylus from 2D mode to 3D mode and vice versa in response to the output signal of a pressure sensor incorporated in the distal end of the optical stylus.

[0057] Optical stylus 10 includes a distal portion 13 that is centered on the optical path of optical stylus 10 and includes an opening 13a at its distal end that leads to the inside of housing 12. Alternatively, a window made of a material that transmits light in a given spectral range may be attached to the inside of the aperture at the distal end of distal portion 13 of housing 12. Distal portion 13 can have, for example, a truncated cone shape, a cone with a sharp tip similar to a writing pen, or a cone with a rounded distal end similar to a ballpoint pen. In an alternative embodiment, as shown in Figures 5 and 5a, optical stylus 10 includes a brush 14 made from a bundle of bristles, filaments, or any other flexible material.

[0058] 5 and 5a, the optical arrangement 15 may comprise an additional mirror in the optical path of the optical stylus 10 to appropriately deflect the light beam.

[0059] Considering the multiple degrees of freedom of a brush with a bundle of filaments, not only the position of the brush 14 but also its shape, bend, tilt, and twist can be acquired by the image sensor 20, and the corresponding optical signals can be processed to extract the edges 14a, 14b, traces 14c, and contact areas 14d of the brush 14 using computer vision algorithms.

[0060] The processed data may be transferred to a host device with a screen or display, such as a television, computer, tablet, or smartphone, incorporating the position-coding surface 50. In one embodiment, the position-coding surface 50 is made of a flexible material applied to the top surface of the host device's screen. In an alternative embodiment, the position-coding surface 50 is a stand-alone device. A trace of the optical stylus can be generated on the host device's screen according to the relative movement between the optical stylus and the position-coding surface. In one embodiment, a trace 14c of the brush 14 can be generated on the host device's screen, taking into account the brush's shape, including its twist, bend, and / or direction. The separation distance D between the brush 14 and the position-coding surface 50 can also be considered to analyze the brush's behavior, as it provides information about the pressure applied to the brush to estimate the corresponding width of the trace 14c.

[0061] The optical stylus 10 may require a light source for detecting the position-coding pattern 52 to avoid relying on ambient illumination, which may be weak in dark environments. In this regard, the housing 12 of the optical stylus 10 further comprises a light-emitting unit 22 comprising one or more LEDs 22a, 22b, 22c (FIG. 1) arranged to illuminate the position-coding pattern 52 of the position-coding surface 50. The light beam emitted by the light-emitting unit 22 is either transmitted through or reflected from the position-coding surface 50, depending on whether the light beam is incident on the position-coding pattern 52 or on an exposed portion of the position-coding surface 50 around the position-coding pattern 52, as shown in FIG. 6a. As a result, the image sensor 20 captures only the reflected light beam to image the position-coding pattern 52. In an advantageous embodiment, the intensity of the light emitted by the light-emitting unit 22 may be varied in response to the output signal of the distance-measuring sensor 32 so that the light intensity decreases and therefore increases as the optical stylus 10 moves towards or away from the position-coding surface 50, respectively. In another embodiment, the emitted light beam is reshaped by a dynamic optical element that operates a variable collimator in response to a signal provided by the control unit.

[0062] In an alternative embodiment, as shown in Figure 6b, the light source can be integrated into the position-coding surface 50 below one or more position-coding patterns 52. In that case, depending on its path, the emitted light is either blocked or passes through the position-coding surface 50. The light that passes through the position-coding surface 50 and is collected by the optical stylus 10 conveys information about the shape of the underlying position-coding pattern(s).

[0063] To reduce the perception of the optical position determination device 1 to a user, the position-coding pattern 52 on the position-coding surface 50 can be made transparent in the visible range of the light spectrum. This can be achieved, for example, by using inks with complex absorption spectra. In this case, the light used to detect the pattern may be outside the visible part of the spectrum. For example, the optical position determination device 1 can use near-infrared light in the range of 700 nm to 1000 nm. These wavelengths are invisible to the human eye but can be recognized by silicon image sensors.

[0064] The invisible position-coding pattern 52 can be incorporated into a graphic display output device such as a display screen. The position-coding pattern 52 is not perceptible to the user and may not significantly alter the displayed image. In some embodiments, the display screen itself may be the light source and may display the generated position-coding pattern 52 for subsequent detection by the optical stylus 10. In that case, an additional patterning layer and illumination system must be incorporated into the display screen.

[0065] The disclosed optical stylus 10 can therefore be used in conjunction with conventional touchscreen technology, allowing the touch sensor to roughly determine the position of the optical stylus, thereby generating a position-coding pattern only in the vicinity of the optical input of the optical stylus, while leaving the remainder of the displayed frame unchanged.

[0066] In one embodiment, the optical stylus 10 is a stand-alone stylus. The housing 12 includes, to that effect, a processing unit 24, a battery 26, and a memory 28. Recognition of any position-coding pattern 52 and calculation of the position of the optical stylus 10 are performed by the processing unit 24, which may execute, for example, dedicated pattern recognition software and / or image perspective analysis software stored in the memory 28. The calculated data, including the position of the optical stylus, may be transferred to a host device in real time or may be temporarily stored in the optical stylus and later transferred to a host device for displaying traces generated by stylus movement and / or for permanent storage.

[0067] In another embodiment, the optical stylus can be configured to transfer data of the captured image of the position-coding pattern 52 to a host device that can run pattern recognition software and / or image perspective analysis software to calculate the position of the optical stylus 10. Data transfer can be achieved via a cable connecting the optical stylus to the host device. Alternatively, the housing 12 of the optical stylus 10 can include a wireless communication unit 30 for wirelessly communicating with the host device via, for example, Wi-Fi or Bluetooth protocols.

[0068] The battery can be recharged in the host device via a cable, or the optical stylus 10 may be docked in a docking station for wireless charging.

[0069] The optical stylus 10 may include one or more additional sensors, such as an accelerometer, a gyroscope, and a proximity sensor, to facilitate determining the position of the optical stylus 10 .

[0070] The block diagram of FIG. 7 essentially shows the different steps of an exemplary embodiment for determining the position of the optical stylus 10 relative to the position-coded surface 50.

[0071] The first step 100 consists of illuminating the position-coding surface 50 by the light-emitting unit 22 .

[0072] A second step 104 may involve adjusting the focal length of the optical arrangement 15 of the optical stylus 10 by adjusting the dynamic optical element 16. In the case of an open-loop approach, the second step may involve, prior to step 102, determining the distance between the distal end of the optical stylus 10 and the position-coding surface 50 by means of a distance measuring sensor 32 integrated into the optical stylus housing 12. The adjustment of the focal length of the optical arrangement 15 may also be performed according to a feedback or closed-loop approach previously described, whereby the previous step 102 is not performed.

[0073] A third step 106 consists of forming an image 21 by the optical arrangement 15 , and a subsequent step 108 consists of acquiring an optical signal by the image sensor 20 .

[0074] In certain embodiments, the step of acquiring the optical signal by the image sensor 20 can trigger the light emitting unit 22 to emit a light pulse only at this stage. This can be particularly useful in applications where the optical stylus 10 is moving, because a short illumination pulse can reduce motion-related blurring of the acquired optical signal. This also reduces energy consumption, thereby extending battery life.

[0075] Acquisition of the optical signal is followed by processing by a processing unit 24 located within or external to the optical stylus housing by a computing device. Signal processing may include any of the following process steps: image thresholding 114 and binarization 116, followed by extraction of rotation and tilt angles for viewpoint correction 118. The improved image is then processed by algorithms for pattern recognition 120 and position calculation 122.

[0076] In one embodiment, the 2D position of the optical stylus is calculated by identifying one or more tags or signatures on the position-coded surface 50 and extracting the position-related data provided by the tags / signatures that correspond to the position of the optical stylus relative to this surface. In a preferred embodiment, the position of the optical stylus is performed by solving a "Perspective-n-point" problem that allows for the determination of the position of the image sensor, including its 2D coordinates relative to the position-coded surface, its distance to this surface, and its orientation in space relative to this surface. Further positions of any reference point of the optical stylus or its projection onto the position-coded surface can be inferred from parameters obtained by solving the n-point perspective problem.

[0077] For more accurate position determination of the optical stylus, the processing unit 24 or an external computing device can take into account additional information from auxiliary sensors. This additional information from the auxiliary sensors can supplement and / or refine the position-related data estimated from the optical signals using a sensor fusion algorithm. For example, the three-dimensional coordinates and azimuth angle of the optical stylus can also be acquired or supplemented by one or more additional sensors, such as an accelerometer, a gyroscope, a magnetometer, and / or an inertial measurement unit (IMU). If multiple sensors are implemented, the data acquired by these sensors can be combined using a fusion algorithm.

[0078] Processing the optical signals of one or more images captured by the image sensor 20 can yield information useful for determining the position of the optical stylus. If the position-coding pattern 52 is asymmetric and its orientation is determined, the rotation angle 54 between the optical stylus 10 and the position-coding surface 50 can be determined, as shown in FIG. 2a. Furthermore, further analysis of the optical signals can reveal perspective distortions associated with the tilt angle 60 of the optical stylus 10 relative to the normal to the position-coding surface 50, which correlates with the single-point projection angle 56 (FIG. 2a), as shown in FIG. 2. This information can be used for perspective correction and to simplify pattern recognition. Further analysis of the optical signals can include calculating the field of view (FOV), which is directly related to the separation distance between the reference point of the optical stylus, e.g., its distal tip, and the position-coding surface 50.

[0079] The IMU can fully determine the 3D pose (3D coordinates and orientation) of the optical stylus 10 relative to a reference position, which can be estimated by processing images of the arbitrary coding pattern 52 of the coding surface 50 acquired by the image sensor 20. Thus, the estimated 3D pose of the optical stylus 10 can be refined using a combination of both the image of the arbitrary coding pattern 52 and the output of the IMU. The IMU-based position estimation can also take over from the image-based position estimation when the arbitrary coding pattern 52 is too small or too out of focus to provide a robust estimate of the 3D pose of the optical stylus 10.

[0080] Finally, a subsequent step 112 is performed to transfer the position-related data to the host device for graphical display. Comparing the current position to previously calculated positions, along with information from sensors such as accelerometers, can also reveal information related to the movement of the optical stylus. [Explanation of symbols]

[0081] Optical Positioning Device Optical Stylus 10 Housing 12 Distal portion 13 Aperture 13a Brush with filament bundle 14 Edge 14a, 14b Trace 14c contact area 14d Optical arrangement 15 Dynamic Optical Element 16 Fixed focus lens (one embodiment) Actuator Piezoelectric element Electrically Adjustable Lens (One Embodiment) Liquid or polymer lenses First and second liquids 16a, 16b Optical Components 18 Filters, mirrors, apertures, fixed focus lenses Function button 19 Image Sensor 20 CCD / CMOS sensors Capture image 21 Light-emitting unit 22 LEDs 22a, 22b, and 22c Processing Unit 24 Battery 26 Memory 28 Wireless communication unit 30 Bluetooth Module Wi-Fi module Additional Sensor 32 Distance measurement sensor Accelerometer Sensor Gyroscope Proximity Sensor Distance measurement sensor Inertial Measurement Unit (IMU) position coding plane 50 Position-coding pattern 52 Rotation angle 54 One-point projected angle 56 light source 58 Tilt angle 60

Claims

1. An optical stylus (10) for an optical position-determining device (1) comprising a position-coding surface (50) with different position-coding patterns (52), said optical stylus (10) comprising: an image sensor (20) for capturing at least one image of any position-coding pattern (52) of said position-coding surface (50); an optical arrangement (15) comprising a dynamic optical element (16) configured to be electrically actuated to adjust the focal length of the optical stylus (10) depending on the separation distance (D) between a reference point of the optical stylus (10) and the position of any position-coding pattern (52) so as to have a substantially focused position-coding pattern (52) corresponding to the position of the optical stylus (10); a control or processing unit (24) for controlling said dynamic optical element (16); a housing (12), an aperture or window (13 a) at the distal end of the optical stylus, the image sensor (20) being disposed within the housing (12), the aperture or window (13 a) being formed in the distal end of the housing (12) such that the image sensor (20) can capture at least one image of the position-coding pattern (52) through the aperture or window (13 a); Equipped with the optical stylus (10) is adapted to identify any substantially focused position-coding pattern (52) on the position-coding surface (50) independently of the separation distance (D) for determining the position of the optical stylus (10) relative to the surface (50); the optical stylus (10) further comprises a distance measuring sensor (32) for measuring the separation distance (D), and the control or processing unit (24) is adapted to operate the dynamic optical element (16) to adjust the focal length of the optical stylus (10) in response to an output signal of the distance measuring sensor (32); the control or processing unit (24) is configured to control the dynamic optical element (16) according to either a 3D mode or a 2D mode; In the 3D mode, an electrical signal is applied to the dynamic optical element (16), the electrical signal varying as a function of the separation distance (D) and varying the focal length of the optical stylus (10) accordingly.

2. An optical stylus (10) as described in claim 1, wherein in 2D mode, no changing electrical signal is applied to the dynamic optical element (16), so that when the distal end of the optical stylus (10) is in contact with the position coding surface (50) or within a range of up to 5 mm of the position coding surface (50), the focal length of the optical stylus (10) remains constant and any coding pattern (52) is substantially in focus.

3. 3. The optical stylus (10) of claim 2, wherein the control or processing unit (24) is configured to switch the optical stylus (10) between the 2D mode and the 3D mode depending on the output signal of the distance measuring sensor (32).

4. 3. The optical stylus (10) of claim 2, further comprising a function button (19) electrically connected to the control or processing unit (24) for switching the optical stylus between the 2D mode and the 3D mode.

5. 3. The optical stylus (10) of claim 2, further comprising a pressure sensor located at the tip of the stylus, and wherein the control or processing unit (24) is configured to switch the optical stylus (10) between the 2D mode and the 3D mode in response to the output signal of the pressure sensor.

6. The optical stylus (10) according to any one of claims 1 to 5, wherein the distance measuring sensor (32) is a time-of-flight sensor.

7. The dynamic optical element (16) comprises two of the following elements: a liquid or polymer lens electrically actuatable to adjust said focal length; a fixed focus lens attached to an actuator configured to shift the fixed focus lens along the optical path of the optical stylus (10) to adjust the focal length, said actuator being preferably a piezoelectric element; The optical stylus (10) according to any one of claims 1 to 6, wherein

8. The optical stylus (10) according to any one of the preceding claims, further comprising at least one light-emitting unit (22) adapted to illuminate the optional position-coding pattern (52).

9. An optical stylus (10) according to any one of claims 1 to 8, further comprising an inertial measurement unit (IMU).

10. 10. The optical stylus (10) of claim 1, further comprising a battery (26) and an electrical circuit including the processing unit (24), the image sensor (20) and a memory (28), wherein the battery (26) is configured to supply power to the electrical circuit, and the memory (28) stores image processing and / or pattern recognition algorithms for determining the position of the optical stylus (10) relative to the position-coding surface (50).

11. 11. The optical stylus (10) of claim 1, further comprising a specific shape at its distal end for writing, painting or calligraphy applications, said specific shape being in the form of a brush (14) with a bundle of filaments, another flexible tip or a pen-like tip, and said aperture or window (13 a) being located at the distal end of said optical stylus (10), such that images of both said specific shape and a corresponding position-coding pattern (52) can be captured together by said image sensor (20).

12. An optical position determination device (1) comprising an optical stylus (10) according to any one of claims 1 to 11, a host device comprising a screen, and a position-coding surface (50) having a different position-coding pattern (52), wherein the host device is configured to receive data from the optical stylus (10) in order to generate a trace on the screen following a movement of the optical stylus (10) relative to the position-coding surface (50).

13. 13. The optical position determining device (1) according to claim 12, wherein the position-coding surface (50) is an integral part of the host device.

14. 14. The optical position determining device (1) according to claim 13, wherein the position-coding surface (50) is made of a flexible material that is applied to the top surface of the screen of the host device.

15. 13. The optical position determining device (1) according to claim 12, comprising a stand-alone position-coding surface (50).

16. Optical position-determining device (1) according to any one of claims 12 to 15, wherein the different position-coding patterns (52) of the position-coding surface (50) are transparent in the visible range of the light spectrum.

17. 17. The optical position determination device (1) of claim 12, wherein the host device or stand-alone position-coding surface (50) comprises one or more light sources (58) configured to emit light outside the visible range of the light spectrum and arranged below the position-coding surface (50) to back-illuminate each of the position-coding patterns (52).

18. A method for determining the position of an optical stylus (10) relative to a position-coding surface (50) comprising different position-coding patterns (52), said optical stylus comprising: an image sensor (20) for capturing at least one image of any position-coding pattern (52) of said position-coding surface (50); an optical arrangement (15) comprising a dynamic optical element (16) adapted to be electrically actuated; a distance measuring sensor (32) for measuring the separation distance (D) between a reference point of said optical stylus (10) and the position of any position-coding pattern (52); a control or processing unit (24) for controlling said dynamic optical element (16); Equipped with The method comprises: a. holding the optical stylus (10) in the direction of the position-coding surface (50); b) actuating the dynamic optical element (16) to adjust the focal length of the optical stylus (10) in response to an output signal of the distance measuring sensor (32); c) acquiring, by the image sensor (20), optical signals of an image formed by the optical arrangement (15) of a substantially focused position-coding pattern (52) of the position-coding surface (50); d. Processing the optical signal to determine the position of the optical stylus (10) relative to the position-coded surface (50); Including, data representing any image projection of the position-coding pattern (52) captured by the image sensor (10) (1) wirelessly to a host device comprising a screen and configured to execute a computer vision algorithm to determine the position of the optical stylus after relative movement of the optical stylus with respect to the position-coded surface before displaying a trace on the screen; or (2) After relative movement of the optical stylus with respect to the position-coding surface, the processing unit (24) of the optical stylus executes a computer vision algorithm to determine the position of the optical stylus before transmitting position-related data to the host device for displaying a trace on the screen.

19. The optical stylus (10) further comprises a gyroscope and / or an accelerometer, and the position of the optical stylus is both the optical signal acquired by the image sensor (20) and the data acquired by the gyroscope and / or accelerometer; or Data acquired by the gyroscope and / or accelerometer The method of claim 18, wherein the calculated value is based on:

20. 20. The method according to claim 18 or 19, wherein the orientation of the optical stylus (10) is measured by a gyroscope and / or an accelerometer at least during acquisition of the optical signals under step c), and the tilt angle is used, for example, for correction of perspective distortion of the captured coding pattern (52) to provide more robust pattern recognition and position determination.

21. The method according to any one of claims 18 to 20, wherein the optical stylus (10) further comprises a light emitting unit (22), and the light intensity of the light emitting unit (22) is controlled according to the output signal of the distance measuring sensor (32).

22. 19. The method of claim 18, wherein the hue, stroke width and / or shape of the trace are adjusted in response to any one or a combination of two or more of the following parameters of an optical stylus: velocity, acceleration, direction and tip pressure on the position-coded surface.

Images