Position detection system for light points on surfaces and in space

Abstract

A position tracking system indicates: at least one position detection device (1); and a number of light sources (20 to 2N) which are controlled by the Position detection device (1) are spaced apart; where the position detection device (1) comprises at least two line image sensors (11, 11') wherein the extension directions of sensor surfaces (12, 12') of the two line image sensors (11, 11') enclose an angle with each other; A slit aperture (13, 13') is provided above each line image sensor (11, 11`) such that the slit aperture (13, 13') forms an angle of 90° with the associated sensor area (12, 12'); The position detection device (1) and the light sources (20 to 2N) each have interfaces for wireless communication, via which the position detection device (1) and the light sources (20 to 2N) are synchronized in time in order to enable detection of a respective light source (20 to 2N) by a time-division multiplexing method when emitting light by the light sources (20 to 2N).

Description

[0001] Systems that can detect the position of a light point (usually the light point of a laser beam) on a projection surface (e.g., a screen or wall) have been known for more than 20 years. The detected position is then sent in the form of X and Y coordinates to a computer, which further processes these X and Y coordinates. For example, the computer can set the position of a mouse cursor to these X and Y coordinates. If keystrokes are also transmitted, these can also be received and taken into account by the computer.

[0002] These systems, as described in WO 00 / 38102, for example, generally work as follows: A digital camera for generating spatially resolved images (e.g. a webcam) with, for example, a resolution of 1920 × 1080 pixels (picture elements) is aimed at a projection surface, and the image data output by the camera is searched for the pixel with the highest brightness value. This is usually the desired point of light, as the point of light is brighter than the rest of the projection surface (otherwise the point of light in the projection surface would not be visible to a user). The position of the pixel found in this way within a receiving surface (e.g. a CCD sensor) of the camera is determined by its design and is therefore known. Subsequently, the position of the point of light on the projection surface can be deduced from the position of the pixel within the receiving surface of the camera.For this purpose, a calibration between the camera and the projection surface can be performed beforehand. If a light source for the light point (e.g., a laser pointer) also has buttons, their status can also be transmitted (wired or wirelessly) to the camera or a computer connected to the camera. As a result, everything on the projection surface that can also be controlled with a computer mouse, for example, can be controlled with a laser pointer with a wireless module.

[0003] When using multiple light sources and therefore multiple light points at the same time, it is not sufficient to simply search for the pixel that outputs the highest brightness value.

[0004] To distinguish between different light points, one approach has been to use (laser) light with different wavelengths in combination with multiple cameras, each with adapted filters. However, this is a very expensive solution due to the use of multiple cameras.

[0005] It was further proposed to use a time-multiplexing method in which the individual light sources are switched on synchronously with the camera in predefined different time windows on the time axis T, but are switched off in the other time windows. This is Fig. 1 illustrates.

[0006] In the example of Fig. 1, the camera sends a synchronization signal S at regular intervals on the time axis T, which is received by the light sources so that all light sources operate synchronously with the camera. The light source with the number 0 emits light in the time window F00 (e.g. by switching on a laser) and interrupts the light output at the end of the time window (e.g. by switching off a laser). Then the light source with the number 1 emits light in the time window F01 and interrupts the light output again at the end of the time window, and so on. After the light has been emitted by light source 99, all light sources wait for the next synchronization signal S and this process is repeated endlessly (e.g. 20 times per second) so that the position of the light points on the projection surface attributable to light sources 1 to 99 can be recorded without excessive jumps.

[0007] Due to the short light output from the individual light sources, the light points generated by the individual light sources on the projection surface are generally not visually perceptible or only poorly perceptible to the user. For better user orientation, a computer can display an individual cursor (e.g., in different colors) for each light source on the projection surface, so that each user of a particular light source knows where they are pointing. The synchronization signal S can be transmitted to the light sources using cables or wirelessly, electrically, light-based, or radio-based.

[0008] However, even these previously known systems have severe limitations regarding the number of light sources that can be used simultaneously (although due to the time-division multiplexing process, only one light source emits light at a time). The reason for this is that (as with any camera), not only the brightness of the light point determines whether the light point can still be detected by the camera, but also the time, because the light energy (power times time) determines which output voltage a sensor generates for each individual pixel. The more light sources are used, the shorter the period of time during which a respective light source emits light during a time window with the time-division multiplexing process.

[0009] To solve this problem, the camera's sensitivity could be increased to compensate for the shorter time during which the light spot is generated. Sensitivity is directly proportional to the area of ​​the camera's individual pixels. However, increasing the size of each pixel in the X and Y directions leads to a quadratic increase in the sensor area, thus significantly increasing manufacturing costs. Furthermore, there are technical limitations, as such a large sensor area must be feasible in the first place.

[0010] With regard to the transmission of the synchronization signal required for the time-division multiplexing process, cable-based synchronization methods, in which the light sources are connected to the camera, are only very slightly susceptible to interference. However, they have the disadvantage that a user of the light source cannot move freely. Light-based or radio-based synchronization methods, in which the synchronization signal is transmitted between the camera and the light source via light or radio, do not have this disadvantage, but they are more susceptible to interference and are often affected by a back-and-forth wandering (jitter) of the synchronization signal S on the time axis T. This jitter results from the fact that the synchronization signal S must be encoded in such a way that it cannot be incorrectly detected. As a result, for example,In the 2.4 GHz band, only bit rates of 1 million bits / second are achieved, although the carrier signal is considerably faster at 2.4 billion oscillations / second.

[0011] From document US 3,951,550, it is known to use two stripe radiation detectors (such detectors are also known as line image sensors) instead of a standard spatially resolving camera to detect the position of a light spot on a projection surface. These detectors are rotated 90 degrees relative to each other and spaced apart in a common plane. The two stripe radiation detectors are separated from each other in a light-tight manner by a partition. Above each stripe radiation detector, a slit diaphragm is provided in a common plane, which in turn is rotated 90 degrees relative to the associated stripe radiation detector. Light can only fall onto the respective stripe radiation detector through the respective slit diaphragm. The plane in which the stripe radiation detectors are arranged is spaced parallel to the plane in which the slit diaphragms are arranged.One fringe radiation detector determines the X position of the pixel, the other fringe radiation detector determines the Y position of the pixel.

[0012] The solution known from US 3,951,550 has the disadvantage that the slit diaphragms assigned to the stripe radiation detectors are spaced apart from one another, resulting in a different aperture angle in the X and Y directions, since the length of the slit influences the field of view in one direction and the length of the stripe radiation detector influences the field of view in the other direction. However, it is desirable to have the same aperture angle in both the X and Y directions. Furthermore, the solution known from US 3,951,550 has the disadvantage that the slit diaphragm for the X direction and the slit diaphragm for the Y direction are contained in a single diaphragm disk, so that a further spatial separation of the stripe radiation detectors is not possible.

[0013] A stripe radiation detector / line image sensor as used in US 3,951,550 is described below with reference to Fig. 2 is explained in more detail.

[0014] Although not disclosed in US 3,951,550, the solution proposed in US 3,951,550 using a stripe radiation detector / line image sensor has the advantage that the sensitivity can be increased by increasing the height (Height H) of the individual pixels transverse to the direction of extension of the stripe radiation detector / line image sensor, without changing the width (W) of the individual pixels and thus the spacing of the individual pixels in the direction of extension of the stripe radiation detector / line image sensor. The width of the individual pixels determines the resolution achievable using the stripe radiation detector / line image sensor (accuracy of position determination); the height of the individual pixels determines the sensitivity proportionally, given the same width. The line image sensor has a total width L (Length), which corresponds to the number of pixels multiplied by the width of the individual pixels.

[0015] It is an object of the present invention to provide a position detection system for light points on surfaces and in space, which overcomes the disadvantages of the above prior art and solves one or more of the following problems: 1. Detection (i.e., determination of the position and discrimination) of extremely short-duration light points (less than 100 microseconds) generated by light sources on a projection surface; these short-duration light points can also be referred to as light flashes; for example, these light points can be generated by a laser diode controlled by a microcontroller, which radiates towards the projection surface. 2. Detection (ie determination of the position and discrimination) of light flashes of extremely short duration (less than 100 microseconds) from light sources moving in a plane (such as light-emitting diodes / LEDs moving on a surface controlled by a microcontroller). 3. Detection (i.e. determination of the position and discrimination) of extremely short-duration light flashes (less than 100 microseconds) from light sources moving in three-dimensional space (such as light-emitting diodes / LEDs controlled by a microcontroller moving in 3D space); the determination of the position in space is made possible by the use of several position detection devices described below, which "view" the space and thus the moving light sources from different angles. 4. Synchronization of a microcontroller control for the light sources used to enable differentiation of the light sources. 5. Transmission of states of multiple buttons of the light sources used or of other status information of the light sources, whereby the transmission takes place in the form of a light / light signal emitted by the respective light source; this light emitted by the respective light source is primarily used to determine the position of the light source or a light point produced by the light source on a projection surface and to distinguish between different light sources / light points.

[0016] These problems are solved by combining the features of the independent claims. Preferred developments are found in the dependent claims.

[0017] Embodiments of the invention are explained in more detail below with reference to the figures. Herein: Fig. 1 schematically shows a time-division multiplexing method from the prior art for distinguishing several light points on a projection surface; Fig. Figure 2 schematically shows a stripe radiation detector / line image sensor as can be used in the prior art and the present invention; Fig. 3 schematically shows a position detection system which uses a position detection device according to the invention in combination with light sources; Fig. 4 schematically shows the idealized output of a stripe detector / line scan sensor when viewing a white surface; Fig. 5 schematically shows a cross section through a line image sensor perpendicular to the direction of extension of a slit diaphragm; Fig. 6 schematically shows the idealized output of a fringe radiation detector / line image sensor when viewing a light flash; Fig. 7 schematically shows the idealized output of a fringe detector / line scan sensor when viewing a flash of light emitted at a flatter angle than in Fig. 6 comes to mind; Fig. 8 schematically shows the idealized output of a fringe radiation detector / line image sensor when viewing a light flash which is projected at a flatter angle than in Fig. 6 occurs, before and after boosting the signal; Fig. 9 schematically shows the signal output by the position detection device according to the invention after processing, which signal is generated by an incident light flash; Fig. 10 schematically shows a time-division multiplexing method for distinguishing multiple light sources according to the present invention; Fig. 11 schematically shows an alternative embodiment of the sensor surface of a line image sensor according to the present invention; Fig. 12 schematically shows a Fresnel lens which, according to the invention, can be arranged in front of a line image sensor in addition to a slit diaphragm or alternatively to the slit diaphragm.

[0018] In Fig. 3 schematically shows a position detection system which uses a position detection device according to the invention.

[0019] The position detection system comprises a position detection device 1 and a number of light sources 20 to 2N. The light sources can be, for example, laser pointers or LEDs.

[0020] The position detection device 1 comprises (at least) two line image sensors 11, 11', wherein the extension directions of sensor surfaces 12, 12' of the two line image sensors 11, 11' are rotated by 90° relative to each other and are arranged in a common plane.

[0021] It is emphasized that the present invention is not limited to an angle of 90° between the sensor surfaces of the line image sensors or to the use of only two line image sensors; rather, it is sufficient if the sensor surfaces of the line image sensors form an angle with each other (i.e., they are not all parallel to each other). Furthermore, the present invention is not limited to sensor surfaces located in a common plane.

[0022] In a plane parallel to the plane in which the sensor surfaces 12, 12' are arranged, a slit diaphragm 13, 13' is provided above each line image sensor 11, 11' such that the slit diaphragm 13, 13' forms an angle of 90° with the associated sensor surface 12, 12'. Fig. 3 shows only the part of the respective slit diaphragm 13, 13' that transmits light (i.e., the squares designated by reference numerals 13, 13' are translucent, whereas the position detection device 1 is otherwise surrounded by an opaque housing into which the slit diaphragms 13, 13' are incorporated; the housing is otherwise not shown). Thus, light can only pass through a respective slit diaphragm 13, 13' onto the associated sensor surface 12, 12'. The sensor surfaces 12, 12' are shielded from one another by partition walls so that the light falling on one sensor surface does not reach the other sensor surface.

[0023] The line image sensors 11, 11' are connected to a controller 15 (for example, a microcontroller) via data lines (not shown). The controller 15 is further connected to internal light sources 14, 14', a wired communication interface 16 for communication with a computer (which can be, for example, a USB interface), a wireless communication interface 17 for communication with the light sources 20 to 2N, and a memory 18 via data lines (not shown). The light sources 20 to 2N also each have interfaces for wireless communication and controllers (for example, microcontrollers), which, however, are not specifically shown, since light sources 20 to 2N with corresponding interfaces and controllers are known from the prior art. Even if only one controller 15 is shown above, multiple controllers can also be provided in the position detection device 1.

[0024] In Fig. 3, it was also omitted to show auxiliary systems such as energy sources and timers (timers), which are present both in the position detection device 1 and in the light sources 20 to 2n.

[0025] The output signal 1 of one of the line image sensors 11, 11', which results when a white surface is viewed with the line image sensor, is in Fig. 4. Here, U indicates a voltage amplitude during the sequential readout of the N pixels of the line image sensor, which are arranged at a position P in the extension direction of the line image sensor (left pixel 0, right pixel N-1): The individual pixels (photoelements) of the line image sensor have the property that they only generate a signal above a certain dark voltage 2. For brightness values ​​below this dark voltage, there is therefore no change in the output signal 1, ie everything below the line 2 generates (unlike in Fig. 4) the same minimum voltage.

[0026] In addition, Fig. 4 is idealized, since in practice the output signal 1 does not produce such a smooth curve, but contains all the light / dark values ​​of the observed white area. Furthermore, different values ​​arise due to the fact that the individual pixels of the line image sensor are not 100% identical and there may be impurities on the top side of the line image sensor. Every small shadow and every minimal difference in brightness of the white area is visible in the output signal and can change at any time, for example if the light in the room changes or if a window is opened or closed, thereby changing reflections in the room. This rather undesirable variation of the output signal 1 is referred to below as the "background image". The reason for the Fig. 4 shown curve shape is described with reference to Fig. 5 explained below.

[0027] If one considers the arrangement of, for example, the line image sensor 11 assigned to the X-direction with the slit diaphragm 13 mounted above it in cross-section transverse to the direction of extension of the slit diaphragm 13, the result is Fig. 5 schematically shown image.

[0028] Out of Fig. 5 shows that a light point LL on the left of the projection surface (or light emitted by a light source located on the left) is incident on a pixel SR on the right of the line image sensor S (reference number 11 in Fig. 3) is imaged, and a light point LR on the right of the projection surface (or light emitted by a light source located on the right) is imaged onto a pixel SL on the left of the line image sensor S. A light point LM in the center of the projection surface (or light emitted by a light source located in the center) is imaged onto the pixel SM in the center of the line image sensor S (this of course only applies if the position detection device is aligned exactly centrally on the projection surface). Part of the light is reflected off an upper (glass) layer of the line image sensor S as reflected light R, the non-reflected remainder reaches a respective photoelement of the sensor surface below. The distance of the line image sensor S to the slit diaphragm of typically 1 cm is negligible compared to the distance between the position detection device 1 and a (not shown) projection surface of typically > 100 cm.

[0029] The Fig. The curve shape shown in Figure 4 has three reasons: 1. In the arrangement shown, the projection point LG hits the sensor surface of the line scan sensor S at an angle of approximately 45°. The distance from the sensor point SL to the projection point LR is therefore, in the example shown, 1.41 times (square root of 2) the distance from the sensor point SM to the projection point LM. Since the light energy decreases quadratically with the distance, the light energy at the outer points of the sensor surface of the line scan sensor S is only half as high as at the center of the sensor surface of the line scan sensor S. 2. The effective slit width decreases with increasing angle (deviation from the light incident perpendicularly on the slit and thus the sensor surface of the line image sensor S), the effective slit width is Fig. 5 becomes narrower from the center outwards and thus lets less light through. 3. With increasing angle (deviation from the light incident perpendicular to the slit and thus the sensor surface of the line image sensor S), more and more light is reflected from the surface of the line image sensor (in Fig. 5 as “R”) instead of reaching photoelements (not shown) of the sensor surface of the line image sensor and becoming effective there.

[0030] If you now create a briefly flashing light point on the projection surface with a light source (light flash) or let a light source located there light up briefly (light flash), then the sequential reading of the N pixels (left pixel 0, right pixel N-1) creates the Fig. 6 schematically shown output signal 1 with a voltage amplitude U.

[0031] The light flash, which in the example is located to the left of the center on the projection surface, generates an amplitude increase 3A (peak) at the output of the line image sensor S.

[0032] The further this flash of light moves to the left in the example, the further the amplitude increase 3B moves to the right, becoming smaller (reasons as explained above) and eventually falling below the dark voltage 2, so that the position of the flash of light can no longer be detected. This is shown in Fig. 7 shown schematically.

[0033] In order to use all pixels of the line image sensor S / 11, it is therefore necessary to raise the signal at the two edges of the line image sensor S / 11, as shown schematically in Fig. 8 is shown.

[0034] This is done according to the invention in such a way that internal light sources (preferably LEDs) 14, 14' controlled by the microcontroller and provided in the position detection device 1 are directed onto the sensor surfaces 12, 12' of the line image sensors 11, 11' in such a way that they produce a Fig. 8. For this purpose, the internal light sources 14, 14' are provided between the sensor surfaces 12, 12' of the line image sensors 11, 11' and the associated slit diaphragms 13, 13'. The light falling from the internal light sources 14, 14' onto the sensor surfaces 12, 12' is added to the output signal 3B (which would result without the use of the light sources 14, 14') by the photoelements of the sensor surfaces 12, 12' of the line image sensors 11, 11' when additional light enters through the slit diaphragms 13, 13' from the outside by the photoelements of the sensor surfaces 12, 12' of the line image sensors 11, 11', thus forming a raised output signal 3C which lies above the dark voltage 2 (this falls in Fig. 8 with the X-axis), so that the position of the light flash can be detected, although the light flash is so far to the left that it is in the output signal 3B of Fig. 7 would no longer be recognized without the use of the internal light sources 14, 14'.

[0035] By using the internal light sources 14, 14', light flashes located far outwards relative to a light incidence perpendicular to the sensor surfaces can also be detected and their positions can be determined.

[0036] It was described above that the internal light sources 14, 14' are provided between the sensor surfaces 12, 12' of the line image sensors 11, 11' and the associated slit diaphragms 13, 13'. An improved variant of the irradiation of the sensor surfaces 12, 12' of the line image sensors 11, 11' according to the invention is an arrangement of the internal light sources 14, 14' next to the line image sensors 11, 11' on a circuit board (not shown) carrying the line image sensors 11, 11'. The internal light sources 14, 14' then radiate from below onto a (in Fig. 5) plate 13P, in which the slit diaphragms 13, 13' are formed. This plate 13P is then provided on its side facing the sensor surfaces 12, 12' of the line image sensors 11, 11' with a white diffuse reflector (e.g., matt white plastic) 131, as shown in Fig. 5. This is located directly next to the respective slit aperture 13, 13' and diffusely reflects the light onto the sensor surface below.

[0037] The light emitted by the internal light sources 14, 14' preferably has the same wavelength as the light emitted by the light sources 20 to 2N, because this light (or the output signal of the line image sensors 11, 11') must optionally also pass through an interference bandpass filter.

[0038] In the following, the signal processing according to the invention by the at least one controller 15 of the position detection device 1 is described.

[0039] At the output of the two line scan sensors 11, 11', there is a preamplifier (not shown) to significantly increase the signal-to-noise ratio. This is immediately followed by a so-called instrument operational amplifier (not shown). This is capable of subtracting another signal from its input signal and then further amplifying it.

[0040] The amplitude values ​​of the subtract signal for the two line image sensors 11, 11' assigned to the X position and the Y position are continuously recalculated by the at least one controller 15 based on the current output data of the two line image sensors 11, 11' and then sent to a digital-to-analog converter assigned to a respective one of the two line image sensors 11, 11' synchronously with the readout of the two line image sensors 11, 11' so that the instrument operational amplifier can subtract this signal. This first background image data is created by the position detection device 1 for both line image sensors 11, 11' in a first "dark image" (DarkSlice1) in the memory 18.When creating this DarkSlice1 in memory 18, which, in time-division multiplexing, preferably precedes the first used time window, no light source 20 to 2N may be active, except for ambient light or light reflected from the projection surface. It serves to record a first background image. The image data of this DarkSlice1 also includes the brightness values ​​of the irradiation of the sensors (especially from the internal light sources 14, 14'), which, of course, must also be subtracted.

[0041] Since the above measures lead to permanent corruption of the image data, these systems must be readjusted. This occurs within the framework of a second "dark image" (DarkSlice2), which the position detection device 1 creates for both line image sensors 11, 11' in the memory 18. In this DarkSlice2, which, in the time-multiplexing process, preferably lies before the first used time window but after DarkSlice1, no light source 20 to 2N may be active, except for ambient light or light reflected from the projection surface. It serves to record another background image, the data of which for the line image sensors 11, 11' is stored in the microcontroller's memory and subtracted from the current data of the line image sensors 11, 11' in the respective used time window of the light source 20 to 2Nn.

[0042] The second "dark image" (DarkSlice2) is also important because the individual pixels in the line scan sensors 11, 11' do not have identical values ​​for the photoenergy-to-voltage ratio (pixel nonuniformity), and the top surface of the line scan sensors 11, 11' may contain contamination. However, since these differences exist not only in the used time windows but also in DarkSlice2, these errors are also removed by subtraction. Errors caused by subtracting DarkSlice1 in the instrument operational amplifier and jumps in the digital-to-analog converter are also removed.

[0043] The corrected data from the line image sensors 11, 11' of the used time windows then pass through a low-pass filter from left to right and then from right to left (or vice versa). A nice symmetrical curve is obtained for each of the two line image sensors 11, 11', which Fig. 9 is shown.

[0044] Subsequently, the at least one controller 15 determines for each of the two line image sensors 11, 11' the mean value of the image processed as above and stored in Fig. 9. This can be done, for example, by drawing a straight line at 80% of the maximum value of the signal and then determining the midpoint between the two intersection points between the straight line and the signal (and thus the Fig. 9). Due to the low-pass filter, a value averaged over several pixels is obtained for both line image sensors 11, 11'.

[0045] To distinguish between different light sources 20 to 2N, a time-multiplexing method can be used, as in the prior art. With all optimizations, this method is as shown in Fig. 10 shown.

[0046] After the synchronous data packet S sent by the position detection device 1, DarkSlice1 (DS1) follows, followed by DarkSlice2 (DS2). In both cases, no light source 20 to 2N is active. In time slot F00, light source 20 transmits, and in the subsequent time slots, light source 21 transmits up to light source 299.

[0047] The final number 99 for the light source is just an example; for shorter time windows there can also be 1000 light sources or more.

[0048] In Fig. 11 shows an alternative arrangement of the sensor surfaces 12, 12' of the two line image sensors 11, 11' according to the invention. This alternative arrangement allows the sensor surface of each line image sensor to be composed of several partial sensor surfaces ZS0 to ZS3, wherein the partial sensor surfaces ZS0 to ZS3 can be arranged in overlapping parallel rows.

[0049] The partial overlap prevents dark zones. After the position detection device 1 has been completed, the overlaps can be determined by "running over" the entire distance L with an LED for the two line image sensors assigned to the X-direction and the Y-direction and can be permanently stored in the (non-volatile) memory 18 of the position detection device 1. This step can be omitted if it is otherwise ensured that the output voltages of the individual (particularly overlapping) line image sensors are not added together. It may also be possible that the averaging and the low-pass filters running in two directions make it possible to omit determining the overlap by measuring by "running over" the entire distance L with an LED.

[0050] By changing the distance between the slit diaphragm and the line scan sensors, the aperture angle of each line scan sensor can also be individually adjusted. In the figures discussed above, this aperture angle was shown as + / - 45 degrees. The aperture angle also directly controls the resolution of the position detection device 1, as well as the length of the line scan sensors and the number of pixels per line scan sensor.

[0051] In the variant of Fig. 11, several line image sensors can also be mounted at different angles according to the invention, so that the line image sensors located further out are mounted higher and higher on the outermost side. Such an effect could also be achieved with flexible line image sensors. Such an arrangement has the advantage that the incident light does not fall at an extremely steep angle (45 degrees) onto the surface of the line image sensor and / or a filter arranged above the line image sensor. Partial reflection can thus be reduced, and the interference bandpass filter can thus be implemented with a narrower band. However, the same effect can also be achieved using a linear Fresnel lens mounted on the line image sensors.

[0052] Detecting a respective light source according to the time-division multiplexing method described above requires that the light flashes generated by the light sources 20 to 2n be synchronized with the position detection device 1. In the case of multiple position detection devices 1, one position detection device is the timing transmitter, and the other position detection devices—like the light sources 20 to 2n (or the microcontrollers contained therein)—are timing receivers. Position detection devices that are timing receivers can also be timing transmitters; however, in this case, each position detection device transmits at different times with specially marked data packets that identify the position detection device as an additional timing transmitter.The use of several position detection devices operating as time-lock transmitters can be advantageous if, in large installations, one position detection device cannot cover the entire space to be detected with radio.

[0053] In principle, it would be conceivable to use 20 to 2n precision quartz crystals in both the position detection device and the microcontrollers of the light sources, achieving an accuracy of, for example, 5 ppm. However, even these would already be 5 microseconds out of the specified time window after just 1 second.

[0054] Since the inventive solution is intended to obtain at least 10,000 values ​​per second from the two line image sensors assigned to the X position and the Y position, a time window for one value is 100 microseconds long. To prevent overlap from one time window into the next adjacent time window, in one example, the duration of a light flash emitted by light sources 20 to 2n can be set to 96 microseconds and a safety distance to 4 microseconds. The synchronization between the position detection device and light sources 20 to 2n, as well as between light sources 20 to 2n, must therefore be better than + / - 2 microseconds.

[0055] This problem could be solved with wired synchronization, but this is associated with the disadvantages mentioned above.

[0056] According to the invention, the following solutions are therefore proposed: 1. Synchronization with light pulses 2. Synchronization with radio signals

[0057] For synchronization with light pulses, the time of the end of an optical carrier wave, for example, at 455 kHz, can be detected. For this purpose, corresponding receivers are installed in the light sources 20 to 2n and connected to the corresponding controllers. Furthermore, the position detection device has a corresponding emitter for the carrier wave.

[0058] For synchronization with radio signals, radio signals, particularly the 2.4 GHz band (DECT, Wi-Fi, Bluetooth, etc.), can be used. However, any other permitted radio band can also be used.

[0059] The problem with using conventional chips to provide synchronization with radio signals is that these systems have built-in functions designed to guarantee the error-free nature of the transmitted data and can even trigger and automatically perform the retransmission of data packets in the event of an error. However, this leads to an unpredictable, fluctuating time at which a data packet is deemed complete and ready for pickup.

[0060] To solve this problem, the use of special transmit and receive controllers in the wireless communication interfaces 17 in the position detection device, as well as the corresponding interfaces of the light sources, is proposed. After each data packet, a short synchronous pattern initiated by the controller 18 of the position detection device is modulated onto the carrier wave (e.g., 2.4 GHz), which is still active without data. The receive controller waits a short time after each data packet to see if such a synchronous pattern appears on the carrier wave. Immediately at the end of the synchronous pattern, a signal is generated that stores a timer value in the microcontroller or, preferably, stores a timer value internal to the receive controller, which can be read out after the data has been decoded and verified. This reduces the jitter (time fluctuations) of the synchronization to less than 100 nanoseconds.

[0061] As an alternative to the use of dedicated transmit and receive controllers, the invention proposes the use of an existing precursor to Bluetooth chips, in which all protocol functions such as CRC detection, etc., are disabled, and a "ready to pick up" signal is emitted immediately after the last bit transmitted on the carrier wave. This signal exhibits almost no jitter. The microcontroller remembers the time of the internal timer and only then decodes the received data, including error detection and verifying whether the data was actually sent by the associated position detection device. This verification is necessary because there are many other transmitters in the 2.4 GHz band.

[0062] Once the received radio signal has been verified to be error-free and the transmitter has been qualified, the transmission end time is compared with the time an internal timer should have (reference constant). If the measured time is behind the reference constant, the respective microcontroller (or its timer) is too fast; if the measured time is ahead of the reference constant, the respective microcontroller (or its timer) is too slow.

[0063] To adjust the speed, it would theoretically be possible to simply slow down or speed up the oscillator of the respective microcontroller or its timer. This is either not possible with quartz-based microcontrollers or is very complex. With RC oscillators, which are preferred for cost reasons, speed adjustment is possible, but in much too coarse steps. Furthermore, RC oscillators change their speed depending on voltage and temperature.

[0064] According to the invention, it is therefore proposed that when a time discrepancy is detected between the clock signal received via radio and an internal clock signal, the time constants at which the microcontroller is to perform certain tasks, such as switching the light sources on or off, are not changed, rather. These constants are stored in a table in the light sources' internal memories and are recalculated by the light source controllers after each time comparison.

[0065] These constants also define a time window in which the controls of the position detection device and the controls of the light sources expect the next synchronization packet including the safety distance, which leads to an additional reduction of errors due to "false transmitters" that do not belong to the position detection system according to the invention.

[0066] A further improvement proposed according to the invention is the use of two or more wireless communication interfaces 17 in the position detection device. Since the 2.4 GHz band is used by many device types (DECT, WLAN, Bluetooth, etc.), it must be ensured that sufficient synchronous data packets can be transmitted error-free. Since every transmitter in the 2.4 GHz band first determines whether a channel (80 frequencies in the 2.4 GHz band) is free before it is allowed to transmit, a wireless communication interface blocks the channel with a single data packet whose end time is known in the position detection device. For this data packet, it is irrelevant whether it is received by the light sources or not; it is marked accordingly in the data and, if error-free, is evaluated at the receiving end as the first synchronous data packet and discarded in the event of an error.In any case, the second synchronous data packet is sent from the second wireless communication interface directly after the first synchronous data packet and marked as the second synchronous data packet. This packet is highly likely to be received by the light sources, unless the channel is intentionally jammed, for example, by a continuous transmitter. This improvement (the use of two wireless communication interfaces) can, of course, be implemented in a single module. Alternatively, two wireless communication interfaces can be used, whose start signals are generated by one of the position detection device's controllers directly based on its internal timer.

[0067] In addition, command packets are sent from the position detection device to the controllers (microcontrollers) of light sources 20 to 2n at predetermined times, allowing users to, for example, feel different vibration patterns depending on the actions they have triggered. In addition to vibration, sound sequences, voice sequences, etc. are also possible. For this purpose, light sources 20 to 2n are equipped with appropriate vibration motors, loudspeakers, or displays.

[0068] In addition, the invention provides for commands from the position detection device to the light sources that set the number of light flashes per second to a maximum value. If such commands are not received, the controllers (microcontrollers) of the light sources 20 to 2n automatically reduce the flashing frequency of the light sources 20 to 2n. Only when the light sources 20 to 2n generate light flashes in the projection area / field of view are these commands regularly sent from the position detection device to the individual light sources 20 to 2n, resulting in a significant decrease in light energy when the light sources 20 to 2n are pointed elsewhere.

[0069] According to the invention, all data packets are transmitted by the position detection device at different transmission powers, preferably initially at low transmission power and then repeatedly transmitted at increasing transmission power. Packet numbering allows the controllers (microcontrollers) of the light sources 20 to 2n to determine whether a data packet has already been received at low transmission power. In this case, the controllers (microcontrollers) of the light sources 20 to 2n discard the duplicates.

[0070] The different transmission powers are necessary because the distance between the position detection device and the controllers (microcontrollers) of light sources 20 to 2n is unknown. Controllers (microcontrollers) of light sources 20 to 2n whose receiver antennas are close to the position detection device will already receive the low-power data packets and may not even be able to receive the following higher-power data packets at all because an amplifier in the light source's receiver is overdriven. Therefore, it is proposed to initially transmit with a low transmission power and then increase it. Controllers (microcontrollers) of light sources 20 to 2n whose receiver antennas are farther away from the position detection device will not receive the low-power data packets at all, but will not overdrive them either, and will wait for the higher-power data packets that match their distance.

[0071] According to the invention, the transmission of the state of the keys or other status information of the light sources 20 to 2n also takes place during the light flash, and with a larger number of bits also shortly after the light flash, but with a high frequency and short ON times and long OFF times, which have almost no influence on the line image sensors, since photoelements always integrate the light output over time.

[0072] A special photosensor 19 in the position detection device 1, which can amplify this high frequency and filter it out from a mixture of interfering light pulses (remote controls, fluorescent lamps, etc.), "observes" the entire projection surface or the entire 3D space. Since the time at which the position light flashes are generated is known to the position detection device, the times at which data bits must be transmitted to the position detection device are also known to within + / - 2 microseconds.

[0073] At these times, the amplifier's output signal is decoded, and the data is transmitted to a computer connected to the position detection device, individually for each light source (20 to 2n). By using the same transmission method (modulated light flash) for the position and the button status, a 100% correlation between position and button press is achieved, which is only possible with a mixed variant (position light, button radio) with considerably greater effort.

[0074] To adapt the position detection system to daylight, appropriate filters can be mounted on the sensors. In particular, a so-called interference bandpass filter can be used, which allows an extremely narrow wavelength band to pass through while filtering out almost 100% of the rest of the light spectrum.

[0075] The wavelength band of the interference bandpass filter depends on the angle of incidence of the light. The interference bandpass filter is selected so that it allows the wavelength of the transmitters used in the wireless communication interfaces to pass through, even at an angle of + / - 45 degrees (relative to 0 degrees perpendicular).

[0076] According to the invention, it is further proposed to use a combination of a narrow interference bandpass filter and a lens, preferably a Fresnel lens or a nano-lens, which is simply considerably flatter and more precise and is mounted directly in front of the filter and directs the obliquely incident light almost straight.

[0077] Such a lens, preferably a Fresnel lens or a nanolens, can also be constructed to replace the slit or to "collect" significantly larger amounts of light energy before the light is directed onto the slit and to the line scan sensor behind it. In this case, the image is simply reduced in size, as in a light-sensitive telescope, where the front lens should have the largest possible diameter.

[0078] A corresponding Fresnel lens is shown schematically in Fig. 12 shown.

[0079] The Fresnel lens collects more light in one direction, but does not change the angle of the incident light in the other direction (the direction of the sensor). The line scan sensor is located along the focal line of the linear Fresnel lens and forms an angle of preferably 90° with the focal line.

[0080] The longitudinal extension of the Fresnel lens is oriented in the same way as the longitudinal extension of a slit diaphragm assigned to a respective line image sensor.

[0081] According to the invention, it is further proposed to use the position detection device for detecting the position of light points in space and for identifying the respective light points.

[0082] In order to use the position detection system for position detection in 3D space, at least two position detection devices, as described above, must detect a respective light flash generated by a light source from different angles. For this purpose, the at least two position detection devices can be mounted, for example, on the ceiling of a room. Using simple linear equations in space, the position of the light source emitting the light flash is calculated in 3D space from the two pairs of XY coordinates determined by the position detection devices.

[0083] When using laser pointers as a light source, an additional LED flash is attached to the laser pointer so that the position detection devices can detect the laser pointer's position in space. Combined with the X / Y coordinate on the projection surface, which results from the spot of light created by the laser pointer on the projection surface, and the position of the laser pointer in 3D space determined from the LED flash, the laser beam path in 3D space can be calculated using simple linear equations in space.

[0084] Preferably, more than two position detection devices are used to determine the position in the room, reducing the likelihood that other people or objects in the same room will block the line of sight between a position detection device and a light source whose position is to be detected. Since the position detection devices transmit an "off-screen" status, only valid pairs can be used, i.e., those reported with a sufficiently high amplitude.

[0085] According to the invention, it is further proposed not to use a single light source (e.g., LED) as the transmitter of an (LED) light flash, but preferably several light sources (e.g., LEDs) that radiate with a smaller aperture angle but thereby generate significantly higher light energy per steradian. Furthermore, the invention proposes not to flash all of these different light sources at the same time, but to assign different time windows to the light sources. Depending on which light sources are detected, the orientation of the light source and the orientation of an object to which the light source is attached can also be detected.

[0086] The proposed use of LEDs as a light source is also particularly advantageous because, since LEDs can transmit significantly higher pulse currents than laser diodes (for safety reasons), it is possible to increase the distance between LEDs and position detection devices and / or shorten the time windows, allowing significantly more light flashes from different light sources to be detected. This allows either more objects / people to be detected, or more tracking points per object / person.

[0087] The invention is repeated below in other words: For over 20 years there have been systems that detect the position of a laser beam on a projection surface and then send this data in the form of X and Y coordinates to a computer. The computer then sets the position of the mouse cursor to these X and Y coordinates and also transmits keystrokes. The principle is very simple: you take a normal camera (e.g. webcam) and point it at the screen. Since the light spot of the laser beam is always the brightest point, you simply look for this brightest point in the camera's image data and can then transmit X and Y. The state of the keys can also be transmitted wirelessly, for example, to the camera or directly to the computer. This means that with a laser pointer with a wireless module you can easily control anything that can be controlled with a computer mouse.

[0088] However, all of these developments could only achieve this with a single laser pointer; there were few that could achieve this with multiple pointers. Solutions such as using lasers with different wavelengths were used, which, however, would have required different filters for multiple cameras. One invention also used a time-division multiplexing process, in which the individual laser pointers are switched on synchronously with the camera in predefined, different time windows on the time axis T, but switched off in the others. [Fig. 1]

[0089] The camera sends a synchronization signal S and the laser pointers receive it so that all laser pointers work synchronously with the camera. The pointer with the number 0 switches its laser on in time window F00 and then immediately off. Then the pointer with the number 1 switches its laser on in time window F01 and then immediately off, and so on. After pointer 99, all laser pointers wait for the next synchronization signal S and this process is repeated endlessly, e.g. 20 times per second so that the cursor movement for each laser pointer is smooth and without jumps. For each laser pointer, you can then create an individual cursor (e.g. different colors) and each user knows where they are pointing.

[0090] Cable-based, light-based or radio-based methods were used as the synchronization signal S.

[0091] However, these systems also had severe limitations regarding the number of laser pointers that could be used. As is well known with any camera, it's not just the brightness of the light point that determines whether it can still be detected by the camera, but also the time, because the light energy (power times time) determines the output voltage a sensor generates for each individual pixel. The more laser pointers you want to support, the shorter the time per laser pointer becomes.

[0092] However, the sensitivity of the cameras could be increased to compensate for the shorter time. Sensitivity is directly proportional to the area of ​​the individual camera pixels. However, the X and Y dimensions of each pixel lead to a quadratic increase in the area of ​​the sensors, thus significantly increasing manufacturing costs, to the point where such a large sensor area can no longer be manufactured at all.

[0093] Cable-based synchronization methods are very susceptible to interference, but have the disadvantage that you cannot move freely.

[0094] Light- or radio-based synchronization methods do not have this limitation, but they are more susceptible to interference and generate a back-and-forth (jitter) of the synchronization signal S along the time axis T. This jitter results from the fact that the synchronization signal S must be encoded in such a way that no misdetection can occur. For example, in the 2.4 GHz band, only bit rates of 1 million bits / second are achieved, even though the carrier signal is considerably faster at 2.4 billion oscillations / second.

[0095] A device has already been described that fundamentally solves the pixel area problem. Instead of an area camera, two so-called linear image sensors are used (even though a different sensor is described in the above-mentioned patent). These are rotated by 90 degrees to each other and above each of which a slit diaphragm is mounted, which in turn is rotated 90 degrees to the line sensor. One linear image sensor determines the X position, the other the Y position. However, the arrangement in the above-mentioned patent is not optimal; a square arrangement for the two sensors should be used, since the length of the slit influences the field of view in one direction and the length of the line sensor influences the field of view in the other direction, and one wants to have the same aperture angle in both the X and Y directions. In addition, one would also like to be able to spatially separate the X and Y sensors, which is not possible with the arrangement described in the above-mentionedThe arrangement shown in the patent would not be possible because the X and Y slit diaphragms are contained in a single diaphragm disk. [Fig. 2]

[0096] The solution using a linear image sensor has the advantage that sensitivity can be increased by increasing the height (H) without changing the pixel width (W). The pixel width determines the resolution (position detection accuracy), and the height proportionally determines the sensitivity for the same width. The line sensor has a total width L (length), which is the number of pixels multiplied by the pixel width.

[0097] This invention solves several problems: 1. Detection of extremely short light flashes (less than 100 microseconds) on a projection surface, which are generated, for example, by a microcontroller-controlled laser diode. 2. Based on this, the detection of extremely short flashes of light, which are generated, for example, by microcontroller-controlled moving LEDs on a surface. 3. Based on this, the detection of extremely short flashes of light, which are generated, for example, by microcontroller-controlled moving LEDs in 3D space, by using several cameras that “view” the space and thus the LEDs from different angles. 4. High-precision synchronization of the microcontroller control for Pos. 1 and Pos. 2 and Pos. 3. 5. The transmission of the states of multiple buttons or other status information in the light signal to Pos. 1 and Pos. 2 and Pos. 3, which is primarily used for position detection.

[0098] If a slit diaphragm as described in the above-mentioned patent is mounted rotated by 90 degrees over a line sensor, then the following output signal 1 with a voltage amplitude U is generated when reading the N pixels sequentially (left pixel 0, right pixel N-1) on a white surface to which this arrangement is directed: [ Fig. 4]

[0099] However, the individual pixels (photoelements) have the property that they only generate a signal above a certain dark voltage 2; for brightness values ​​below this dark voltage there is no change in the output signal, ie everything below line 2 generates the same minimum voltage.

[0100] Furthermore, the output signal is naturally not such a smooth curve, but contains all the light / dark values ​​of the white area, including varying values, due to the fact that the individual pixels of the line sensor are not 100% identical and there may be impurities on the surface of the line sensor. Every small shadow and every minimal difference in brightness of the white area is visible in the output signal and can change at any time, for example, if the light in the room is changed or even if a window is opened or closed. In the following, we will refer to this as the background image.

[0101] If you look at the arrangement of the X line sensor with the slit diaphragm mounted above it from the side, [ Fig. 5]

[0102] then you can see that a light point LL on the left of the projection surface is imaged onto a pixel SR on the right of the line sensor S, a light point LR on the right of the projection surface is imaged onto a pixel SL on the left of the line sensor. The light point LM in the middle of the projection surface is imaged onto the pixel SM in the middle of the line sensor. This of course only applies when the camera is aligned exactly in the middle of the projection surface. Part of the light is reflected by the upper glass layer of the sensor (R), the rest reaches the respective photoelement below. The distance between the sensor and the slit diaphragm is negligible (e.g. 1 cm) compared to the distance between the camera and the projection surface (100 cm).

[0103] There are three reasons for the shape of the curve: 1. In this + / -45-degree arrangement, the distance between sensor point SL and projection point LR is 1.41 times (square root of 2) the distance between sensor point SM and projection point LM. Since the light energy decreases quadratically with the distance, the light energy at the outer points is only half that of the center. 2. The slit width decreases with increasing angle (0 degrees center), it becomes narrower towards the outside and thus lets through less light. 3. As the angle increases, more and more light is reflected from the sensor surface instead of reaching the photoelements.

[0104] If you now create a briefly flashing light point on the projection surface with a laser (flash of light) or an LED located there (flash of light), the following output signal 1 with a voltage amplitude U is generated when reading out the N pixels sequentially (left pixel 0, right pixel N-1): [ Fig. 6]

[0105] The light flash located to the left of the center on the projection surface creates an amplitude increase (3A, peak) at the sensor output.

[0106] The further you move this flash of light to the left, the further the peak (3B) moves to the right, becoming smaller (3B, reasons as above) and eventually falling below the dark voltage (2), so that the position of the flash of light can no longer be recognized: [ Fig. 7]

[0107] In order to use all pixels of the line sensor, the signal must be increased at both edges: [ Fig. 8]

[0108] To do this, we direct LEDs controlled by the microcontroller onto the surface of the line sensors in such a way that they would generate an output signal 4 there, to which the output signal 3B is added by the photoelements and the output signal 3C is created, although it is further out than the output signal 3B.

[0109] Now the outer flashes of light are also visible and their positions can be determined.

[0110] An improved version of illuminating the sensors is to place the LEDs next to the line sensors on the circuit board and direct the light upwards onto a white, diffuse reflector (e.g., matte white plastic). This reflector is located directly below the slit aperture and diffusely reflects the light onto the line sensors. This light must have the same wavelength as the laser diodes or LEDs, because this reflected light must also pass through the interference bandpass filter.

[0111] The entire device in the camera's position detection section consists of one or more microcontrollers, two line sensors, each with a 90-degree slit mounted above it. The two line sensor arrays are in turn rotated 90 degrees relative to each other. The 90 degrees are not mandatory but are provided as an example, but not 0 degrees (parallel).

[0112] At the output of the two line sensors, there is a preamplifier to significantly increase the signal-to-noise ratio. This is immediately followed by an instrumentation operational amplifier. This subtracts another signal from the input signal and then further amplifies it.

[0113] The amplitude values ​​of the subtract signal for X and Y are continuously recalculated by one of the aforementioned microcontrollers based on the current X and Y image data and then sent to the respective X and Y digital-to-analog converter synchronously with the readout of the X and Y line sensors so that the instrument operational amplifier can subtract this signal. This initial background image data is created in memory by the camera for both X and Y sensors in a first "dark image" (DarkSlice1). In this DarkSlice1, which preferably occurs before the first time window used, no laser pointer or LED may be active; it is used to record an initial background image. The image data for this DarkSlice1 also contains the brightness values ​​of the sensors' irradiation, which of course also have to be subtracted.

[0114] Since all of these measures lead to permanent image data corruption, these systems must be adjusted. This occurs within the framework of a second "dark image" (DarkSlice2), which the camera creates in memory for both X and Y sensors. In this DarkSlice2, which preferably occurs before the first time slot used but after DarkSlice1, no laser pointer or LED may be active. It is used to capture another background image, whose X and Y data are stored in the microcontroller's memory and subtracted from the current data for the laser pointer or LEDs in the respective time slot used.

[0115] The second "dark image" (DarkSlice2) is also extremely important, as the individual pixels in the line sensors do not have identical values ​​for the photoenergy-to-voltage ratio (pixel nonuniformity), and the top surface of the line sensors may contain contamination. However, since these differences exist not only in the used time windows but also in DarkSlice2, these errors are also removed by subtraction. The errors generated by subtracting DarkSlice1 in the instrument operational amplifier and the jumps in the digital-to-analog converter are also removed.

[0116] A low-pass filter in the camera firmware is now run over the corrected X and Y data of the used time windows from left to right and then from right to left (or vice versa) and the result is a nice symmetrical curve in the X and Y channels: [ Fig. 9]

[0117] Now, there are several methods for determining the center point M. The simplest is to draw a straight line at, say, 80% of the maximum value and then determine the center point between the two intersection points of the straight line and the curve. This gives you an X or Y value averaged over several pixels (due to the low-pass filter).

[0118] The time-multiplex diagram with all optimizations looks like this: [ Fig. 10]

[0119] The synchronous data packet is followed by DarkSlice1 (DS1), then DarkSlice2 (DS2). In both cases, no laser diode or LED is active. In time window F00, the laser diode or LED transmits 0, then 1 up to 99.

[0120] The final number 99 is just an example; for shorter time frames it can also be 1000 or more.

[0121] Another advantage of this sensor arrangement is the possibility of placing several line sensors in a row: [ Fig. 11]

[0122] The partial overlap prevents dark zones. After the camera is completed, the overlaps can be detected by "tracing" the entire distance L with an LED for the X and Y sensors and permanently stored in the camera's non-volatile memory. This step can be omitted by ensuring that the output voltages of the individual line sensors are not added together. It may also be possible to omit it by determining the average value and using the bidirectional low-pass filters.

[0123] By changing the distance between the slit aperture and the line sensors, the aperture angle of each line sensor can also be individually adjusted. In previous drawings, this aperture angle was represented as + / -45 degrees. The aperture angle also directly controls the camera's resolution, as well as the length of the line sensors and the number of pixels per line sensor.

[0124] Multiple line sensors can also be mounted at different angles, so that the outermost line sensors are mounted higher and higher on the outermost side. This could also be achieved with flexible line sensors, although these are not currently available. This arrangement has the advantage that the incident light does not fall at an extreme angle (45 degrees) onto the surface of the filter and / or the line sensors. This reduces partial reflection and allows the interference bandpass filter to be implemented with a narrower bandwidth. However, the same effect can also be achieved with a linear Fresnel lens mounted on the line sensors.

[0125] The entire process can only work if the laser pointers or LEDs generate light flashes synchronously with the camera(s). In the case of multiple cameras, one acts as the timing transmitter, and the others, like the microcontrollers of the laser diodes or LED transmitters, act as timing receivers. Cameras that act as timing receivers can also act as timing transmitters, but in this case, each camera receives a different time with specially marked data packets that identify them as additional timing transmitters. This is necessary when a camera cannot wirelessly cover the entire area to be recorded in large installations.

[0126] One could use precision quartz crystals in both the camera and the flashlight microcontrollers, achieving an accuracy of, say, 5 ppm. But even these would be 5 microseconds out of the specified time window after just 1 second.

[0127] Since we want to obtain at least 10,000 X / Y values ​​per second, a time window is 100 microseconds long. To prevent overlap from one time window to the next, we define the flash duration as 96 microseconds, with 4 microseconds as a safety margin. Synchronization must therefore be better than + / - 2 microseconds.

[0128] In addition to the unhelpful wired synchronization, two methods are used: 1. Synchronization with light pulses 2. Synchronization with radio signals

[0129] We already used synchronization with light pulses in 2000. This involved detecting the end of an optical carrier wave at 455 kHz. However, since the components that decode light signals with 455 kHz carriers are no longer available, and we also want to be able to send commands to the laser pointers, we opted for radio and the 2.4 GHz band (DECT, Wi-Fi, Bluetooth, etc.). However, any other permissible radio band will also work.

[0130] However, all chips for these systems have built-in functions that are intended to guarantee the error-free nature of the transmitted data and can even trigger and independently carry out the repetition of data packets in the event of an error.

[0131] However, this leads to an unpredictable fluctuating time at which a data packet is deemed to have been received and is ready for collection.

[0132] An optimal version for this can be implemented in custom transmit and receive controllers by modulating a short synchronous pattern initiated by the camera microcontroller after a data packet onto the carrier wave (e.g., 2.4 GHz), which is still active without data. The receive controller waits a short time after each data packet to see if such a synchronous pattern appears on the carrier wave. Immediately at the end of the synchronous pattern, a signal is generated that stores a timer value in the microcontroller or, preferably, stores a timer value internal to the receive controller, which can be read after decoding and verifying the data. This reduces the jitter (time fluctuations) of the synchronization to less than 100 nanoseconds.

[0133] Until we can manufacture our own chips that can control the beginning and detect the end of the carrier wave (2.4 GHz), we use a precursor to Bluetooth chips. In this chip, we disable all protocol functions such as CRC detection, etc., and receive a "ready to pick up" signal immediately after the last bit transmitted on the carrier wave. This signal has almost no jitter. The microcontroller remembers the time of the internal timer and only then decodes the received data, including error detection and checking whether the data was actually sent by our camera, since there are numerous transmitters in the 2.4 GHz band.

[0134] Once we've verified the transmitter is error-free and qualified, we compare the transmission end time with the time the internal timer should have (reference constant). If the measured time is behind the reference constant, our microcontroller is too fast; if the measured time is ahead of the reference constant, our microcontroller is too slow.

[0135] Now, theoretically, it would be conceivable to simply make the microcontroller's oscillator slower or faster. This is either impossible with quartz-based microcontrollers or is very complex. With RC oscillators, like those we use in our inexpensive microcontrollers, adjusting the speed is possible, but in much too coarse steps. Furthermore, RC oscillators change their speed depending on voltage and temperature.

[0136] So, we're not changing the speed of the oscillator, but rather the time constants at which the microcontroller should perform certain tasks, such as turning the laser diodes or LEDs on or off. These constants are stored in a table and are recalculated after each time comparison.

[0137] These constants also define a time window in which the microcontroller expects the next synchronization packet including the safety margin, which leads to an additional reduction of errors caused by “false transmitters”.

[0138] A further improvement is the use of two or more transmitters in the camera. Since the 2.4 GHz band is used by many types of devices (DECT, WLAN, Bluetooth, etc.), it must be ensured that enough synchronous data packets can be transmitted error-free. Since every transmitter in the 2.4 GHz band first determines whether a channel (80 frequencies in the 2.4 GHz band) is free before it is allowed to transmit, a module blocks the channel with a single data packet whose end time is known in the camera. For this data packet, it does not matter whether it "gets through" or not; it is marked accordingly in the data and is evaluated on the receiving side as the first synchronous data packet if there are no errors and discarded if there is an error. In any case, the second synchronous data packet with the second module is sent directly after the first synchronous data packet and is marked as the second synchronous data packet.This will most likely "get through," unless someone intentionally jams the channel with, for example, a continuous transmitter. This improvement can, of course, be implemented in a single module using dedicated transmitter controllers. Until then, we'll use two transmitter modules, whose start signals are generated directly by one of the camera microcontrollers based on the internal timer.

[0139] In addition, command packets are sent from the camera to the microcontrollers of the laser diodes or LEDs at predefined times, allowing users to feel different vibration patterns, for example, depending on the actions they have triggered. In addition to vibration, sound sequences, voice sequences, etc. are also possible.

[0140] There are also commands that set the number of light flashes per second to the maximum value. If such commands aren't received, the microcontroller automatically reduces the flashing frequency of the laser diodes or LEDs. Only when the laser diodes generate light flashes within the projection area / field of view are these commands regularly sent from the camera to the individual laser pointers, resulting in a significant decrease in light energy when the laser pointers are pointed elsewhere.

[0141] All data packets are sent by the camera at different power levels, preferably with a low power level at the beginning and then repeated several times with increasing power levels. Packet numbering allows the microcontroller to determine whether it has already received a data packet at a low power level. In this case, the microcontroller discards the duplicates.

[0142] The different transmission powers are necessary because the distance between the camera and the microcontrollers is unknown. Microcontrollers whose receiver antennas are close to the camera will already receive the low-power data packets and may not even receive the following higher-power data packets at all because the amplifier in the receiver is overdriven. Hence the preferred direction from low to high. Microcontrollers whose receiver antennas are farther away from the camera will not receive the low-power data packets at all, but will not overdrive them either, and will wait for the higher-power data packets that match their distance.

[0143] The transmission of the state of the keys or other status information also takes place during the light flash, with a larger number of bits also shortly after the light flash, but with a high frequency and short ON times and long OFF times, which have almost no influence on the line sensors, since photo elements always integrate the light output over time.

[0144] A special photosensor in the camera, which can amplify this high frequency and filter it out from a mixture of interfering light pulses (remote controls, fluorescent lamps, etc.), "views" the entire projection surface or the entire 3D space. Since the camera knows the time at which the position light flashes are generated, the time at which data bits must be transmitted to the camera is also known to within + / - 2 microseconds.

[0145] At these times, the amplifier's output signal is decoded, and the data is transmitted to the computer, individually for each laser pointer or LED transmitter. By using the same transmission method (modulated light flash) for the position and button status, we also achieve 100% correlation between positions and button presses, which would only be possible with a mixed method (position light, button radio).

[0146] To make the system suitable for daylight, we mount special, custom-developed filters onto our sensors. We use a so-called interference bandpass filter, which allows an extremely narrow wavelength band to pass through while filtering out almost 100% of the rest of the light spectrum.

[0147] However, the wavelength band of the interference bandpass filter depends on the angle of incidence of the light. We had to modify the filter so that it allows the wavelength of our transmitters to pass through even at + / -45 degrees of incident light (0 degrees perpendicular).

[0148] An improved version uses a combination of a narrower interference bandpass filter and a lens, preferably a Fresnel lens or a nano-lens, which is simply considerably flatter and more precise and is mounted directly in front of the filter, directing the obliquely incident light almost straight.

[0149] Such a lens, preferably a Fresnel lens or a nanolens, can also be constructed to either dispense with the slit aperture entirely or to "collect" significantly greater light energies before directing the light onto the slit aperture and the line sensor behind it. In this case, the image is simply reduced in size, as in a light-sensitive telescope, where the front lens should have the largest possible diameter.

[0150] A preferred version is a so-called Linear Fresnel lens: [ Fig. 12]

[0151] This collects more light in one direction, but does not change the angle of the incident light in the other direction (the direction of the sensor). The line sensor is located along the "focal point" of the Linear Fresnel lens. Detection of light points in the room

[0152] To use the system for position detection in 3D space, at least two cameras, preferably mounted on the ceiling, must detect the X and Y coordinates of an LED flash from different angles. Using simple linear equations in space, the coordinates in 3D space are calculated from the two X / Y pairs.

[0153] When using laser pointers, such a flashing LED is mounted on the laser pointer in such a way that cameras, for example, on the ceiling, can detect the laser pointer's position in the room. Combined with the X / Y coordinates on the projection surface, the laser beam's path in 3D space is determined using simple linear equations in space.

[0154] Preferably, however, more than two cameras are used to reduce the likelihood of other people or objects in the same room blocking the view. Since the cameras transmit an "off-screen" status, only valid pairs can be used, i.e., those reported with a sufficiently high amplitude.

[0155] As the transmitter for an LED flash, you don't use just one LED, but preferably several. These LEDs emit light with a smaller aperture angle, but thereby generate significantly higher light energy per steradian. These different LEDs are then preferably not all flashed at the same time, but rather assigned different time windows. Depending on which LEDs are detected, you can also detect the inclination of objects.

[0156] Because LEDs can transmit significantly higher pulse currents than laser diodes (for safety reasons), the distance between LEDs and cameras can be increased and / or the time windows shortened, so that significantly more light flashes from different sources can be detected, either more objects / people or more tracking points per object / person. QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] WO 00 / 38102

[0002] US 3,951,550 [0011, 0012, 0013, 0014]

Claims

[1] A position detection system comprises: at least one position detection device (1); and a number of light sources (20 to 2N) spaced from the position detection device (1); wherein the position detection device (1) comprises at least two line image sensors (11, 11'), wherein extension directions of sensor surfaces (12, 12') of the two line image sensors (11, 11') enclose an angle with one another; a slit diaphragm (13, 13') is provided above each line image sensor (11, 11') such that the slit diaphragm (13, 13') forms an angle of 90° with the associated sensor surface (12, 12'); the position detection device (1) and the light sources (20 to 2N) each have interfaces for wireless communication, via which the position detection device (1) and the light sources (20 to 2N) are synchronized in time to enable detection of a respective light source (20 to 2N) by a time-division multiplex method upon emission of light by the light sources (20 to 2N).

Images