Device for the precise positioning of arrow-like objects in a target using a camera and light sources

Abstract

Device for the precise position determination of arrow-like objects (10) in or on a target (9) with at least two light sources, a projection surface (6) which is illuminated by the light sources from different perspectives, a camera (5) and an evaluation unit for calculating the positions, characterized in that the camera is arranged so that it can capture the projection surface and the shadows (14) of objects created thereon by the light sources, the light sources are almost point-like and are controlled in such a way that they produce different shadow images of the objects, which are used to calculate the position by image analysis, the relative positions of the camera to the target, to the projection surface and to the individual light sources are arranged in such a way that the angles of the arrowheads to the light sources can be determined based on the position of the shadows in the camera image in order to determine the position of the arrow in the target by triangulation.

Description

background

[0001] There are numerous dart machines known as soft darts or e-darts that detect and evaluate hits on a target using pressure sensors located behind them. However, in steel darts, the original dart game with steel tips, precise hit detection is more complex, which is why, for example, AI-supported image analysis has only been available for a few years. There are already some state-of-the-art proposals for implementing precise dart position detection in steel darts. The following list contains patents or patent applications that require a prepared target and therefore do not work with conventional dartboards.

[0002] CA2279060C describes dart detection using electromagnetic induction in coils individually integrated into each segment of the dartboard. When a dart enters a segment, an induced voltage is detected, and the hit segment is determined.

[0003] US20030134700A1 describes dart detection by placing a camera behind a shoot-through target. This is neither compatible with the typical setup of a dartboard mounted on a wall, nor does a natural fiber dartboard offer the possibility of through-shoots.

[0004] The following are patents or patent applications that do not require a prepared target, but have at least one of the disadvantages listed: • General inaccuracy or susceptibility to errors in positioning • Inaccuracy in certain arrow constellations, e.g. mutual occlusion • Complex and / or cost-intensive technology • Vulnerability due to the use of a mechanism with moving parts

[0005] WO1998031979A1 describes a system for determining the position of darts using a light source and opposing light sensors positioned laterally on the dartboard. By interrupting the light with the dart, angles to the light sensors can be determined, and the point of impact of the dart can be determined by triangulation. Due to the number of light sensors required to evaluate even mutually obscuring dart configurations, production effort and costs are correspondingly high.

[0006] DE4207497A1 describes a system that uses a light barrier that continuously rotates around the dartboard to determine the point of impact. This mechanism is complex to produce, fragile, and disrupts the feel of the game.

[0007] WO2018082745A1 describes a system for determining the position of darts by determining the intersection line of at least two cameras. However, to avoid occlusion between darts, more than two cameras are required in practice. The number of cameras and the associated computing power required for evaluating the camera images increase the cost of this technology.

[0008] According to current research, all leading systems on the market utilize a system of multiple cameras that view the dartboard from different perspectives. The weaknesses of this method include high manufacturing costs, due in part to the number of cameras and the resulting computing power required for image analysis, thus requiring more expensive analysis units. The positioning of the cameras throughout the room to capture the target from appropriate angles is often perceived as distracting and space-consuming. Likewise, the reliability of position detection is not always guaranteed, which sometimes leads to incorrect calculations.

[0009] The invention is based on the object of enabling reliable and accurate position calculation of the arrows in the target with a simple and cost-effective design. This object is achieved by a device according to the features of claim 1. The proposed device and the methods implemented therewith require only one camera, several light sources, and a projection surface. The invention is explained below using an exemplary embodiment. Structure of the device

[0010] The device is used to detect hits from darts thrown or shot at a target by analyzing their shadows when illuminated from different perspectives. It is designed specifically for steel-tipped darts, but is also suitable for other games such as magnetic darts or suction-cup darts that stick to the target. As described in Fig. 1 and Fig. 2, it consists of several light sources (1, 2, 3, 4), arranged at a sufficient distance from one another to allow different perspectives, a camera (5) and a projection surface (6) which is captured by the camera's field of view. To simplify the calculation of the position of arrows, it is advisable to keep the light sources, camera and projection surface at the same radius from the center of the target (9), i.e. to use a circular holder (7) for these elements, from which they are directed towards the interior of the circle. To ensure that the target is exactly centered within the holder, both can be mounted on a fixture (8), which in turn is attached at the back to a wall, for example. Alternatively or additionally, the holder can be fixed to the edge of the target. The target itself and anyThese encircling surrounds for catching misthrows that miss the disc are independently replaceable and therefore not a direct part of the device.

[0011] It is recommended to place the camera and light sources below the target and the projection surface above it. Since both ring sections only require approximately 100° - 120° of the circumference, a portion of the ring holder on the left and right can be left out for space and aesthetic reasons, thus dividing the holder into two separate ring sections (thicker lines in Fig. 2). The camera should capture the central, scoring area of ​​the target (radius 17 cm, 20 segments) at a recommended viewing angle of approximately 60°. Therefore, a radius of approximately 38 cm is recommended for the ring mount. This also ensures sufficient safety distance and space for a surround around the dartboard. High-power LEDs can be used as light sources. These are arranged in the device in such a way that, using different lighting perspectives, dart positions in every scoring-relevant area of ​​the target can be determined with sufficient accuracy, with a maximum tolerance of 1 mm. At least two, but preferably four or six light sources are recommended, arranged equally to the left and right of the camera, if possible. An LED strip is also a practical solution despite the unnecessarily large number of light sources, as they are inexpensive and can be easily controlled individually.

[0012] The portion of an arrowhead (10) that just protrudes from the target is evaluated by the shadows (11, 12, 13, 14) cast onto the projection surface by the light sources (1, 2, 3, 4). These shadows change their perspective position depending on the light source. This allows very precise conclusions to be drawn about the positions of the arrows in the target.

[0013] The camera only looks at the target from the side, not from above. This way, it captures not only the shadows on the projection surface but also the arrowheads themselves. This allows another angle to the arrow—the angle from the camera's perspective—to be included in the position calculation.

[0014] A very cost-effective and sufficiently powerful implementation of the evaluation unit for control, image analysis, and calculation of arrow positions is an ESP32. To reduce the memory and processing power requirements, monochrome images in the 1024:100 format can be used, since the side view of the target requires only a very small image height. Furthermore, the ESP32 already has Wi-Fi and Bluetooth capabilities to send the calculated positions, i.e., throw results, directly to a cloud or app, for example. A corresponding app can optionally also record a picture or short video of the player upon receiving the throw data, for example, to check fairness or authenticity.

[0015] To ensure clear, distinct shadows from the arrowhead and avoid overlapping perspectives from different light sources, the shadows cast by the different light sources must be clearly distinguishable. One solution to this is to use a different color, or wavelength, for each light source. A shadow is then defined by the absence of a specific color component on the projection surface—namely, the component whose light source is obscured by the arrowhead at that point.

[0016] However, the solution favored here, and described in more detail below, takes a different approach: activating the light sources sequentially, so that only one is active at a time. This also allows the shadows of the arrowheads to be clearly assigned to a specific perspective. To avoid unwanted lighting effects, infrared LEDs and a camera with an appropriate IR filter can be used as light sources.

[0017] The less external light influence can interfere with the detection of shadows, the clearer the image analysis results will be. Therefore, it is recommended to shield the side of the approximately 1-2 cm high projection surface facing the room with a protective edge, a 2-4 cm wide cover acting as a screen. This cover keeps both external light and the player's gaze or arrows away from the projection surface. A color contrast on the cover also makes it easier to determine the height of the covered projection surface during a calibration process. Working methodCalibration

[0018] Several calibration steps are recommended to increase the accuracy of the measurement results. After assembling the target, the exact position of the individual elements and segments on the board can be determined by taking a top-down photo of the entire device using an external camera from an associated app. For this purpose, recognizable landmarks are provided at various points on the device. The device's camera can also be used to analyze the colors of certain dart segments, thus detecting any slight rotation of the dartboard and later taking this into account in the throwing results. The course of the projection surface in the image from the opposite camera can be easily determined by color differences.A protective border (cover) above the projection surface not only serves as a screen to shield against unwanted light, but also to limit the bright projection surface through color contrasts, i.e. darker colors. If the projection surface is around 1-2 cm high, its upper boundary is identified by the differently colored protective border. Even if the camera's view is as flat as possible over the target, the projection surface is naturally captured by the camera with a slight curvature. Identifying this and examining only the relevant pixels is also the task of calibration. To check the relative alignment of the camera to the projection surface, the surface itself can contain a calibration pattern, e.g. a ruler. This can also be used to compensate for camera-induced image distortions in software with the help of a LUT, and thus to derive correct camera angles from all X-positions of the image.

[0019] All of the calibration functions mentioned above are usually only necessary once. Saving the projection surface without shadows as a comparison image for later image analysis can be done before each game start. Monochrome images can be used for image analysis. While the camera's image format requires a good resolution in the width (X), 100 pixels in the height (Y) are sufficient, which leads to resource savings. Position calculation

[0020] The position calculation of a new arrow is started after it lands on the target, provided the system is in the corresponding wait state during a game. This event can be triggered by one of the following methods, among others: 1. A piezo sensor on the target triggers the event via structure-borne sound 2. The event is triggered via a microphone near the target 3. Through constant image comparisons of the camera, the event is triggered when there is a significant pixel change

[0021] The first light source is switched on and the camera image now taken is compared with a comparison image of a previous version without the arrow. If a new shadow appears on the projection surface, it should be easy to detect due to changes in pixels, and its angle to the camera can be determined using the X-position. Any shadows cast by arrows already in the target are not taken into account again. This means that new positions can be determined even when arrows are already in the target. Because the arrangement of the objects in the device means that only the arrowheads create shadows, these shadows are very thin, only a few pixels wide, with the horizontal center being crucial for maximum accuracy.

[0022] The Y-axis of the shadow can be used to check whether the arrowhead is perpendicular to the target, allowing for any angled arrows to be taken into account. However, in general, the further down the Y-axis the shadow is captured in its X-axis position on the projection surface, the more accurate the result will be, as this corresponds to the position of the arrowhead closest to the target.

[0023] The same image analysis process can be repeated for all other light sources if the result is unclear, with only one light source active at a time. In addition to the shadows, the arrow itself is also captured by the camera, clearly distinguishable by the fact that its position relative to the camera remains unchanged when the light sources change. Since each light source only needs to be activated briefly for the camera image, the entire process can be completed in significantly less than a second.

[0024] Geometric methods can now be applied to calculate the position of the arrows. Here, a method is described that requires a circular arrangement of the camera, light sources, and projection surface, as well as known distances between the light sources and the camera. It utilizes geometric properties of the circle (isosceles triangles, angle sum). First, the viewing angles of the shadows of the camera and light sources are calculated in relation to their center (center of the target), forming isosceles triangles ( Fig. 3).

[0025] When calculating the camera's viewing angle of an object (arrow or its shadow), it must be taken into account that the camera is not located in the center of the circle, but at the edge. If the camera in the device looks vertically from below at the target, the center of the target is exactly in the middle of the horizontal axis of the camera image (X direction). For a given 60° viewing angle, this results in deviations from this center of -30° to +30°, within which objects can be detected. With a perspective, non-linear distribution, the angle between two pixels is larger in the center than at the edge, but with a resolution of, for example, 1024 pixels across the width, it is generally accurate enough. Calculations for specified dimensions and viewing angles: in the center (0°) ≈0.0586° per pixel, at the edge (-30°, +30°) ≈0.0732° per pixel.

[0026] Because the relative positions of the elements in the device within the circular arrangement are known, the angles of the shadows to the camera can now be converted into the perspective of the light sources that caused them. From the perspective of this light source, the angle of a luminous light source to the arrow is identical to the angle to the projected shadow. The camera and light source, including the arrowhead, form a triangle with a known line segment (camera - light source) and two known adjacent angles, from which the common intersection point and thus the position of the arrow can be determined.

[0027] Ideally, this can be achieved with just one light source. In practice, however, it can happen that arrowheads obscure others, i.e., that they are in the same line, or that angles are unfavorable for precise calculations, or that the shadows lie outside the projection surface. Therefore, the use of multiple light sources is necessary. Calculations can be used for all light sources related to the camera and also for all light sources among themselves. In case of discrepancies in the results, the average or the initial values ​​with the best expected result should be used.

[0028] Since the device also knows the distance and angle of all elements (camera and light sources) to the center of the target, the position information can be converted relative to the center of the board in the next step. For a dartboard, the hit segment is then determined from this data. This result can be used directly or sent to a system for controlling the game. Example (step-by-step solution for calculating a new litter)

[0029] A dart lands on the dartboard and is registered by a piezo element, e.g., due to the vibration caused by structure-borne sound, triggering a new position calculation. Images are captured consecutively with exactly one activated light source at a time and then analyzed. In each image, significant changes in individual pixels can occur in two areas of the projection surface: the area of ​​the dart and the area of ​​its shadow. The area of ​​the dart itself is identical for all light sources. The shadow cast, however, depends on the currently active light source.

[0030] In the following, the calculation for a specific light source (4) is considered ( Fig. 3). For a clearer explanation of the calculation steps, let the camera (5) be point A, the light source (4) point B, the shadow (14) point C, the dart (10) point D, and M be the center of the circle. A, B, and C lie on the same circle, while D lies within it. The angle of the light source to the camera, or rather the shift of the view to the center, is given and always the same; in the example, ∠AMB = 48°, represented in the drawing by dashed lines. In the example, the arrow is captured by the camera slightly to the left of its center of view, and an angle of ∠MAD = 2.41° is determined for this. The shadow of the arrowhead is captured even further to the left and determined as an angle of ∠MAC = 18.5°. The latter must first be converted from the camera perspective to the perspective of the light source, since only from this perspective are the angles to the shadow and to the arrow itself identical.Intermediate steps are necessary here, which can be carried out using the interior angle theorem for isosceles triangles and the knowledge that A, B, and C lie on a circle. Central angle LAMC = 180° - 2 x 18.5° = 143°, LBMC = 360° - 48° - 143° = 169°, LMBC = (180° - 169°) / 2 = 5.5°, which corresponds to the angle of view of the light source on the shadow and also on the arrow. Step 1: Calculate the coordinates of B (always constant in the device) 1. MA points along the negative y-axis (270°) 2. ∠AMB = 48° → MB is 48° counterclockwise from MA 3. Angle from MB to x-axis: 270° + 48° = 318° 4. Coordinates of B: ◯ B_x = cos(318°) = cos(42°) ≈ 0.7431 (X coordinate of B) ◯ B_y = sin(318°) = -sin(42°) ≈ -0.6691 (Y coordinate of B) ◯ → B ≈ (0.7431, -0.6691)Since B describes the light source (4), whose relative position to the camera (5) is always the same in the device, this step can of course be skipped in a device with known dimensions. Step 2: Determine straight line AD 1. MA: from M(0,0) to A(0,-1) → vertical line 2. ∠MAD = 2.41° → AD deviates from MA by 2.41° (clockwise) 3. Slope of AD: tan(270° + 2.41°) = tan(272.41°) ≈ -24.028 4. Equation of a line AD (through A(0,-1)): y+1=−24.028(x−0)→y=−24.028x−1 Step 3: Determine straight line BD 1. MB: from M(0,0) to B(0.7431,-0.6691) Slope MB = -0.6691 / 0.7431 ≈ -0.900 2. ∠MBD = 5.5° → BD deviates by 5.5° from MB (clockwise) 3. New angle: arctan(-0.900) - 5.5° ≈ -42.0° - 5.5° = -47.5° 4. Slope BD: tan(-47.5°) ≈ -1.091 5. Equation of the straight line BD (through B): y+0.6691=−1.091(x−0.7431)→y≈−1.091x+0.142 Step 4: Calculate intersection point D 1. Equating the lines: -24.028x - 1 = -1.091x + 0.142 2. Dissolve: −22.937x=1.142x≈−0.0498 3. Calculate y-value: y≈−24.028*(−0.0498)−1≈1.196−1=0.196 4. → D ≈ (-0.050, 0.196) Step 5: Converting the coordinate into a segment of the dartboard

[0031] The angle and distance from the center point must now be calculated from the coordinates. This can be done using the arctangent function and the Pythagorean theorem. In the example for dart D, this results in an angle of 104.3°. Rotated by 90° for the view from above = 14.3° and a center point distance of 0.202 with a radius of 1. The dartboard is divided into 20 segments of 18° each, with the 20, the top segment, spanning from -9° to +9°. The hit is therefore in the adjacent segment to the left of it, which on a dartboard represents the number 5. The distance to the center point is also used to determine whether a double or triple field of the corresponding segment was hit, which is not the case here. If the hit is close to the middle of two segments, meaning the result is not clear, additional light sources can be evaluated.Even if no shadow can be measured for the light source because it doesn't fall on the projection surface or overlaps with other object shadows, additional light sources are required for evaluation. In this specific example, the calculation with just one light source is sufficient, and the throw result can be output or sent to the game controller. Drawings • Fig. 1: Spatial arrangement of the individual objects in the device • Fig. 2: Relationships between objects in position calculations • Fig. 3: Illustration of an example of a position calculation List of reference symbols 1 (1-4) light sources 5 Camera 6 Projection screen 7 Mounting ring for light sources, camera and projection screen 8 Mounting for mounting ring and target 9 Target, e.g. dartboard 10 Arrowhead stuck in the target, e.g. from darts 11 (11-14) Shadow of an arrowhead on the projection surface QUOTES CONTAINED IN THE DESCRIPTION

[0000] This list of documents submitted by the applicant was generated automatically and is included solely for the convenience of the reader. This list is not part of the German patent or utility model application. The DPMA assumes no liability for any errors or omissions. Cited patent literature

[0000] CA 2279060C

[0002] US 20030134700A1

[0003] WO 1998031979A1

[0005] DE 4207497A1

[0006] WO 2018082745A1

[0007]

Claims

[1] Device for the precise determination of the position of arrow-like objects (10) in or on a target (9) with at least two light sources, a projection surface (6) which is illuminated by the light sources from different perspectives, a camera (5) and an evaluation unit for calculating the positions, characterized by , that the camera is arranged so that it can capture the projection surface and the shadows (14) of objects created thereon by the light sources, the light sources are almost point-like and are controlled in such a way that they produce different shadow images of the objects, which are used to calculate the position by image analysis, the relative positions of the camera to the target, to the projection surface and to the individual light sources are arranged in such a way that the angles of the arrowheads to the light sources can be determined based on the position of the shadows in the camera image in order to determine the position of the arrow in the target by triangulation. [2] Device according to claim 1, characterized by that the device comprises at least four light sources (1, 2, 3, 4) for illumination from different perspectives. [3] Device according to claim 1 or 2, characterized by that the light sources can be individually switched on and off via an electronic circuit in order to create the different shadow images and prevent overlapping of different perspectives. [4] Device according to claim 1 or 2 characterized by that the light sources operate in different wavelength ranges to produce the different shadow images. [5] Device according to one of claims 1 to 4, characterized by that the camera is positioned to capture the arrowheads, thereby providing a complementary perspective for positioning in addition to analyzing the shadows. [6] Device according to one of claims 1 to 5, characterized by that the light sources include light-emitting diodes. [7] Device according to one of claims 1 to 6, characterized by that the light sources emit infrared light in order to make the illumination necessary for position analysis invisible to the human eye. [8] Device according to one of claims 1 to 7, characterized by that the projection surface is provided with a one-sided cover as a shutter, which shields external light from the room and at the same time stands out in color from the projection surface in order to be able to better identify its height during image analysis. [9] Device according to one of claims 1 to 8, characterized by that lines or color contrast patterns are introduced into the projection surface which can be detected by the camera to calibrate its exact position relative to the projection surface and to detect and compensate for image distortions.

Images