An optical image positioning device and method for a stylus based on retroreflection

Abstract

This invention discloses a stylus optical image positioning device and method based on retroreflection, comprising: proposing a neural network model for stylus optical image positioning based on retroreflection; designing a stylus structure that meets positioning requirements; setting retroreflective material calibration points according to the actual screen size, and calculating the inverse perspective transformation matrix for calculating the transformation coordinates; performing preprocessing such as grayscale conversion and Gaussian filtering on the acquired image; performing subsequent operations such as threshold segmentation, convex hull shaping, and centroid extraction on the preprocessed image to obtain the image coordinates of the stylus tip; inputting the stylus tip image coordinates and the transformation coordinates into the positioning neural network, and outputting the precise position of the stylus through the positioning neural network; this invention, based on retroreflection technology, can realize the passive positioning function of the writing pen, omitting the implantation of electronic circuits, and achieving multi-point positioning solely through optics and mechanics.

Description

Technical Field

[0001] This invention relates to the field of image positioning technology, and specifically to an optical image positioning device and method based on retroreflection stylus. Background Technology

[0002] Stylus positioning is a technology that converts the actual position of a stylus into a digital signal to enable writing, drawing, or interaction on an electronic screen. With the development of digital technology, stylus positioning technology has been widely used in education, design, and meeting recording. Existing stylus positioning technologies mainly employ various methods such as electromagnetic induction, ultrasonic positioning, and optical positioning. Electromagnetic induction technology uses the interaction between the stylus's built-in electromagnetic induction element and the electromagnetic induction plate on the screen to achieve stylus positioning; ultrasonic positioning technology determines the stylus's position by emitting ultrasonic waves and receiving reflected signals; optical positioning technology uses a camera to capture the stylus's movement trajectory on the screen and achieves positioning through image processing technology. However, both optical and ultrasonic positioning technologies can be affected by environmental factors such as light and sound, leading to decreased positioning accuracy or failure, and slow positioning time, thus affecting the user's writing experience. High-resolution cameras can provide high-precision optical positioning, but the processing time is long; in contrast, infrared positioning is faster, but the accuracy may be relatively lower.

[0003] To address the shortcomings of existing technologies, this patent proposes a method and device for optical image positioning using a retroreflective stylus. The method includes: proposing a retroreflective stylus optical image positioning model; designing a stylus structure that meets positioning requirements; setting calibration points for retroreflective materials based on the actual screen size and calculating the inverse perspective transformation matrix; constructing a received power model; performing preprocessing on the acquired image, such as grayscale conversion and Gaussian filtering; performing subsequent operations on the preprocessed image, such as threshold segmentation, convex hull shaping, and centroid extraction, to obtain the image coordinates of the stylus positioning area; performing inverse perspective transformation on the image coordinates to obtain transformed coordinates; and then using a neural network to fit the coordinates to obtain the precise position of the stylus on the screen. This invention designs a regular polygonal multi-segment truncated pyramid structure for the stylus tip based on the retroreflective properties of the retroreflective material. The stylus adopts a passive design, eliminating the need for embedded electronic circuitry. It proposes an optical positioning system layout and algorithm that uses a camera to acquire bright retroreflective images of the stylus tip, reducing the equipment complexity and positioning time of the positioning system, achieving multi-point positioning solely through optics and mechanics. Summary of the Invention

[0004] To address the issues of optical positioning technology being susceptible to positioning errors due to light exposure, requiring numerous devices, and involving long calculation times, this patent proposes a retroreflective stylus-based optical image positioning method. This method minimizes the impact of ambient light, achieving the benefits of simplified equipment, accurate positioning, and rapid calculation.

[0005] The technical solution of the present invention is as follows:

[0006] An optical image positioning device based on retroreflection stylus includes a stylus and a screen. The stylus is passive, and its tip is covered with retroreflective material. This material reflects light to form a bright spot on the image for positioning. The stylus tip employs a multi-segment truncated pyramid structure to ensure high-brightness retroreflection at any angle (high-brightness retroreflection refers to an average reflectivity of greater than or equal to 70% observed from various angles, ensuring that reflected light forms a bright spot on the image). A camera is positioned diagonally above the screen to capture the position information of the stylus tip. The position information is input into a positioning neural network model, which outputs the precise position of the stylus.

[0007] Furthermore, the cross-section of the pen tip is selected as a regular polygon or a circle to ensure that the pen tip has high brightness retroreflection in all directions in the lateral direction.

[0008] Furthermore, the pen tip is multi-segmented in the longitudinal direction to ensure high brightness retroreflection in all directions in the longitudinal direction.

[0009] Furthermore, the pen tip is preferably a regular decagonal, four-segment pen tip.

[0010] Furthermore, a press-type telescopic structure is provided above the pen tip. When pressed, the pen tip will extend for positioning, and when not pressed, the pen tip will retract and not be positioned, thus protecting the pen tip.

[0011] Furthermore, retroreflective material is placed at the four boundary points of the screen to locate the screen area and to perform inverse perspective transformation matrix calculations.

[0012] Furthermore, the camera is an infrared camera with infrared LEDs, pointing in front of the screen, and the camera is set at a predetermined distance from the edge of the screen to ensure that the entire screen can be captured by the camera.

[0013] A positioning method for a retroreflective stylus optical image positioning device includes the following steps:

[0014] Step 1) Image Input:

[0015] Use a camera to capture screen images with bright spots from the pen tip;

[0016] Step 2) Preprocessing:

[0017] The camera is calibrated to obtain its intrinsic and extrinsic parameters. The acquired images are then corrected using the calibrated intrinsic and extrinsic parameters. The corrected grayscale images containing RGB color information are then converted into grayscale images containing only brightness information. Image processing methods such as Gaussian filtering, morphological erosion, thresholding, mask extraction, and convex hull shaping are used for preprocessing.

[0018] Step 3) Extract the centroid to obtain the image coordinates, and perform inverse perspective transformation to obtain the transformed coordinates:

[0019] By extracting the centroid of the shaped bright spot outline, the centroid image coordinates of the pen tip can be obtained. The centroid image coordinates of the bright spots at the four vertices of the screen are obtained after binarization. Then, the actual size of the screen to be mapped is specified, and the inverse perspective transformation matrix is ​​obtained using the four-point perspective transformation method. The pen tip centroid image coordinates are then transformed into an inverse perspective transformation using the perspective transformation matrix obtained from screen calibration, converting the tilted screen image into a front view of the screen, and the transformed coordinates are obtained.

[0020] Step 4) Train the localization neural network:

[0021] Using the pen tip image coordinates and the transformed coordinates after inverse perspective transformation as input data, and the actual coordinates on the screen as output data, both are used as training data. By training with a large amount of data from different locations, the optimal localization neural network model is obtained.

[0022] Step 5) Locate the fitted coordinates from the neural network output, and map the coordinates to obtain the pixel coordinates:

[0023] The pen tip image coordinates and transformed coordinates obtained in real time are input into the trained optimal positioning neural network model for fitting, and the output is the precise fitted coordinates of the stylus; the fitted coordinates are then mapped according to the displayed pixel size to obtain the displayed pixel coordinates.

[0024] Furthermore, the positioning neural network model includes one input layer, three hidden layers, and one output layer; the features during model training are the pen tip image coordinates and transformed coordinates, and the labels are the pen tip true coordinates; the Euclidean distance between the output coordinates and the true coordinates is used as the evaluation function, and the model with the smallest Euclidean distance is saved as the optimal model.

[0025] The design principle of this invention is as follows: Infrared LEDs around an infrared camera actively illuminate the positioning screen area. The light is highly reflected by the retroreflective material covering the stylus tip. The infrared camera captures the screen image with the bright spot of the stylus tip. By performing image processing and neural network fitting, the actual position of the stylus can be obtained quickly and accurately.

[0026] Design concept of this invention:

[0027] 1. A stylus optical image localization model based on retroreflection is proposed.

[0028] A camera with an LED light source is positioned diagonally above the positioning screen to capture images of the screen area, including the retroreflective pen tip. The retroreflective pen tip reflects bright light onto the image, forming a bright spot area. The image coordinates of the pen tip and the transformed coordinates are then input into a positioning neural network, which outputs the precise position of the stylus.

[0029] 2. Design a stylus structure that meets positioning requirements.

[0030] The stylus features a passive design, with an anti-reflective tip covered in anti-reflective material. It employs a multi-segmented frustum structure, preferably with a regular polygonal cross-section, ensuring high-brightness anti-reflection at any angle. Furthermore, the tip structure has a telescopic function to switch the anti-reflection on and off.

[0031] 3. Set the retroreflective material calibration points according to the actual screen size, and calculate the inverse perspective transformation matrix to calculate the transformation coordinates.

[0032] 4. Perform preprocessing on the acquired images, including image correction, grayscale conversion, Gaussian filtering, and morphological erosion.

[0033] 5. Perform subsequent operations such as threshold segmentation, mask extraction, highlight shaping, and centroid extraction on the preprocessed image to obtain the image coordinates of the stylus.

[0034] 6. Image coordinates are transformed to real-world coordinates, i.e., inverse perspective transformation is performed to obtain the transformed coordinates of the stylus on the actual screen. A localization neural network model is trained, the optimal trained model is used for localization and outputs accurate fitted coordinates, and then the fitted coordinates are transformed to obtain the screen display coordinates.

[0035] Preferably, the stylus tip is covered with retroreflective material, and the tip adopts a four-segment truncated pyramid design with a regular decagonal cross-section and a press-type telescopic structure on top.

[0036] Preferably, the precise position of the stylus is obtained quickly and accurately using optical image processing methods such as threshold segmentation and inverse perspective transformation, as well as neural network fitting methods.

[0037] This invention is particularly suitable for scenarios such as interactive whiteboards, interactive sand tables, and virtual meetings, enabling precise pen positioning. In the education field, this technology can be applied to real-time writing and drawing interactions in interactive whiteboards and smart classrooms, allowing for the real-time display and recording of handwritten content. In the design industry, this technology facilitates digital sketch creation and review, improving designers' work efficiency. In virtual meeting systems, this invention can be used as a precise input and control device, enhancing the remote collaboration experience. This technology can also be extended to smart office equipment, augmented reality (AR) systems, and virtual reality (VR) interactions, providing precise positioning solutions for various scenarios.

[0038] In summary, the present invention has the following beneficial effects:

[0039] 1) Retroreflective materials are applied to the stylus, and a multi-segment truncated pyramid structure is designed for the pen tip. This ensures that the stylus tip can reflect light in all directions, both horizontally and vertically, for positioning. A press-type telescopic structure is used to switch the retroreflection on and off and protect the pen tip. Based on retroreflective technology, the writing pen can achieve passive positioning function, eliminating the need for electronic circuitry and relying solely on optics and mechanics to achieve multi-point positioning.

[0040] 2) The camera captures a screen image with a bright spot from the pen tip. The image coordinates and transformed coordinates of the pen tip are input into the positioning neural network. The positioning neural network outputs the precise position of the stylus, achieving multi-point positioning solely through optics and mechanics. The average positioning error of the positioning system is 2.24mm. If edge computing processing at hundreds of megahertz is used, the average positioning time is as low as a few milliseconds (3ms). Attached Figure Description

[0041] Figure 1 This is a diagram of an optical image positioning model based on retroreflection stylus in an implementation scheme; 1 is a passive stylus, 2-1 is the positioning screen area, 2-2 is the retroreflective material calibration point, 3-1 is the infrared camera module, 3-2 is the infrared fill light module, and 3-3 is the edge computing processing.

[0042] Figure 2 This is a flowchart illustrating a stylus optical image localization method based on retroreflection in an implementation scheme.

[0043] Figure 3 The schematic diagram of the passive stylus in the implementation method is shown, with the pen tip structure 1-1, which has a four-section truncated design and a cross-sectional shape of a regular decagon, the press-type telescopic structure 1-2, and the pen body 1-3 specifically marked.

[0044] Figure 4 This is a polar coordinate distribution diagram of the longitudinal incident angle and reflectivity of the retroreflective pen tip in the implementation method.

[0045] Figure 5 This is a polar coordinate distribution diagram of the lateral incident angle and reflectivity of the retroreflective pen tip in the implementation method.

[0046] Figure 6 This is a diagram of the localization neural network model in the implementation method;

[0047] Figure 7 This is a distribution diagram of the true coordinates and fitted coordinates of the positioning point during the positioning error analysis in the implementation method;

[0048] Figure 8 This is a graph showing the cumulative distribution function of the average positioning error during the positioning error analysis in the implementation method. Detailed Implementation

[0049] The method of the present invention will be further described in detail below with reference to the accompanying drawings and through simple examples.

[0050] An optical image positioning device for a stylus based on retroreflection:

[0051] Retroreflective pen tip structure design:

[0052] Retroreflection, also known as backlighting, is a type of reflection in which the reflected light ray returns in the opposite direction to the incident light ray. This property is maintained even when the direction of the incident light ray changes over a large range. However, at large angles of incidence, very little light is reflected back in the opposite direction.

[0053] Figure 3 This is a schematic diagram of a passive stylus module according to an embodiment of the present invention. The pen tip is covered with retroreflective material, and the rest is a black surface. A press-type telescopic structure is provided above the pen tip. The pressing area is connected to the pen tip by a spring (other press-type telescopic structures can also be used; this is just one embodiment of the present invention). When pressed, the pen tip extends for positioning; when not pressed, the pen tip retracts and does not perform positioning, thus protecting the pen tip.

[0054] The pen tip uses a four-segment truncated pyramid structure with a regular decagonal cross-section. The truncated pyramids are spaced at a certain angle, and each surface is covered with retroreflective material. The angle is set based on the maximum incident angle of the retroreflective material, which is experimentally determined to maximize retroreflection. In this embodiment, the measured angle is 40°, so a preferred angle is 144°. This ensures that the pen tip can still be highly retroreflected by the retroreflective material on one segment of the structure even at any longitudinal tilt angle. The decagonal cross-section also ensures that the pen tip can still be highly retroreflected by the retroreflective material on one segment of the structure even at any lateral rotation angle.

[0055] like Figure 4 and Figure 5As shown, polar coordinate distribution diagrams of longitudinal and transverse incident angles and reflectivity are drawn based on the properties of retroreflective materials for pen tip design. The preferred regular decagonal four-segment pen tip has an average reflectivity of about 70% in all directions of the longitudinal 180° and an average reflectivity of about 79% in all directions of the transverse 360°.

[0056] A retroreflective stylus optical image localization method:

[0057] S100: Image Input

[0058] Figure 2 This is a side view of an embodiment of the present invention. An infrared camera with infrared LEDs is mounted diagonally above and around the periphery of a screen positioning area measuring 810mm × 550mm. The camera points towards the front of the screen and is positioned at a certain distance from the screen edge to ensure that the entire screen can be captured by the camera. Appropriately sized retroreflective material is placed at the four boundary points of the positioning area to locate the screen area and perform inverse perspective transformation matrix calculations. The infrared camera used has 1 million pixels and an image size of 1280 × 720. Depending on the screen size, cameras with several million pixels or higher resolution are generally used to ensure image quality. The infrared LEDs provide active illumination. The light is highly retroreflected by the retroreflective material on the tip of the passive stylus and enters the camera for capture. A bright spot area that is clearly separated from the dark background will appear on the image.

[0059] The solar spectrum is an absorption spectrum of different wavelengths, divided into visible and invisible light. The intensity of visible light in sunlight decreases with increasing wavelength, while the near-infrared band of invisible light exhibits a large trough at 940nm, i.e., a local minimum. Acquiring images in the 940nm infrared band minimizes interference from sunlight or ambient light; therefore, this embodiment employs a 940nm infrared narrowband camera.

[0060] S110: Preprocessing including grayscale conversion, Gaussian filtering, and morphological erosion.

[0061] The camera is calibrated to obtain its intrinsic and extrinsic parameters. The acquired image is then corrected using the calibrated intrinsic and extrinsic parameters. The corrected grayscale image, containing RGB color information, is then converted to a grayscale image containing only luminance information. If a distortion-free camera is used to acquire the image, image correction has minimal impact on the positioning results; therefore, this embodiment does not perform correction to reduce computation time.

[0062] Applying Gaussian filtering to a grayscale image can smooth the image, remove noise, and thus make the image clearer and smoother, while preserving edge information relatively well. The Gaussian filter uses a 3×3 matrix with a standard deviation of 2, and its representation is as follows:

[0063]

[0064] S120: Threshold segmentation, mask extraction, convex hull shaping algorithm

[0065] The obtained image is segmented using a threshold. The threshold is selected based on the lowest gray value of the actual bright spot. Generally, the average gray value of the spot area at the retroreflective material calibration point farthest from the camera can be selected as the lowest gray value of the bright spot. In this embodiment, a gray value of 220 is selected as the segmentation threshold. Background noise is removed by mask extraction, and the bright spot area of ​​the pen tip is shaped using a convex hull shaping algorithm to describe the contour of the spot.

[0066] S130: Centroid extraction yields image coordinates, inverse perspective transformation yields transformed coordinates.

[0067] By extracting the centroid of the reshaped highlight outline, the centroid image coordinates of the pen tip can be obtained. The centroid extraction formula is:

[0068]

[0069] Commercially available display or projection screens are all rectangular in shape. Therefore, appropriately sized retroreflective material is placed at the four vertices of the rectangular screen. The centroid coordinates of the bright spots at the four vertices of the screen are obtained after binarization from the image. Then, the actual size of the screen to be mapped is defined, and the inverse perspective transformation matrix is ​​obtained using the four-point perspective transformation method. The four-point perspective transformation is as follows:

[0070]

[0071] In the above formula (x s y s (x) represents the centroid image coordinates of the boundary point. d y d The coordinates are the actual screen dimensions. Inverse perspective transformation is a homogeneous coordinate transformation that normalizes the parameters of the matrix. The 9th element M in the matrix... 33 It can be fixed at 1. Therefore, there are 8 independent unknowns in the perspective transformation matrix. To determine these 8 unknowns, we need at least 4 pairs of source points and target points. The source points are the centroid image coordinates of the four vertices, and the target points are the real coordinates corresponding to the actual screen size. In this embodiment, the screen size is 810mm × 550mm, and the four target points are selected as [0,0], [810,0], [0,550], [810,550]. Each pair of source points and target points can provide two independent equations, so the 4 pairs of points can provide a total of 8 equations, which meets the requirement of solving these 8 unknowns.

[0072] The pen tip centroid image coordinates are transformed using the perspective transformation matrix obtained from screen calibration to perform an inverse perspective transformation, converting the tilted screen image into a front view of the screen, thus obtaining the transformed coordinates. The inverse perspective transformation formula is:

[0073]

[0074] S140: The localization neural network outputs fitted coordinates, and the coordinates are mapped to obtain pixel coordinates.

[0075] Localization neural network models such as Figure 6 As shown, (the positioning neural network in this embodiment uses a fully connected neural network based on the BP algorithm). The input data consists of four elements: the pen tip image coordinates and the transformed coordinates after inverse perspective transformation. The output data consists of two elements: the actual screen coordinates. Both are used as training data, and the model is trained using a large amount of data from different locations to achieve good generalization ability. The model consists of five neural layers: an input layer, three hidden layers, and an output layer. The activation function used is tanh, and the expression for tanh is:

[0076]

[0077] During model training, the Adam optimizer was used with a batch size of 16 and an initial learning rate of 0.001. Gradient descent was employed, reducing the learning rate to 10% of its original value every 500 training iterations. After approximately 1100 training iterations, the training set loss and test set loss were 0.00016384 and 0.00016908, respectively. In the prediction phase, the Euclidean distance between the output data and the real data was used as the evaluation function. The smaller the Euclidean distance, the better the model training. The model with the smallest Euclidean distance was saved as the optimal model.

[0078] The pen tip image coordinates and transformed coordinates obtained in real time are input into the trained optimal positioning neural network model for fitting, and the output is the stylus's accurate fitted coordinates (x... f y f ).

[0079] If the pixel size used for display is a×b, then the fitted coordinates need to be converted to pixel coordinates. Assuming the actual screen size is c×d, the conversion formula is:

[0080]

[0081]

[0082] For x in the above formula p and y p Rounding up yields the final display pixel coordinates (x). p yp ).

[0083] Error Analysis: For a screen measuring 810mm × 550mm, stylus positioning was performed. Starting from 30mm, positioning points were set at 60mm intervals, for a total of 14 × 9 positioning points. The error between the actual coordinates of the reference points obtained using the above positioning method and the fitted screen coordinates is shown in the following figure. Figure 7 As shown. The positioning error is evaluated as the Euclidean distance between the actual coordinates and the fitted screen coordinates. The average positioning error for 14×9 positioning points is 2.24 mm. From Figure 8 The cumulative distribution function of the average positioning error shows that the positioning error of more than 75% of the positioning points can reach within 2mm.

[0084] In this embodiment, using a 150MHz edge computing processor, the average positioning time is a few milliseconds (3ms). This invention effectively satisfies the requirements of accuracy and real-time performance for optical positioning with a stylus.

[0085] This patented solution is not only applicable to infrared optical systems, but can also be extended to other optical bands to achieve wider device compatibility. Furthermore, this stylus technology is suitable for various interactive devices, including but not limited to interactive whiteboards, interactive sand tables, virtual meetings, augmented reality (AR) systems, and virtual reality (VR) interactions, opening up newer and broader application scenarios for interactive technologies.

Claims

1. A positioning method for a stylus optical image positioning device based on retroreflection, characterized in that, The device includes a stylus, a screen, and a processor. The stylus is passive, and its tip is covered with retroreflective material. This material reflects light to form a bright spot on the image for positioning. The stylus tip uses a multi-segment truncated pyramid structure to ensure high-brightness retroreflection at any angle. High-brightness retroreflection means that the average reflectivity of the retroreflective tip observed from all directions is greater than or equal to 70%, ensuring that the reflected light forms a bright spot on the image. A camera is positioned diagonally above the screen to capture image data from the stylus tip. The processor processes the image data to obtain the stylus tip's position information, inputs this position information into a positioning neural network model, and outputs the precise position of the stylus. The method includes the following steps: Step 1) Image Input: Use a camera to capture screen images with bright spots from the pen tip; Step 2) Preprocessing: The camera is calibrated to obtain its intrinsic and extrinsic parameters. The acquired images are then corrected using the calibrated intrinsic and extrinsic parameters. The corrected grayscale images containing RGB color information are then converted into grayscale images containing only brightness information. Image processing methods such as Gaussian filtering, morphological erosion, thresholding, mask extraction, and convex hull shaping are used for preprocessing. Step 3) Extract the centroid to obtain the image coordinates, and perform inverse perspective transformation to obtain the transformed coordinates: By extracting the centroid of the shaped bright spot outline, the centroid image coordinates of the pen tip can be obtained. The centroid image coordinates of the bright spots at the four vertices of the screen are obtained after binarization. Then, the actual size of the screen to be mapped is specified, and the inverse perspective transformation matrix is ​​obtained using the four-point perspective transformation method. The pen tip centroid image coordinates are then transformed into an inverse perspective transformation using the perspective transformation matrix obtained from screen calibration, converting the tilted screen image into a front view of the screen, and the transformed coordinates are obtained. Step 4) Train the localization neural network: Using the pen tip image coordinates and the transformed coordinates after inverse perspective transformation as input data, and the actual coordinates on the screen as output data, both are used as training data. By training with a large amount of data from different locations, the optimal localization neural network model is obtained. Step 5) Locate the fitted coordinates from the neural network output, and map the coordinates to obtain the pixel coordinates: The pen tip image coordinates and transformed coordinates obtained in real time are input into the trained optimal positioning neural network model for fitting, and the output is the precise fitted coordinates of the stylus; the fitted coordinates are then mapped according to the displayed pixel size to obtain the displayed pixel coordinates.

2. The positioning method of the optical image positioning device based on retroreflection stylus according to claim 1, characterized in that, The pen tip cross-section is selected as a regular polygon or a circle to ensure that the pen tip has high brightness retroreflection in all directions in the horizontal direction.

3. The positioning method of the optical image positioning device based on retroreflection stylus according to claim 2, characterized in that, The pen tip is multi-segmented in the longitudinal direction to ensure high brightness retroreflection in all directions.

4. The positioning method of the optical image positioning device based on retroreflection stylus according to claim 3, characterized in that, The pen tip is preferably a regular decagonal, four-segment pen tip.

5. The positioning method of the optical image positioning device based on retroreflection stylus according to claim 1, characterized in that, The pen tip is equipped with a press-type telescopic structure. When pressed, the pen tip will extend and be positioned. When not pressed, the pen tip will retract and not be positioned, thus protecting the pen tip.

6. The positioning method of the optical image positioning device based on retroreflection stylus according to claim 1, characterized in that, Retroreflective material is placed at the four boundary points of the screen to locate the screen area and to perform inverse perspective transformation matrix calculations.

7. The positioning method of the optical image positioning device based on retroreflection stylus according to claim 1, characterized in that, The camera is an infrared camera with infrared LEDs, pointing in front of the screen. The camera is set at a predetermined distance from the edge of the screen to ensure that the entire screen can be captured by the camera.

8. The positioning method of the optical image positioning device based on retroreflection stylus according to claim 1, characterized in that, The localization neural network model includes one input layer, three hidden layers, and one output layer. The features used during model training are the pen tip image coordinates and transformed coordinates, and the labels are the pen tip's true coordinates. The Euclidean distance between the output coordinates and the true coordinates is used as the evaluation function, and the model with the smallest Euclidean distance is saved as the optimal model.

Images