Short-focus projection touch geometry optical adaptive accurate calibration method

Abstract

The application discloses a short-focus projection touch geometric optical adaptive accurate calibration method, relates to the technical field of projection touch calibration, and is used for solving the problems of projection and touch geometric coupling, complicated calibration and long-term overall drift correction, and comprises the following steps: a unified geometric optical model and a layered state vector are established; a factory side establishes a model geometric optical priori, a residual base and a drift mode library based on multiple sample mechanisms; when a terminal is installed, a few-point joint calibration is completed by combining attitude and distance observation and a small amount of user clicks; in a running stage, projection side and touch side drift are identified by using multi-source observation data, and a state vector is updated; based on screen area error, uncertainty and interactive density calculation information value, an interface control layout is adjusted, and an explicit calibration trigger area is adjusted, so that the interactive area can maintain high and stable projection alignment accuracy and touch positioning consistency under different installation conditions and long-term operation.

Description

Technical Field

[0001] This invention relates to the field of projection touch calibration technology, specifically a short-throw projection touch geometric optics adaptive precision calibration method. Background Technology

[0002] Interactive projection systems combining short-throw and ultra-short-throw projectors with touch detection devices have been widely used in education, corporate meetings, and exhibitions. A typical structure involves fixing the projector close to a wall or screen, using a short throw ratio to create a large image over a short distance. This is then combined with touch devices such as infrared touch frames, infrared cameras, or depth cameras to enable interactive operations like writing on an electronic whiteboard, clicking on interface controls, and dragging and dropping images or courseware. To ensure a intuitive user experience, existing products generally offer projection geometry correction functions (such as keystone correction, four-corner correction, and multi-point geometry correction) and touch calibration functions (such as multi-point click calibration). Some systems also incorporate camera-assisted automatic keystone correction or automatic image shaping to simplify image adjustment during installation.

[0003] In such systems, geometric correction of the projected image and calibration of the touch coordinates are typically performed independently by different functional modules, establishing separate geometric mappings from projection pixels to the screen plane and from the touch sensor to the screen plane, lacking a unified geometric model and parameter constraints. Short-throw and ultra-short-throw optical structures are characterized by large projection angles and significant lens distortion, making the projection optical path highly sensitive to minor projector displacements, bracket deformations, and partial screen warping. Touch devices such as touch frames or cameras are also prone to slight positional or ergonomic shifts during wiping, collisions, and disassembly / reassembly. Since image correction and touch calibration are often calculated separately at a specific moment and used long-term, when the aforementioned physical changes occur, the projection side may automatically or manually correct the image to realign with the wall or writing board, while the touch side maintains its original calibration; or the touch device position may change without synchronous adjustment of the projection geometry, leading to discrepancies in the correspondence between the same screen position in the two mappings. This results in inconsistencies between the user's click or writing position and the system's recognized position, especially noticeable at screen edges and corners, affecting interaction accuracy and increasing the workload of repeated on-site debugging and recalibration.

[0004] The technical problem addressed by this invention is that in short-throw or ultra-short-throw projection touch systems, when the projector mounting posture, projection surface state, and touch detection device position change over time, the existing technology's projection geometry correction and touch coordinate calibration are independent of each other and lack a mechanism for coordinated maintenance of the two under a unified geometric model. This makes it difficult to maintain a stable and consistent correspondence between the projected image and the touch coordinates under long-term use and environmental changes. Summary of the Invention

[0005] (a) Technical problems to be solved

[0006] To address the shortcomings of existing technologies, this invention provides a short-throw projection touch geometric optics adaptive precision calibration method. By establishing a unified geometric optics model and hierarchical state vectors, the factory side constructs a model geometric optics prior, residual basis, and drift mode library based on multiple prototypes. During terminal installation, a small-point joint calibration is completed by combining attitude and distance observations and a small number of user clicks. During operation, multi-source observation data is used to identify the drift between the projection side and the touch side and update the state vectors. Based on screen area error, uncertainty, and interaction density, the information utilization value is calculated, and the layout of interface controls and the explicit calibration trigger area are adjusted. This ensures that the interactive area maintains high and stable projection alignment accuracy and touch positioning consistency under different installation conditions and long-term operation, thus solving the technical problems described in the background art.

[0007] (II) Technical Solution

[0008] To achieve the above objectives, the present invention provides the following technical solution:

[0009] The short-throw projection touch geometric optics adaptive precision calibration method includes: establishing a geometric optics model, defining the coordinate system of projector pixels, screen and touch sensor, decomposing the projection and touch mapping into projection optical mapping and touch mapping, and forming a state vector by the static structural parameters of the model, installation posture parameters and optical and touch residual states;

[0010] Based on calibration data from multiple prototypes in different postures, the factory side fits the model-level static structural parameters to obtain geometric optical priors, and extracts residual basis functions and drift mode libraries based on optical residuals and touch residuals samples on the screen.

[0011] During installation, geometrical optics priors are loaded, installation attitude parameters are obtained based on attitude and distance observations, and markers are projected in the interactive area to guide users to click on calibration points. The initial coefficients of optical residual state and touch residual state are obtained based on calibration observations.

[0012] During the operation phase, observation data is collected, projection and touch coordinates are calculated based on the state vector, and projection residuals and touch residuals are obtained by comparing with actual observations. The residuals are mapped to the drift pattern library to identify drift, and the state vector is updated accordingly.

[0013] The error and uncertainty of each sub-region are calculated based on the updated state vector. The information utilization value is obtained by combining the regional observation density and control type. The layout of interface controls and the explicit calibration trigger area are determined according to the information utilization value.

[0014] Furthermore, the geometric state vector includes a global state component for describing the shared geometric relationships of the entire device and a local state component for describing the geometric relationships of the screen interaction area. The local state component has a coordinate definition that is consistent with the projector pixel coordinate system, the screen geometric coordinate system, and the touch sensor coordinate system.

[0015] Furthermore, the static structural parameters of the projector include the intrinsic parameters of the projector's optical system, lens distortion description parameters, and geometric arrangement parameters of the mirrors or refractive elements in the projection path. The installation posture parameters include the three-dimensional position and orientation of the projector relative to the screen's geometric coordinate system, as well as the three-dimensional position and orientation of the touch sensor relative to the screen's geometric coordinate system.

[0016] Furthermore, the residual basis is obtained by establishing a regular control grid on the screen geometric coordinate system and fitting the optical and touch residuals of multiple prototypes using bicubic spline interpolation. The drift mode library is established by performing principal component decomposition on the sample matrix composed of long-term residual sequences and selecting several principal directions as drift modes.

[0017] Furthermore, based on the static structural parameters and installation posture parameters of the device model, the terminal divides the screen interaction area into multiple sub-regions, assigns corresponding local state components to each sub-region, and simultaneously obtains the initial values ​​of each local state component during joint solution to form a region-level geometric description in the geometric state vector.

[0018] Furthermore, the marker graphic includes multiple discrete markers arranged at predetermined positions in the screen's geometric coordinate system. The terminal assigns a number to each marker and determines the coordinates of the calibration point in the screen's geometric coordinate system based on the corresponding number after the user completes the calibration click. This coordinate is used to participate in the joint solution of the geometric state vector.

[0019] Furthermore, the multi-source observation data includes attitude and distance observations from the inertial measurement unit and distance measurement sensor, projected images acquired by the camera, and touch coordinate sequences output by the touch sensor. The terminal performs time alignment on the observations within a time window before using them to calculate the residuals.

[0020] Furthermore, the terminal calculates the observation weight for each touch event. The observation weight is determined by the stability of the touch trajectory, the touch dwell time, and the type of control corresponding to the touch on the interface. When calculating the residual, the residuals of each touch event are weighted and accumulated according to the observation weight to update the geometric state vector.

[0021] Furthermore, the terminal employs a recursive estimation algorithm when updating the geometric state vector. Based on multi-source observation data, it calculates the update gain and updates the installation attitude parameters, optical residual state, and touch residual state. After each update, the updated geometric state vector is used for the next round of residual calculation.

[0022] Furthermore, the terminal calculates the utilization value of regional information based on the geometric error and observation density of each area of ​​the screen. The geometric error is determined by the statistical results of the optical residual and touch residual within the area, and the observation density is determined by the number of touch events and the dwell time of the area within the time window.

[0023] Furthermore, the terminal sensors include an inertial measurement unit, a tilt sensor, and a distance measurement sensor. The terminal collects the relative distance, tilt angle, and attitude data between the projector and the screen to roughly solve for the installation attitude parameters, and uses the rough result as the initial estimate when jointly solving for the initial values ​​of the installation attitude parameters.

[0024] Furthermore, when arranging controls on the interface, the terminal prioritizes placing fine controls that require precise positioning in screen areas with higher information utilization value, while placing larger controls or controls with lower positioning requirements in screen areas with lower information utilization value.

[0025] Furthermore, the terminal sets a threshold for the global error. When the global error calculated multiple times exceeds the threshold, a visible calibration prompt is projected on the screen area with high value for the use of regional information. After the user completes the visible calibration, the updated geometric state vector is written to the storage medium.

[0026] (III) Beneficial Effects

[0027] This invention provides an adaptive and precise calibration method for short-throw projection touch geometric optics, which has the following advantages:

[0028] A unified coordinate system is established for the projector pixel coordinate system, the screen geometric coordinate system, and the touch sensor coordinate system. The static structural parameters, installation posture parameters, optical residual state, and touch residual state of the projector are represented by a hierarchical geometric state vector. This allows the projection side and the touch side to be described in the same model, which facilitates unified updates and maintenance during factory calibration, terminal installation, and operation. It also supports consistent calling and transmission of geometric relationships in subsequent steps.

[0029] By fitting calibration data from multiple prototypes under various operating conditions, the geometric optical prior of the model is obtained. Optical residual basis, touch residual basis, and drift pattern library are extracted on the screen geometric coordinate system. The common characteristics and drift rules of the model are pre-fixed on the factory side, so that the field terminal only needs to adjust the installation posture parameters and residual coefficients on a small number of parameters to adapt to the field installation. This realizes the separation of calibration responsibilities at the model level and single-unit level, which facilitates large-scale deployment and maintenance.

[0030] By combining attitude and distance sensors to obtain an initial estimate of the installation attitude, and projecting a small number of coded marked graphics in the interactive area, the user is guided to complete a small-point calibration. Using these observations and the model's geometric optics prior, the initial values ​​of the installation attitude parameters, optical residual state, and touch residual state are solved. During the initial deployment or position adjustment, there is no need for large-scale dot matrix calibration, so that a small number of high-quality observations can establish the initial geometric state covering the entire screen.

[0031] During equipment operation, touch events, projected images, and attitude and distance observations are continuously collected and uniformly converted into projection residuals and touch residuals. These are then matched with a drift pattern library to distinguish between projection optical-side drift and touch-side drift. Based on the drift type, the installation attitude parameters, optical residual status, and touch residual status are updated respectively, so that the geometric status update is concentrated on the drift location. This enables differentiated processing of different error sources and suppresses error accumulation during long-term use.

[0032] By placing controls with lower precision requirements in low-value areas and prioritizing the triggering of explicit calibration prompts in high-value areas when global errors exceed limits, the interface layout and error distribution are linked, continuously generating effective observations of geometric calibration during normal teaching and meeting interactions, thus achieving synergy between interface design and geometric optical calibration. Attached Figure Description

[0033] Figure 1 This is a schematic diagram of the short-throw projection touch geometric optics adaptive precision calibration method of the present invention. Detailed Implementation

[0034] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0035] Please see Figure 1 This invention provides a short-throw projection touch geometric optics adaptive precision calibration method, including,

[0036] Step 1, the first step of the short-throw projection touch geometric optics adaptive precision calibration method, is to establish a geometric optics model and state variables that correspond one-to-one with the actual engineering structure based on the physical optical path and touch detection structure. All subsequent calibrations and updates are performed using a set of parameters, thereby providing a unified geometric representation for steps 2 to 5.

[0037] In short-throw or ultra-short-throw projection systems, the projector's light-emitting surface, screen surface, and touch detection plane do not overlap in space. If the projection pixels and touch coordinates are directly used for empirical fitting, it is difficult to find the source of error and it is inconvenient to make adjustments when the equipment structure changes.

[0038] Therefore, in three-dimensional space, it is necessary to establish projection geometry and touch geometry for the projection plane, screen geometry plane and touch detection plane respectively in space, and connect them with a unified coordinate system.

[0039] Any change in geometric state can be physically identified as a change in structure or attitude, allowing engineers to directly understand the meaning of parameters based on the structure.

[0040] First, a three-dimensional world coordinate system is selected at the installation site, incorporating the floor, walls, and installation reference points of the classroom or conference room space into the description. Then, three working coordinate systems are derived from this system: projector pixel coordinates, screen geometric coordinates, and touch sensor coordinates. Specifically, a horizontal line near the bottom edge of the screen in the classroom space can be used as a reference line near the origin of the world coordinate system, and the screen normal direction can be used as one of the coordinate axes of the world coordinate system, establishing a fixed correspondence between the world coordinate system and the building structure. The imaging chip or light valve plane inside the projector is defined as a two-dimensional coordinate plane of the projector pixel coordinate system, where each integer grid point corresponds to a pixel position in the projected image. The screen surface, as an approximate plane or slightly curved surface, is represented parametrically in the world coordinate system, and a screen geometric coordinate system is established on this surface to describe the actual landing points of the projected image and touch operations. The working plane or sensing plane of the touch detection device (such as an infrared touch frame or depth camera) is defined as the touch sensor coordinate system, used to record the measurement coordinates of each touch event.

[0041] Based on the above coordinate definition, an actual physical point near the center of the screen can be used as the reference point between the world coordinate system and the screen geometric coordinate system, a point near the light output center of the projector lens can be used as the reference point between the world coordinate system and the projector pixel coordinate system, and a physical point near the geometric center of the touch frame can be used as the reference point between the world coordinate system and the touch sensing coordinate system.

[0042] By measuring or designing drawings, the approximate positions of these reference points in the world coordinate system can be given, providing initial values ​​for subsequent parameter solutions. In a typical classroom embodiment, the installer only needs to fix the ultra-short-throw projector to the wall mount above the blackboard and fix the infrared touch frame around the blackboard. The system can then automatically initialize the relationship between the world coordinates and the three working coordinate systems in the background based on information such as the installation height and mounting hole positions designed in the product design.

[0043] This method of establishing three working coordinate systems allows users to simply fix the projector, screen, and touch frame in their approximate designed positions during installation. The system then uses this structural information to generate a structured geometric model in the background. Since the relationship between the three coordinate systems is described using the world coordinate system, when any component changes position, the corresponding attitude parameters have a clear geometric meaning, facilitating targeted adjustments in subsequent steps.

[0044] The parameters in each geometric model correspond to physical components. When maintenance personnel see changes in the parameters, they can immediately identify whether the bracket is loose, the curtain is sagging, or the touch frame is shifting.

[0045] Based on the definitions of the three working coordinate systems, the geometric transformation of the entire system is divided into two levels of mapping: from projector pixels to screen geometry and from screen geometry to touch sensing. A geometric state vector is also defined. First, a pixel in the projector pixel coordinate system is denoted as the projection pixel. The corresponding point projected onto the screen surface by the projector's optical system is denoted as the screen geometric point. From the projected pixels To screen geometry point Represented using a parameterized projection function, such as:

[0046]

[0047] Among them, screen geometric points : Two-dimensional coordinates in the screen's geometric coordinate system, taking values ​​from all points within the effective display area of ​​the screen; projected pixels. : Two-dimensional coordinates in the projector pixel coordinate system, taken as the effective pixel grid on the projector's imaging chip; model-level static structural parameters : Used to describe lens focal length, principal point offset, lens distortion coefficient, and internal reflection structure, etc., which are related to specific models but remain basically unchanged after installation; installation posture parameters Used to describe the rotation angle and translation distance of the projector relative to the screen in the world coordinate system; optical residual state. : Used to describe the residual geometric errors outside of the ideal optical model, caused by shell deformation, local optical path errors, etc., whose components change with the screen position.

[0048] Projection optical mapping function It consists of three parts: the ideal projection part: using a pinhole projection model or equivalent model, and employing static structural parameter vectors. Construct the intrinsic parameter matrix using the installation attitude parameter vector. Construct an extrinsic parameter matrix to store pixel coordinates. After normalization, the image is projected onto the plane of the screen; lens distortion correction: based on the static structure parameter vector. The radial and polynomial distortion coefficients in the image are used to perform radial and tangential distortion compensation on the projection points on the normalized imaging plane; residual field superposition: utilizing the optical residual state vector and optical residual basis functions in screen geometric coordinates Calculate the residual vector Then add it to the ideal projection result.

[0049] In one specific implementation, the projection geometry mapping function It can be implemented in the following order:

[0050] Based on the model-level static structural state vector The intrinsic parameter matrix in the middle represents the projector pixel coordinate vector. Normalization yields normalized imaging coordinates; using The radial and tangential distortion coefficients are used to correct the distortion of the normalized imaging coordinates; the distortion-corrected ray direction is then determined based on the installation attitude state vector. The projector pose is transformed to the world coordinate system, and the intersection point with the screen plane equation is found to obtain the initial values ​​of the screen's geometric coordinates. According to the optical residual state vector and optical residual basis functions Calculate screen points residual vector at , and take:

[0051]

[0052] This yields the final screen geometric coordinates.

[0053] Furthermore, the geometric points on the screen are mapped to measurement points in the touch sensing coordinate system, thus mapping the screen geometric points... The corresponding touch coordinates are the touch measurement points. Then it can be represented by a touch mapping function as:

[0054]

[0055] Among them, touch measurement points Two-dimensional coordinates in the touch sensing coordinate system, representing the effective sensing area of ​​the touch detection device; model-level static structural parameters. This section describes fixed characteristics such as the nominal aperture, sensing area, and pixel arrangement of the touch frame or camera; mounting posture parameters. This section includes the rotation and translation of the touch detection device relative to the screen; touch residual state. This describes the local nonlinear error that occurs in touch detection devices during long-term use; its components vary with screen position. A unified geometric state vector is constructed. Geometric state vector Each item in the text can be found in the physical structure.

[0056] Touch geometry mapping function It can be constructed using the following steps: through static structure parameter vectors and installation attitude parameter vector Establish a rigid body transformation from the screen geometric coordinate system to the touch sensor coordinate system (for infrared touch frames, this is a planar perspective mapping; for camera touch, it's a projection from the screen plane to the camera image plane); use this rigid body transformation or homography matrix to transform the screen geometric coordinate system... Convert to ideal touch coordinates; use touch residual state vector Combining touch residual basis functions Calculate the local residual vector This is superimposed onto the ideal touch coordinates.

[0057] In one specific embodiment, for a system employing an infrared touch frame structure, the touch geometry mapping... It can be constructed as follows: During the factory calibration phase, through multi-touch testing, a homography transformation or polynomial mapping is fitted between the screen geometric coordinate system and the touch frame local coordinate system. The coefficients of this mapping serve as the model-level static structural state vector. Part of; during the terminal's operation phase, when the screen's geometric coordinate vector Given the initial values, the initial values ​​of the touch coordinates are first obtained using the homography transformation obtained in step (1). Based on the touch residual state vector and touch residual basis function Calculate screen points Corresponding touch residual vector , and take:

[0058]

[0059] This yields the final touch coordinates.

[0060] In another embodiment, for a system that uses a camera to acquire touch trajectories, the camera can be treated as a conventional camera. During the factory calibration phase, the camera's intrinsic and extrinsic parameters are determined. These parameters are then used to calibrate the homography between the screen plane and the image plane. The system is then constructed in a manner similar to the infrared touch frame described above. .

[0061] This avoids fitting the entire projection pixel to touch coordinates into a black box function, making it easier to identify and correct different error sources later, thereby reducing the need for overall recalibration.

[0062] In short-throw and ultra-short-throw optical systems, projection distortion and touch errors are usually manifested at the edges and local areas of the screen. If only a few global parameters are used to express the residuals, the structure will be obscured, making it impossible to perform calibration effectively.

[0063] Therefore, while maintaining a consistent state vector, it is necessary to consider the optical residual state. and touch residual state By adopting a locally expressible form, the system can enhance its descriptive capabilities in important areas while maintaining simplicity in insensitive areas, thus achieving both globality and locality within a limited range of parameters.

[0064] optical residual state It is interpreted as a two-dimensional vector field in the screen's geometric coordinate system, used to describe the small-amplitude movement of projected pixels when they are projected onto the screen through an ideal optical model.

[0065] To describe the two-dimensional vector field within a finite number of parameters, a set of basis functions covering the entire interactive area can be arranged on the screen's geometric coordinate system. For example, several control points can be divided according to a regular grid, and the displacement of each control point can be used as a parameter to continuously extend the field between the control points using a bicubic interpolation function.

[0066] Therefore, the optical residual state It can be represented as a set of displacement parameters for all control points, with the displacement of each control point corresponding to the projection error trend within a local area of ​​the screen. During engineering implementation, control points near the edges and corners of the screen can be arranged more densely to enhance the ability to describe edge distortion of the short-focal-length optical system, while the density of control points can be appropriately reduced in the central area of ​​the screen, thereby effectively capturing distortion features without increasing too many parameters.

[0067] Adopting the optical residual state A similar approach is used for touch residual states. Construct a residual field representation defined in the screen's geometric coordinate system. Touch residual state. This can be understood as the difference between the actual geometric position of a point on the screen and the estimated screen position calculated by the touch detection system. This difference forms a two-dimensional vector field as the position changes. Similarly, a grid of control points with interpolation functions can be used to arrange control points for the touch residual field, incorporating the offset parameters of each control point into the touch residual state. The difference lies in the fact that the touch residual field focuses more on the field of view coverage of the touch detection device and the area related to the density of infrared emitters and receivers. Therefore, the number of control points can be appropriately increased near the edge of the touch device, in areas with dense sensors, or in frequently used interaction areas to more precisely characterize the touch error distribution.

[0068] Step 2: Under the premise of a unified geometric optical model and geometric state vector, construct a model-level static structural prior and drift mode library through multi-sample prototypes and multi-scenario data from the factory, so that the state solution at the installation site and operation phase has a specific initial range and typical evolution path to follow.

[0069] The common optical structural principles shared by this model of short-throw or ultra-short-throw projection touch system are expressed using a set of stable model-level static structural parameters and finite-dimensional optical residual basis functions, thereby separating the commonalities and individual differences between devices of the same model. Through this pre-separation, subsequent on-site adjustments to individual parameters within a small offset range are sufficient to meet geometric accuracy requirements, eliminating the need to construct entirely new optical models for each device.

[0070] First, the projection geometry is jointly calibrated for multiple prototypes of the same model in a controlled factory environment. In practice, a flat standard screen is set up in a dark room, and multiple short-throw or ultra-short-throw projectors are fixed sequentially or in parallel on a standard support, ensuring the initial relationship between the projector pixel coordinate system and the world coordinate system is geometrically controllable. Each projector sequentially projects a series of clearly structured images, such as test images containing regular grids, diagonals, and circular markers. Images projected onto the standard screen are captured by a high-precision camera, and the positions of each feature point in the image within the screen's geometric coordinate system are measured.

[0071] At this point, a row of projectors can be seen projecting test images onto the same standard screen in turn, while cameras remain stationary and continuously collect data. Each prototype takes turns emitting light, forming multiple sets of paired data on the projection pixel position and the screen's geometric position.

[0072] After the above operations are completed, the measurement data of all prototypes are compiled and unified to standardize the model-level static structural parameters in the geometric state vector. For unknown quantities, a cost function is used to characterize the geometric deviation between the model prediction and the actual measurement, and the fitting results of the model-level static structural parameters are obtained by solving the minimum point of the cost function.

[0073] The objective function can be in the following form:

[0074]

[0075]

[0076] Where: model-level static structural parameters The obtained model-level static structural parameters are determined by the allowable range of optical design parameters; objective function Model-level static structural parameters The weighted sum of the overall geometric deviations of all prototypes and all calibration points; prototype number The range of values ​​is arrive This indicates the number of prototypes participating in the joint calibration;

[0077] Calibration point number : Value arrive , indicating the first The number of feature points projected onto a standard screen by the prototype;

[0078] weight : No. prototype The weight of each calibration point reflects its importance or measurement reliability in the screen geometric coordinate system; it is a positive real number. (Screen geometric measurement points) The first result measured manually or by image algorithms on a standard screen. prototype The screen geometric coordinates of each calibration point;

[0079] Projected pixels : Corresponding projector pixel coordinates; standard installation posture parameters The nominal installation posture of the prototype during factory calibration is generally constrained within a small range by the fixture structure;

[0080] zero vector Optical residual state During the initial fitting at the factory, detailed residuals are not introduced; only model-level static structural parameters are used. Capture the dominant geometric relationships.

[0081] The objective function described above can be solved using a nonlinear minimization algorithm with damped terms, such as an iterative solver based on the Gauss-Newton approach for model-level static structural parameters. Iterative updates are performed until the geometric deviation converges. In engineering implementation, factory engineers only need to fix multiple prototypes on standard supports, play a set of test images, acquire calibration images, and run the solution program to obtain a set of static structural parameters applicable to the entire model. This yields model-level static structural parameters. In subsequent steps, it can be directly loaded onto each new device as a unified optical model basis.

[0082] In use, a large number of optical parameters that originally needed to be solved individually on each device are now uniformly fitted at the factory stage. This allows for minor adjustments to be made during on-site installation to account for individual differences, reducing the computational workload of on-site calibration and the reliance on professional personnel. Furthermore, because the collected data comes from multiple prototypes and various attitudes, the model-level static structural parameters are also optimized. Insensitivity to individual errors of a particular device leads to a change in the subsequent residual state. It can focus on describing individualized biases.

[0083] Obtaining model-level static structural parameters Subsequently, by utilizing the residual distributions of multiple prototypes under various postures, an optical residual basis function defined on the screen geometric coordinates was constructed.

[0084] In practical implementation, a grid covering the entire effective display area can be divided on a standard screen, and the position of each grid node in the screen's geometric coordinate system can be denoted as... The residual vectors of all prototypes near the node are statistically analyzed. The residual vectors are derived from the difference between the ideal geometric mapping prediction position and the actual measurement position when the prototype projection feature point is near the node.

[0085] To express these residual forms within a finite number of parameters, a set of basis functions that smoothly vary over the screen geometry are used to represent the optical residual states. This can be represented as a linear combination of these basis functions. It can be expressed as:

[0086]

[0087] Where: optical residual vector At the geometric coordinates point on the screen The optical residual at a point is the offset between the ideal geometrically predicted position and the actual position; the screen geometric point. Arbitrary two-dimensional coordinates in the screen geometric coordinate system; number of basis functions The number of basis functions chosen to represent optical residuals, whose values ​​are determined by a trade-off between engineering-acceptable parameter dimension and representational accuracy; basis function coefficients. : No. The weights of each optical residual basis function will be used as the optical residual state. Part of; optical residual basis functions : Geometric points on the screen A changing two-dimensional vector function can physically correspond to typical distortion patterns such as local stretching, local twisting, or edge warping.

[0088] Two-dimensional spline functions, radial basis functions, or principal modes extracted from principal component analysis can be selected as optical residual basis functions. Furthermore, a set of representative basis function forms were obtained by solving the residual field samples from multiple prototypes.

[0089] When the specific equipment is running in the field, only the basis function coefficients need to be adjusted. This allows for the expression of optical distortion of a specific device within a unified basis function space, without altering the model-level static structural parameters. .

[0090] Optical residual state during use Instead of being a high-dimensional quantity that is independently assigned to each screen position, it is represented by finite-dimensional basis function coefficients. This approach simplifies subsequent state estimation; simultaneously, all devices share the same set of optical residual basis functions. This allows the drift pattern library and online update strategy to be compared and migrated within the same function space, which is beneficial for reusing experience across multiple devices.

[0091] Furthermore, the geometric errors of the touch detection part and their temporal evolution patterns under different operating scenarios are solidified using the touch residual basis and drift mode library. In subsequent on-site joint calibration and multi-source observation status updates during operation, it is possible to quickly determine whether the error belongs to a certain mode based on prior knowledge and make local modifications within the parameter space of that mode.

[0092] First, focus on the geometric behavior of the touch detection device in a factory environment. In an environment similar to optical calibration, multiple touch frames or touch cameras of the same model are paired with a projector and installed around a standard screen. The projector operates according to the pre-fitted model-level static structural parameters. Perform geometric configuration and project a series of test patterns for touch calibration onto the screen.

[0093] Follow the instructions to tap different locations on the screen using a stylus or finger. The system records the touch measurement point in the touch sensing coordinate system for each tap. And record the geometric points on the screen using a camera or manually. The actual location. For each screen geometry position. This allows us to obtain the difference between the estimated position given by the touch system and the actual position, which is defined as the touch residual vector.

[0094] Based on touch residual samples collected from multiple prototypes and multiple clicks, the touch residual state is... It can also be represented as a vector field defined on the screen's geometric coordinates, and expressed through a weighted combination of a finite number of touch residual basis functions. It can be written as:

[0095]

[0096] Where: Touch residual vector : On screen geometry points The touch error generated by the touch detection device refers to the offset between the actual geometric position and the screen position estimate given by the touch system; the screen geometric point... The meaning is consistent with the above; the number of touch residual basis functions The number of basis functions selected to represent touch error is determined by a combination of factors including the spatial resolution of the touch device, the layout of the infrared transmitter and receiver, and the complexity of the engineering implementation.

[0097] basis function coefficients : No. The weights of each touch residual basis function will be used as the touch residual state. Part of; Touch residual basis function : Geometric points on the screen The changing two-dimensional vector function can reflect features such as nonlinear compression near the edge of the touch frame and edge deformation caused by the camera angle.

[0098] Optical residual field and touch residual field are compared through screen geometric coordinates The basis function expansion on is in the form of:

[0099]

[0100] Optical residual vector Touch residual vector : Represents screen geometric coordinates respectively Local offset between projection geometry and touch geometry. Optical residual basis function. Touch residual basis function : A two-dimensional vector function defined on the screen's geometric coordinate plane. Optical residual coefficients. Touch residual coefficient : Set into optical residual state vector Touch residual state vector In a preferred embodiment, the optical residual basis function and touch residual basis function Constructed using bicubic B-spline interpolation. Specifically:

[0101] A control point grid is laid out according to a row and column rule on the screen geometric coordinate system. The geometric coordinates of each control point serve as a node of the B-spline control grid, and the number of control points is equal to the number of basis functions. or The two-dimensional displacement vector of each control point is used as the coefficient of the corresponding basis function. The residual distribution between control points is obtained by interpolation using the bicubic B-spline standard formula; optical residual state vector. The system stores the two-dimensional displacement of all optical control points and the touch residual state vector. The system stores the two-dimensional displacement of all touch control points; the above interpolation process can be implemented directly using existing B-spline interpolation libraries or public algorithms.

[0102] During use, touch residual state Instead of being difficult-to-understand high-dimensional data, it appears in the form of a limited number of physically meaningful touch residual basis functions, allowing subsequent steps to identify and adjust touch-side deviations individually.

[0103] In actual use, short-throw or ultra-short-throw projection touch systems are affected by factors such as long-term stress on the support structure, thermal expansion and contraction, screen loosening, and minor impacts to the touch frame, which can affect the optical residual state. and touch residual state It will change slowly over time. In order to quickly determine whether the trend of the current state changes belongs to a certain type of known drift behavior with limited observations during the operation phase, it is necessary to collect long-term data in various use scenarios in the factory or small-scale pilot phase, and extract typical drift patterns from them.

[0104] In practice, several prototypes can be selected and run long-term in a real classroom or conference room to visualize the geometric state vectors of these devices. The estimation results from different dates were recorded, with particular attention paid to the optical residual state. and touch residual state The trajectory of changes in the basis function coefficients.

[0105] The time-series data are clustered according to the feature vector jointly formed by the optical residual coefficients and the touch residual coefficients, resulting in several typical drift patterns. Each pattern includes not only a static residual field shape but also the approximate direction and speed of the residual coefficient evolution over time. Subsequently, a corresponding parameter update strategy can be formulated for each drift pattern; for example, in some patterns, the optical residual state is mainly adjusted. In some modes, the main adjustment is to the touch residual state. These patterns and their strategies are then stored in the drift pattern library.

[0106] In practice, the complex temporal evolution behavior is reduced to a finite number of nameable drift patterns, which reduces the state search space required at runtime and provides a basis for subsequent updates of drift sources and parameters based on patterns. The runtime adaptive capability does not have to rely on a large amount of historical data and can converge faster through well-organized pattern knowledge.

[0107] Step 3: After the terminal is initially installed or restored to factory settings, based on the model prior and drift pattern library formed in Step 2, and using the attitude and distance information measured by the terminal's onboard sensors, as well as a small number of calibration samples obtained by user clicks, the installation attitude parameter sub-vector of the current device is jointly solved. and residual parameter subvectors , The initial coefficients are used to establish the initial geometric state covering the entire interaction area.

[0108] When the terminal is first powered on, the actual installation conditions often differ significantly from the factory test conditions. The actual position of the screen, the relative distance between the projector and the screen, and the installation height and tilt angle of the touch device may all be different. If all installation posture parameters and residual parameters are derived by directly relying on the user clicking a small number of calibration points, it is easy to have an excessively large solution space and many local minima, which would require increasing the number of calibration points and would defeat the purpose of minimal calibration.

[0109] Therefore, it is necessary to first utilize the geometric information provided by hardware such as inertial measurement units, tilt sensors, and time-of-flight distance sensors to determine the sub-vectors of static structural parameters at the aircraft level. Under the constraints, the installation attitude parameter subvector We provide a set of rough estimates that are close to the true values. At the same time, based on the actual screen outline and projection coverage relationship, we divide the interactive sub-regions in the screen geometric coordinate system, providing a structural framework for the subsequent local solution of residual parameters.

[0110] After the terminal is powered on, the device-level static structure parameter sub-vectors obtained in step two are first loaded from local storage or via cloud configuration. Optical residual basis functions, touch residual basis functions, and drift mode library.

[0111] By reading the gravity direction and body attitude information output by the inertial measurement unit, and combining the pitch and roll angles fed back by the tilt sensor, the angle between the projector's optical axis and the horizontal plane is calculated.

[0112] Distance distribution across the screen area is collected using a time-of-flight distance sensor, and the approximate position and normal direction of the screen's geometric coordinate system relative to the world coordinate system are fitted. Using this measured angle and distance information as input, the aircraft-level static structural parameter sub-vectors are then used. Under constraints, install attitude parameter subvectors Initialize to a set of coarse values. After coarse estimation, the system projects a light-colored background image covering the entire screen and uses a camera or photosensitive element to detect the boundaries of image brightness and darkness, thereby identifying the outline of the actual usable area of ​​the screen. Based on the positional relationship of the screen's geometric coordinate system, the area is divided into several interactive sub-regions, such as the main area for writing, the side area for menu operation, and the top area that may have slight curvature. An independent local residual parameter index is prepared for each interactive sub-region.

[0113] After the terminal is powered on, the geometry calibration program first loads the static structural parameter sub-vectors corresponding to this model from the read-only memory or network interface. And the definitions of optical and touch residual basis functions. The program-controlled inertial measurement unit continuously collects several sets of acceleration vectors and gyro angular velocity vectors in a stationary state. Through the attitude calculation logic inside the sensor, the attitude estimate of the fuselage in the world coordinate system is obtained, such as the pitch angle of the fuselage about the horizontal axis and the yaw angle about the vertical axis.

[0114] Furthermore, the readings provided by the tilt sensor can serve as redundant information for attitude estimation, while the time-of-flight distance sensor returns distance values ​​at different positions on the screen by scanning multiple angles in the direction the screen is in.

[0115] In a typical embodiment, the terminal device is mounted on a wall-mounted bracket above the blackboard in the classroom, and the projector projects an image onto the blackboard below. When the device is first turned on, the projector remains stationary, and the direction of gravity measured by the inertial measurement unit is stably pointing to the ground. The distance values ​​measured by the time-of-flight distance sensor at the top, middle, and bottom of the screen do not change much with the height, which can be used to infer that the screen is an approximately vertical plane.

[0116] Based on the above sensor data, the geometry calibration program will install attitude parameter sub-vectors. The component representing rotation is set to an initial value consistent with the direction of gravity and the normal direction of the screen, while the component representing translation is set to the sum of the screen distance measured by the distance sensor and the projector installation height.

[0117] The specific evaluation process can use geometric trigonometric relationships, treating the projector lens exit point as a known point in the world coordinate system, and the origin of the screen geometric coordinate system as the point to be evaluated. The relative rotation angle between the two coordinate systems is constrained by the direction of gravity, and the translation is constrained by distance measurement.

[0118] In use, the terminal's inherent sensor information is utilized to determine the static structural parameter sub-vectors at the device level. Under the given optical structural constraints, quickly generate subvectors of the mounting attitude parameters. By providing a set of physically reasonable initial values, subsequent point calibration and solution do not need to start the search from arbitrary initial poses, thus reducing the solution space and alleviating the strict requirements on the distribution of user click positions.

[0119] Install attitude parameter subvectors Given the initial estimate, the effective display area of ​​the screen is further identified, and interactive sub-regions are divided in the screen's geometric coordinate system to provide a flexible local geometric expression for subsequent few-point calibration.

[0120] In one implementation process, after the terminal completes the initial attitude rough estimation, it projects a white background image with a thin outline superimposed on the edge of the projected image, and the camera captures an image of the front of the screen. The image processing program detects changes in edge brightness, identifies the contour of the screen boundary on the camera's imaging plane, and then uses the projection geometry mapping established in steps one and two to perform the image processing. and static structure parameter subvectors Install attitude parameter subvectors The current estimate maps the screen outline back to the screen geometric coordinate system, resulting in the polygonal boundary of the actual usable interactive area.

[0121] Based on this, the interactive area is divided into multiple sub-regions according to the definition. For example, the main projection area can be divided into an equal-area grid, or it can be divided into a writing area, a presentation area, and a control area according to the application. A local state index is set for each interactive sub-region, which is the global optical residual parameter sub-vector. and touch residual parameter subvector The subscript of the basis function coefficient array is the set of coefficient subscripts that are preferentially associated with the subregion.

[0122] When in use, the observation samples obtained from subsequent few-point calibration are associated with specific screen regions through screen contour recognition and interactive sub-region division. This allows the basis function coefficients that need to be updated to be selected according to sub-regions when solving residual parameters, improving the targeting of parameter updates and avoiding indiscriminate adjustments in the complete residual space. This helps to reduce the number of calibration points and improve the local accuracy of the initial state.

[0123] Install attitude parameter subvectors Even with reasonable initial values ​​and the interactive area already defined, it is still necessary to explicitly reflect the difference between the device-level prior and the current actual installation in the optical residual parameter subvector through a small number of user-participated calibration clicks. and touch residual parameter subvector Among them.

[0124] If we simply use the traditional four-point or nine-point calibration and fit a uniform transformation between the projected pixels and the touch coordinates, we cannot make full use of the residual basis functions and drift mode library that have been built in step two, and it is also difficult to make the components of the residual parameter vector converge to a suitable range under the condition of a small number of calibration points.

[0125] Therefore, with a small number of high-quality calibration points as constraints, the static structural parameter subvectors at the model level are... Under the condition of invariance, for the subvector of installation attitude parameters By jointly solving the residual basis function coefficient vector, the parameters at both levels can be adapted to the current environment.

[0126] First, by projecting a specific calibration pattern, the user is guided to touch several representative locations within the interactive area, and the projector pixel coordinate vector corresponding to each calibration point is recorded. Touch sensor coordinate vector and the screen geometric coordinate vector derived from the camera or geometry. Then, install the attitude parameter subvector. The current estimated value, optical residual parameter subvector and touch residual parameter subvector The initial coefficients (which can be derived from the initial patterns in the drift pattern library) are used as the starting point for the joint solution. By constructing a calibration error objective function and using the projection-touch error of all calibration points as constraints, the system is gradually adjusted. and residual basis function coefficients, so that the projective geometric mapping and touch geometry mapping The theoretical touch position is brought as close as possible to the actual measured touch sensing coordinate vector. .

[0127] Once the terminal enters calibration mode, the projector projects patterns with a few geometrically coded markers onto the screen. For example, a small square with a cross and a number is projected in the upper left, lower right, and center positions of the main interaction area, and a small rectangular marker is projected in the menu area. The user taps these marked areas sequentially with a stylus or finger according to the numbers displayed on the screen. The system records the touch sensor coordinate vector of each touch event in the touch sensor coordinate system. Simultaneously, the projector pixel coordinate vector corresponding to the marker center is obtained through the projector's frame buffer or driver interface. If the device is equipped with a camera, it will simultaneously capture images of the screen area during the user's click, and use image recognition to determine the position of the marker center in the screen's geometric coordinate system. If no camera is provided, the projective geometry mapping constructed in steps one and two can be utilized. and static structure parameter subvectors Install attitude parameter subvectors The current estimate will be the projector pixel coordinate vector. The predicted values ​​are mapped to screen geometric coordinates and then fine-tuned based on the actual screen outline.

[0128] During use, through a carefully designed set of calibration patterns and acquisition procedures, observation samples containing information from projection, screen, and touch input are collected while keeping the user's workload low. This data is then used to jointly solve for the installation attitude parameter subvectors. and residual parameter subvectors , It provides constraints with complete information.

[0129] Several triplet samples have already been obtained. and installation attitude parameter subvectors Optical residual parameter subvector Touch residual parameter subvector Based on the initial estimate, the above parameters are jointly adjusted by constructing a calibration error objective function. For each calibration point numbered... The touch error vector at this point can be defined as:

[0130]

[0131] Wherein, error vector : in the At each calibration point, the current geometric state vector... The given theoretical touch position and the actual measured touch sensing coordinate vector Deviation between them; touch sensor coordinate vector : No. Measurement results of each calibration point in the touch sensing coordinate system;

[0132] Projective geometric mapping : In static structure parameter subvectors and installation attitude parameter subvector and optical residual parameter subvectors Under the control of [the system], the projector pixel coordinate vector [is...]. Functions mapped to the screen's geometric coordinate system; touch geometry mapping : In static structure parameter subvectors and installation attitude parameter subvector and touch residual parameter subvector Under the control of [the system], it is a function that maps the screen's geometric coordinates to the touch sensor coordinate system.

[0133] Construct a total error objective function at all calibration points:

[0134]

[0135] Wherein, objective function : Install attitude parameter subvectors Optical residual parameter subvector and touch residual parameter subvector The value is the sum of the weighted Euclidean norms of the touch errors at all calibration points when the current estimate is taken;

[0136] Number of calibration points The total number of user-clicked calibration points in this small-point calibration is between 3 and 5; weight. : No. The weights of each calibration point in the objective function are positive real numbers;

[0137] The optical residual parameter subvector corresponding to a certain initial mode in the drift mode library constructed in step two. and touch residual parameter subvector As initial values, the attitude parameter subvector will be installed. The obtained coarse estimate is then adjusted iteratively. The basis function coefficients in the residual parameters, which are related to the interactive sub-region where the current calibration point is located, make the objective function... It tends to be smaller.

[0138] When using it, in the model-level static structural parameter subvector Under the premise of keeping the position unchanged, the installation attitude parameter sub-vectors are adjusted jointly by a small number of calibration points. and residual parameter subvectors , This ensures that the initial geometry of the current device closely matches the actual installation environment; at the same time, by using the initial mode and interactive sub-region index of the drift mode library to control the parameter update range, the risk of under-constraint caused by few-point calibration is reduced.

[0139] Step four involves multi-source observation acquisition, drift source identification, and state updating during the operational phase. This aims to establish the initial geometric state, establish the spatial distribution and temporal evolution characteristics of touch and optical residuals based on multi-source observations within a time window, and determine, under the constraints of the drift pattern library, whether the current geometric offset is primarily caused by the projection optics side, the touch detection side, or both. The installation attitude state vector is then adjusted according to the drift source type. Optical residual state vector Touch residual state vector Perform differentiated updates to maintain geometric accuracy during daily use.

[0140] Even after the terminal completes the multi-point joint calibration in step three, the installation environment will still change slowly or intermittently over time. For example, slight wall deformation, loosening of bracket fasteners, drooping of a corner of the screen, or slight displacement of the touch frame after being touched by cleaning personnel. If we rely solely on the initial calibration state and stop observation, the geometric model cannot reflect these changes in real time, eventually leading to a gradual accumulation of discrepancies between the projected image and the touch position. On the other hand, the device will inevitably generate a large amount of image, sensor, and touch data during operation. If this data is continuously recorded and organized without disrupting user teaching or meetings, an evolutionary picture of the touch residuals and optical residuals can be established over time.

[0141] During operation, the terminal device divides its continuous operating time into multiple time windows according to a set time interval, and each time window is numbered as follows: This corresponds to a time period lasting several seconds or minutes. Within a certain time window... Inside, the system first records the attitude data output by the inertial measurement unit and the distance data output by the time-of-flight distance sensor in chronological order, and then adds timestamps to these observations to form a time series.

[0142] Additionally, there are methods that embed low-visibility geometric markers, such as short line segments or corner points, into the gaps or edges of the teaching or demonstration content, capture screen images at a low frame rate, detect the position of each geometric marker in each frame, and output the geometric coordinate vector of the image.

[0143] For touch observation, the system records the touch sensor coordinate vector, timestamp, touch dwell time, trajectory shape, and trajectory jitter level for each user touch event. It also records the type and location of the control to which the touch event belongs by exchanging information with the interface layer. For example, whether the user clicked a menu button, a page-turning area, or a handwriting pad area, the interface layer stores this semantic information along with the touch observation.

[0144] In one classroom embodiment, the teacher uses the interactive projection device normally to write formulas, turn pages of courseware, and click to play videos. The device records these touch events one by one in the background within the current time window; the camera takes a picture of the screen every ten seconds or so, and the geometric marks near the edge of the blackboard in the image are detected and converted into screen geometric coordinates; the inertial measurement unit records whether the projector's attitude shifts at the beginning and end of each class. Throughout the process, the teacher does not need to perform any additional operations or stop lecturing; the device quietly accumulates observational data for geometric diagnosis simply during normal writing and clicking.

[0145] Multi-source observations are organized into a set of time-labeled observation records within a time window. The system can associate camera observations, touch observations, and attitude observations within the same time period based on the timestamp to form a multi-source observation set within that time window. The system calculates a stability score for each touch observation and calculates the recognition confidence for the geometric markers detected in each frame of the image. These quality indicators are used as weight references for subsequent residual calculations.

[0146] When used, the time window organization method and observation quality are explicitly evaluated. The multi-source data generated naturally during the operation can be stored in an orderly manner according to the time series, and can provide basic data with reliability labels for subsequent residual calculation and drift judgment, avoiding interference from state updates caused by observation quality being lower than required.

[0147] In the time window Internally using the current geometry state vector , , , For each touch and optical observation, a prediction is made, and the predicted value is compared with the actual observed value to construct the touch residual and optical residual. These residuals are then mapped onto the screen plane through the screen geometric coordinate vector, thus forming a residual field defined in the screen geometric coordinate space. For the For a specific touch observation within a time window, the system knows its corresponding projector pixel coordinate vector. (Determined by the currently displayed content), touch measurement coordinate vector and the current geometric state vector , , , Therefore, the touch residual vector can be constructed.

[0148]

[0149] Among them, the touch residual vector : in the At the touch observation point, the touch measurement coordinates and the projection geometric function and touch geometry functions The difference between the calculated theoretical touch coordinates; touch measurement coordinate vector : The coordinates output by the touch detection device in this touch event; Projector pixel coordinate vector The screen position corresponding to the touch event is represented in the projector pixel coordinate system;

[0150] Model-level static structure state vector During the operation phase, maintain the values ​​determined in steps two and three; install the attitude state vector. The relative positions and orientations of the projector, screen, and touch device at the current moment; the optical residual state vector. : The optical error field after expanding the optical residual basis matrix and the corresponding coefficient vector on the screen geometric coordinates;

[0151] Touch residual state vector : The touch error field after expanding the touch residual basis matrix and the corresponding coefficient vector on the screen geometric coordinates.

[0152] For optical observation, if the camera identifies a geometric mark belonging to the projected content in a certain frame of the image, its position in the projector's pixel coordinate system can be determined as a pixel coordinate vector through the process of generating the projected content. The actual measured position of the object in the screen geometric coordinate system is obtained by image processing to obtain the screen geometric measurement coordinate vector. Then the optical residual vector can be defined as

[0153]

[0154] Among them, the optical residual vector : in the At the optical observation point, the screen geometric measurement coordinates and the coordinates through the projected geometric function The difference between the calculated screen predicted coordinates; screen geometric measurement coordinate vector The position of the marker in the screen's geometric coordinate system, as identified by the camera after image processing.

[0155] Based on this, the optical residual vector is calculated. Then, these residual vectors are plotted on the screen plane according to their corresponding screen geometric coordinates, forming a continuous residual field in the sub-region through interpolation. The mass weight and time label of each residual point are recorded. In use, both the direction and magnitude information of the residuals and the spatial distribution of the residuals on the screen are preserved. This allows for a direct observation of phenomena such as a sudden increase in touch residuals in certain areas while the optical residuals do not change significantly, or vice versa, when identifying drift sources and selecting update strategies in specific sub-regions of the screen. This provides quantitatively usable spatial features for drift source identification.

[0156] After obtaining the touch residual field and the optical residual field, if we do not further analyze their spatial morphology and temporal evolution characteristics, but only use a simple threshold judgment to determine the installation attitude state vector... and residual state vector , Performing a unified update can easily lead to unclear update directions or even situations where updates cancel each other out.

[0157] For example, the overall tilt of the projection optics caused by the slow drooping of the bracket, and the increased touch error caused by the local loosening of the touch frame, may manifest as error changes in different areas of the residual field. If the source of error is not distinguished and the two types of residuals are mixed for updating, the error may be reduced in some areas but amplified in others.

[0158] Step two has already constructed a drift pattern library, abstracting typical drift behaviors in different use cases into drift patterns. The goal of this sub-step is to use the drift pattern library to project and match the current residual field, thereby determining which drift patterns the geometric offset of the current time window is closer to, and selecting the installation attitude state vector based on this. Optical residual state vector and touch residual state vector Different update channels and update ranges.

[0159] First, the touch residual vectors and optical residual vectors collected in the time window are aggregated according to the screen sub-regions. For example, the residuals in the writing main area are combined with the residuals in the menu area and the desktop extension area to form multiple region-level residual vectors. Then, the optical drift mode and touch drift mode given in the drift mode library are projected onto the drift mode space to estimate the contribution of different types of drift modes in the current time window.

[0160] Assume the optical drift mode matrix in the drift mode library is The touch drift mode matrix is The optical residual aggregation vector obtained by splicing the optical residual vectors of each region is: The touch residual aggregation vector obtained by concatenating the touch residual vectors of each region is given by [formula missing]. Then, the optical drift coefficient vector and the touch drift coefficient vector can be obtained:

[0161]

[0162] Among them, the optical drift mode matrix Each column represents an optical drift pattern extracted from long-term experimental data in step two, corresponding to a specific residual distribution in the screen's geometric coordinate space; Touch drift pattern matrix. Each column represents a touch drift pattern, reflecting the typical shape of the error distribution of the touch detection device in a specific scenario; optical residual aggregation vector. : A high-dimensional column vector formed by sequentially arranging all optical residual vectors within this time window, where each component corresponds to the optical residual at a specific screen location; Touch residual aggregation vector : A high-dimensional column vector formed by sequentially arranging all touch residual vectors within the current time window;

[0163] Optical drift coefficient vector : Projection coefficients of the current optical residual aggregation vector on each optical drift mode, with components taking values ​​in the range of real numbers; Touch drift coefficient vector : Projection coefficients of the current touch residual aggregation vector in each touch drift mode.

[0164] In one embodiment, the optical drift mode matrix This was obtained by performing principal component analysis on long-term collected optical residual aggregated vector samples. Specifically:

[0165] Optical residual aggregation vectors of a certain model and multiple prototypes under various operating conditions are stacked column-wise to form a sample matrix; the covariance matrix of the sample matrix is ​​decomposed eigenvalues, and the top few eigenvectors with larger eigenvalues ​​are selected and stacked column-wise to form a sample matrix. Touch drift mode matrix It can be constructed in the same way, based on touch residual samples.

[0166] By analyzing the optical drift coefficient vector With touch drift coefficient vector The absolute value, relative proportion, and distribution among different modes can be used to determine which optical drift modes and which touch drift modes the geometric offset within the current time window is closer to. For example, if the optical drift coefficient vector The component in a certain slow-descent mode is significantly larger than in other modes, while the touch drift coefficient vector... If the overall value is small, it indicates that the current residual mainly originates from the attitude change on the projection optics side; conversely, if the touch drift coefficient vector is large... The component in the loose mode on one side of a certain touch frame is significantly more prominent, while the optical drift coefficient vector... If the level remains low, it can be determined that the main source of drift is currently from the touch side.

[0167] In use, the complex touch residual field and optical residual field are mapped to the drift mode coefficient space through the drift mode matrix. Based on the drift experience obtained in step two under different environments, the system can perform a patterned interpretation of the residual distribution of the time window, distinguish the primary and secondary relationship between projection optical drift and touch detection drift, and provide a basis for subsequent state updates.

[0168] After obtaining the drift source classification results, it is necessary to select the appropriate state vector for the installation attitude. Optical residual state vector and touch residual state vector Different update channels and update step sizes are used to avoid the three types of state vectors being significantly adjusted simultaneously by the same residual field and thus interfering with each other.

[0169] Therefore, based on the drift mode coefficient vector , Based on historical state change records, determine the adjustable range of various state vectors within the current time window, and construct a state correction matrix. The residual information is mapped to the state correction value.

[0170] In simplified form, the state update relationship can be written as follows:

[0171]

[0172] Among them, the updated state vector : The geometric state vector at the end of the current time window, which contains the installation attitude state vector. Optical residual state vector and touch residual state vector The updated value; the state vector before the update. : The geometric state vector at the start of the time window;

[0173] State correction matrix This is a matrix that maps the observation residual aggregation vector to the state space correction. Each sub-block can be adjusted based on the drift source classification results, for example, in cases where optical drift is dominant. The corresponding optical residual state vector The sub-block values ​​are relatively large, while the corresponding touch residual state vector The sub-block values ​​are relatively small;

[0174] In one implementation, the state update matrix Refer to the construction method of Kalman gain:

[0175] set up Let the current state covariance matrix be... Let Jacobian matrix be the observation function with respect to the state vector. To determine the observation noise covariance matrix; take:

[0176]

[0177] If you do not wish to maintain the complete covariance matrix, you can It is simplified to a diagonal block matrix, where each diagonal element is a preset learning rate coefficient, and the learning rate corresponding to different state components is dynamically adjusted according to the classification results of the drift mode.

[0178] Observation residual aggregation vector Aggregate vector of touch residuals within the current time window and optical residual aggregation vector The state correction matrix is ​​obtained by combining components with certain weights, and the range of its component values ​​is determined by the magnitude of the residual field and the mass weights. It should be noted that the state correction matrix... Initial values ​​can be pre-set in the factory stage using the drift mode library, and then adjusted according to the drift mode coefficient vector during the runtime stage. , The statistical distribution is adjusted appropriately; at the end of each time window, the system calculates a new geometric state vector according to the above state update relationship. And use it as the initial state for the next time window.

[0179] During the update process, the degree of screen area occlusion and touch behavior patterns are also detected. When it is detected that a student stands on the screen for a long time and occupies most of the area or a child taps the screen multiple times, the aggregated vector of the observation residuals in the corresponding time window is reduced in weight or even ignored in extreme cases, so as to avoid abnormal behavior from affecting the state update.

[0180] In a real classroom scenario, the projector was mounted on a ceiling bracket. After one semester, the bracket slightly bent due to gravity, causing the projected image to shift slightly downwards. Teachers did not notice any significant change during normal times, only occasional click deviations at certain menu buttons. The device detected optical residual aggregation vectors in multiple time windows. The upper edge region shows a gradually increasing trend, while the touch residual aggregation vector The change was relatively gradual. Analysis of the drift mode coefficient vector determined that the drift conformed to a slow-descent optical drift mode of the support structure. Therefore, the optical residual state vector was analyzed over multiple consecutive time windows. and installation attitude state vector By making minor adjustments, the touch residuals in each area of ​​the screen gradually returned to their initial levels, and the teacher did not need to intervene in any way during the process.

[0181] When in use, the drift source classification results are used to control the state correction matrix. The structure and size of the installation attitude state vector Optical residual state vector and touch residual state vector The updates during the operation phase are no longer mechanical adjustments with uniform proportions, but rather selectively strengthen the updates of a certain type of state vector and weaken the changes of other state vectors based on the current drift mode. This ensures that the geometric error is gradually reduced while avoiding unnecessary parameter oscillations and mutual interference.

[0182] Step 5: In the current geometric state vector , , , Under the premise of continuous updates during operation, the distribution of touch residuals and optical residuals on the screen geometric coordinates is transformed into measurable error and uncertainty indicators. Combined with interaction frequency and control semantic information, the information utilization value of each area for geometric state correction in the future period is calculated. On this basis, the layout position of interface controls and explicit calibration prompt strategy are adjusted in a constrained manner to form a stable correspondence between interface form and geometric state evolution.

[0183] Since it is now possible to calculate the touch residual vector and the optical residual vector within the time window, and accordingly adjust the geometric state vector... , , Targeted updates should be made. However, for the UI layer, residual values ​​alone are insufficient to directly guide control layout and user guidance design. UI layout needs to know which areas have large geometric errors, which areas have scarce observations, and which areas are frequently interacted with by users and are sensitive to precision, so as to arrange appropriate control types and interactive prompts within the limited screen space.

[0184] Therefore, regional error measures and uncertainty measures are introduced to aggregate the residual information from step four into interface-interpretable parameters, and on this basis, the information utilization value of each region is constructed.

[0185] Following the region division determined in steps one and two within the screen's geometric coordinate system, the entire effective interactive area is divided into several non-overlapping screen sub-regions, each numbered as follows: .

[0186] For any screen sub-region The system collects touch and optical residual vectors falling within a set observation window, forming a residual set corresponding to that region. For unified processing, the system constructs a composite residual vector for each observation, concatenating touch and optical residual vectors at the same screen location in a predetermined order, denoted as the composite residual vector. And assign a corresponding quality weight to each residual. Quality weight The results are jointly determined by the observation stability, control semantic weights, and occlusion detection results obtained during residual calculation in step four.

[0187] Based on this, for each screen sub-region Define the area error metric The calculation is performed using the following formula:

[0188]

[0189] Among them, the area error measure Within the current observation time window, the screen sub-region The comprehensive geometric error intensity on the index set is a non-negative real number; : In screen sub-region The set consisting of the numbers of all residual samples collected internally, whose elements Integer index; quality weight :serial number The combined residual vector The contribution coefficient in the area error metric is a non-negative real number, which comprehensively considers the stability of the touch event in which the residual sample is located, the control type, and the occlusion situation; the comprehensive residual vector :serial number The error vector splicing result of the observation in the touch coordinate system and the screen geometric coordinate system may include the components of the touch residual vector and the optical residual vector.

[0190] In addition to regional error measurement In addition, the system also needs to measure the uncertainty of each region, which describes the region's current geometric state vector. The reliability of geometric predictions is assessed. Regional uncertainty indices can be constructed by comprehensively considering the fluctuations in regional error measures over recent time windows, the number of observations received in the region, and drift pattern matching results.

[0191] In one specific implementation, the state uncertainty can be calculated in the following form:

[0192] When using recursive least squares or a Kalman-like scheme to estimate the geometric state vector At that time, the internal maintenance state covariance matrix For any screen point Calculate the Jacobian matrix of the predicted coordinates of this point with respect to the state vector. and take

[0193]

[0194] Then The trace or the largest eigenvalue is used as the uncertainty scalar for that point. .

[0195] For example, when a region experiences large residual variations across multiple time windows and has a small number of observations, its uncertainty index will be high; conversely, when a region experiences relatively small long-term residual variations and has a small number of observations, its uncertainty index will be relatively low. In a classroom example, the writing area in the middle of the blackboard has a large number of touch residual samples due to frequent writing and erasing by the teacher, leading to greater confidence in the geometric prediction of this area and a lower uncertainty index. The upper right corner of the blackboard frequently displays class announcements but does not accept touch operations. Although the system can observe the projection through the camera, the touch residual samples are few, resulting in a larger uncertainty index.

[0196] In practice, the touch residuals and optical residuals from step four, which are granular at the level of a single observation, are summarized into error indices and uncertainty indices at the level of screen sub-regions. This allows the interface design and control algorithm to directly adjust the layout and design interactions on a region-by-region basis. Simultaneously, by introducing quality weights and time windows, the impact of individual abnormal residuals on regional error measurement is avoided. This would have too great an impact, thus reducing sensitivity to chance factors while maintaining sensitivity to geometric accuracy.

[0197] After calculating each screen sub-region Regional error measurement In addition to the uncertainty index, we also need to introduce two dimensions: interaction frequency and control semantics, to integrate these factors into a regional information utilization value index. The technical motivation lies in the fact that the degree to which different areas are worth utilizing in subsequent calibration is not entirely determined by the magnitude of geometric errors. Areas with large geometric errors but few user touches can be obtained by guiding users to make a few explicit clicks through interface adjustments, and it is not necessarily necessary to place high-value controls on them for a long time. On the other hand, areas with moderate geometric errors but frequent user clicks and whose control semantics are highly dependent on precise positioning maintain high geometric accuracy during the operation phase, contribute more to the overall interactive experience, and should be reflected in the area information utilization value index.

[0198] Therefore, for each screen sub-region Calculate the interaction frequency index and semantic sensitivity index Interaction frequency index The semantic sensitivity index can be derived from the number and duration of touch events in the area within a few recent time windows. The semantic sensitivity index is determined based on the precision requirements of existing control types within the area; for example, small buttons and slider controls correspond to larger semantic sensitivity indices. Large areas of writing correspond to smaller semantic sensitivity indicators. .

[0199] Based on this, the system defines regional information utilization value indicators. for:

[0200]

[0201] Among them, the regional information utilization value index Screen sub-region The potential for a combined contribution to geometric state correction and interactive experience in the near future, a non-negative real number; semantic sensitivity index. Screen sub-region The sensitivity of internal controls to geometric errors is a non-negative real number, which can be weighted according to the control type or the type of control combination; area error measurement. The meaning is the same as before, indicating the intensity of geometric error in the region; interaction frequency index : Screen sub-regions within a specified time window set The density of internal touch events, which is a non-negative real number, can be calculated by dividing the number of touch events by the duration and taking into account factors such as the average duration of touch events.

[0202] In one specific implementation, the interaction frequency index The calculation can be performed as follows: Within the set statistical time window, the statistical sub-area of ​​the screen is analyzed. Total number of in-touch events and total stay Press again

[0203]

[0204] Calculation, where , These are normalized coefficients. Region weights. The function of each area in the application is preset, for example, a larger value is used for the main writing area and a smaller value is used for the decorative area.

[0205] When using it, the value indicators are utilized through regional information. This system organically combines geometric error, uncertainty, interaction frequency, and control semantic sensitivity, enabling it to intuitively distinguish between areas with large geometric errors but limited impact on interaction, and areas with slightly smaller geometric errors but high frequency and high sensitivity. This provides a more accurate basis for interface layout and user guidance, allowing for a more rational arrangement of controls and prompts used for auxiliary calibration within limited interface resources. This is achieved by obtaining the position of each screen sub-region. Regional information utilization value indicators Next, these metrics need to be translated into specific operations at the interface layer, including fine-tuning the layout and position of controls, selecting control types, and triggering conditions for explicit calibration prompts.

[0206] If only recorded in the background Without providing feedback to the interface, it's impossible to fully utilize the touch observations generated by users in daily use; conversely, drastic adjustments to the interface layout might disrupt the operating habits of teachers or speakers. Therefore, the design goal of this sub-step is to adjust the precise positions of some controls, the timing of the appearance of some auxiliary controls, and the projection area of ​​simplified explicit calibration graphics within a limited range, without disrupting the overall application interface structure, so as to maximize the utilization value of regional information. Higher-quality areas receive more high-quality touch observations or explicit calibration observations in subsequent time periods, thereby improving the efficiency of overall geometric state correction.

[0207] Interface controls are categorized into two types: adjustable controls and stable controls. Adjustable controls include interface elements that have some floating space on the screen but do not depend on a fixed absolute position, such as page-turning buttons, some toolbar icons, and accessibility function entries. Stable controls include controls that are highly coupled to the whiteboard content or a fixed area, such as a fixed function bar at the edge of the whiteboard or a cursor position indicator area. For adjustable controls, the system utilizes the value index based on the area information updated within the current time period. Select one or more high-value areas and limit the target positions of these adjustable controls to a predefined set of candidate points within these areas. The control layout adjustment procedure is called during each break between classes or meetings, based on the previous stage... Distribute and select new control locations so that high-information-value areas can accommodate more controls with high precision requirements and frequent interactions in subsequent time periods.

[0208] In terms of usage, it utilizes regional information value indicators. The adjustable control layout in the higher area enables this area to generate more touch events with high semantic sensitivity and observation quality during subsequent operation, making the residual sample distribution of the state update in step four more concentrated; furthermore, the control layout can be adjusted to times such as breaks or meetings, or to control smaller amplitudes, to ensure that users do not change their usage habits.

[0209] In addition to enhancing the contribution of natural interaction to geometric state correction through fine-tuning of control layout, it is also necessary to trigger a simplified explicit calibration process in a timely manner when the overall geometric error remains at a high level for a long period of time and cannot be further reduced by natural interaction alone, and to utilize value indicators based on regional information. Choose the projection position of the explicit calibration graphic so that a limited number of calibration clicks are concentrated as much as possible in the region most sensitive to the correction of the geometric state vector.

[0210] Therefore, the system uses regional information utilization value indicators. Define global information content index based on This is used to comprehensively reflect the degree of need for additional calibration observations in the current overall state of the screen, and is defined as follows:

[0211]

[0212] Among them, the global information content index : The overall demand intensity for additional calibration observations across the entire screen area during the current evaluation period, with values ​​ranging from non-negative real numbers; area weight. Screen sub-region The relative importance in the global information content index, a non-negative real number, can be pre-set according to the functional positioning of the region in the application, such as the regional weight of the writing main area. It can be larger than the area used solely for decorative display; the value index of area information utilization. The meaning is the same as before, representing a sub-region of the screen. The value of utilizing local information.

[0213] In actual operation, when the global information volume index When the preset threshold is exceeded within several consecutive evaluation cycles, a simplified explicit calibration prompt will pop up at an appropriate time (such as during class breaks, meeting breaks, or when the user actively opens the settings menu). Numbered calibration graphics will be projected in several areas of high screen information utilization value to guide the user to click in sequence.

[0214] The observation samples generated by these clicks will be called again by the calibration and solution logic in step three to adjust the installation attitude state vector. and residual state vector , Significant corrections were made. Since explicit calibration graphics only appear in a few areas and the number of calibration steps is limited, the additional operational burden on users is small; simultaneously, the regional information in these areas has high utilization value. The higher the value, the more significant the effect of explicit calibration clicks on the correction of the geometric state vector.

[0215] When using it, use the global information content indicator. With the introduction of this feature, the system can objectively determine whether it is necessary to add a small number of explicit calibration operations when the correction of geometric errors by natural interaction has approached its limit, and utilize value indicators with the help of regional information. Selecting the most suitable explicit calibration region allows for a trade-off between limited additional user operations and improved geometric state vector accuracy. , , The maximum magnitude correction is achieved, thereby enabling long-term stable geometric consistency maintenance without adding too much calibration burden.

[0216] Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution. Those skilled in the art can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this application.

[0217] Those skilled in the art will understand that, for the sake of convenience and brevity, the specific working processes of the systems, devices, and units described above can be referred to the corresponding processes in the foregoing method embodiments, and will not be repeated here.

[0218] In the several embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.

[0219] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment according to actual needs.

[0220] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A short-throw projection touch geometric optics adaptive precision calibration method, characterized in that: include, Establish a geometric optical model, define the coordinate system of projector pixels, screen and touch sensor, decompose the mapping between projection and touch into projection optical mapping and touch mapping, and form a state vector by the static structural parameters of the model, installation posture parameters and optical and touch residual states; Based on calibration data from multiple prototypes in different postures, the factory side fits the model-level static structural parameters to obtain geometric optical priors, and extracts residual basis functions and drift mode libraries based on optical residuals and touch residuals samples on the screen. During installation, geometrical optics priors are loaded, installation attitude parameters are obtained based on attitude and distance observations, and markers are projected in the interactive area to guide users to click on calibration points. The initial coefficients of optical residual state and touch residual state are obtained based on calibration observations. During the operation phase, observation data is collected, projection and touch coordinates are calculated based on the state vector, and projection residuals and touch residuals are obtained by comparing with actual observations. The residuals are mapped to the drift pattern library to identify drift, and the state vector is updated accordingly. The error and uncertainty of each sub-region are calculated based on the updated state vector. The information utilization value is obtained by combining the regional observation density and control type. The layout of interface controls and the explicit calibration trigger area are determined according to the information utilization value.

2. The short-throw projection touch geometric optics adaptive precision calibration method as described in claim 1, characterized in that: The geometric state vector includes a global state component that describes the shared geometric relationships of the entire device and a local state component that describes the geometric relationships of the screen interaction area. The local state components have coordinate definitions that are consistent with the projector pixel coordinate system, the screen geometric coordinate system, and the touch sensor coordinate system.

3. The short-throw projection touch geometric optics adaptive precision calibration method as described in claim 2, characterized in that: The static structural parameters of the projector include the intrinsic parameters of the projector's optical system, lens distortion description parameters, and geometric arrangement parameters of the mirrors or refractive elements in the projection path. The installation posture parameters include the three-dimensional position and orientation of the projector relative to the screen's geometric coordinate system, as well as the three-dimensional position and orientation of the touch sensor relative to the screen's geometric coordinate system.

4. The short-throw projection touch geometric optics adaptive precision calibration method as described in claim 3, characterized in that: The residual basis is obtained by establishing a regular control grid on the screen geometric coordinate system and fitting the optical and touch residuals of multiple prototypes using bicubic spline interpolation. The drift mode library is established by performing principal component decomposition on the sample matrix composed of long-term residual sequences and selecting several principal directions as drift modes.

5. The short-throw projection touch geometric optics adaptive precision calibration method as described in claim 4, characterized in that: The terminal divides the screen interaction area into multiple sub-regions based on the static structural parameters and installation posture parameters of the model. It assigns a corresponding local state component to each sub-region and obtains the initial values ​​of each local state component during joint solution to form a region-level geometric description in the geometric state vector.

6. The short-throw projection touch geometric optics adaptive precision calibration method as described in claim 5, characterized in that: The marker graphic includes multiple discrete markers arranged at predetermined positions in the screen's geometric coordinate system. The terminal assigns a number to each marker and determines the coordinates of the calibration point in the screen's geometric coordinate system based on the corresponding number after the user completes the calibration click. This coordinate is used to participate in the joint solution of the geometric state vector.

7. The short-throw projection touch geometric optics adaptive precision calibration method as described in claim 6, characterized in that: Multi-source observation data includes attitude and distance observations from inertial measurement units and distance measurement sensors, projected images acquired by cameras, and touch coordinate sequences output by touch sensors. The terminal performs time alignment of the observations within a time window before using them to calculate residuals.

8. The short-throw projection touch geometric optics adaptive precision calibration method as described in claim 7, characterized in that: The terminal calculates the observation weight for each touch event. The observation weight is determined by the stability of the touch trajectory, the touch dwell time, and the type of control corresponding to the touch on the interface. When calculating the residual, the residuals of each touch event are weighted and accumulated according to the observation weight to update the geometric state vector.

9. The short-throw projection touch geometric optics adaptive precision calibration method as described in claim 8, characterized in that: When updating the geometric state vector, the terminal uses a recursive estimation algorithm to calculate the update gain based on multi-source observation data, update the installation attitude parameters, optical residual state and touch residual state, and use the updated geometric state vector for the next round of residual measurement after each update.

10. The short-throw projection touch geometric optics adaptive precision calibration method as described in claim 9, characterized in that: The terminal calculates the utilization value of regional information based on the geometric error and observation density of each area of ​​the screen. The geometric error is determined by the statistical results of the optical residual and touch residual within the area, and the observation density is determined by the number of touch events and the dwell time of the area within the time window.

11. The short-throw projection touch geometric optics adaptive precision calibration method as described in claim 10, characterized in that: The terminal sensors include an inertial measurement unit, a tilt sensor, and a distance measurement sensor. The terminal collects data on the relative distance, tilt angle, and attitude between the projector and the screen to roughly solve for the installation attitude parameters, and uses this rough result as the initial estimate when jointly solving for the initial values ​​of the installation attitude parameters.

12. The short-throw projection touch geometric optics adaptive precision calibration method as described in claim 11, characterized in that: When arranging controls on the interface, the terminal prioritizes placing fine controls that require precise positioning in screen areas with higher information utilization value, while placing larger controls or controls with lower positioning requirements in screen areas with lower information utilization value.

13. The short-throw projection touch geometric optics adaptive precision calibration method as described in claim 12, characterized in that: The terminal sets a threshold for the global error. When the global error calculated multiple times exceeds the threshold, a visible calibration prompt is projected on the screen area where the regional information is of high value. After the user completes the visible calibration, the updated geometric state vector is written to the storage medium.

Images