A touch point real-time calibration simulation system based on digital twinning

Abstract

The application discloses a kind of real-time calibration simulation systems of touch point based on digital twinning, including touch point data acquisition module, and the basic data of touch point calibration are collected;Correction point screening module, effective correction point pair set is screened by Tukey screening algorithm;Coordinate mapping calibration module, initial coordinate transformation matrix is generated by constructing digital mapping relationship;Twinning model construction module, touch point digital twinning model is constructed;Touch point deviation twinning simulation module, mapping deviation simulation is carried out by Delaunay deviation field simulation;Calibration strategy collaborative decision module determines target correction point and calibration mode;Visual interactive display module, double-track visual rendering is carried out and interactive adjustment instruction is received;Feedback update module, initial coordinate transformation matrix is updated by DLT-SVD reconstruction algorithm.The application improves the real-time of touch point calibration simulation by Tukey screening and Delaunay deviation field simulation.

Description

Technical Field

[0001] This invention relates to the field of digital twin simulation technology, and in particular to a real-time calibration simulation system for touch points based on digital twins. Background Technology

[0002] With the continuous development of smart education, conference presentations, industrial interactive control, and large-screen visualization applications, laser interactive systems have attracted widespread attention because they can extend ordinary projection screens or display planes into touch-sensitive interactive interfaces. Existing laser interactive systems typically use cameras to capture the image coordinates of laser touch points or correction points, and then use screen sampling points to establish a coordinate transformation relationship between the image coordinates and the screen coordinates, thereby achieving touch point positioning and interactive response. However, the following problems commonly exist in practical applications: After installation, the system is susceptible to factors such as camera position offset, projection image scaling, screen movement, environmental vibration, and temperature changes, causing the original coordinate transformation relationship to drift and resulting in deviations between the touch point mapping position and the actual screen target position. Existing calibration methods mostly rely on one-time static calibration. When local point offsets occur, it is usually necessary to re-perform full-screen calibration, which is time-consuming and can easily interrupt ongoing interactive applications. Some systems only provide calibration point acquisition results or single path coordinate display, lacking a direct comparison between the image-side mapping point and the screen target point, making it difficult for users to judge the direction and degree of deviation in a timely manner. At the same time, due to the influence of reflection, obstruction, accidental touches, and sampling noise, abnormal correction points are prone to participate in matrix solving, causing a decrease in the stability of the coordinate transformation matrix, which in turn affects the touch positioning accuracy and real-time calibration reliability.

[0003] Therefore, how to provide a real-time calibration simulation system for touch points based on digital twins is a problem that urgently needs to be solved by those skilled in the art. Summary of the Invention

[0004] One objective of this invention is to propose a real-time touch point calibration simulation system based on digital twins. This invention fully utilizes digital twin modeling, Tukey screening, Delaunay deviation field simulation, and DLT-SVD reconstruction algorithm. It details the implementation process of touch point acquisition, calibration point screening, coordinate mapping calibration, deviation simulation, dual-track visual interaction, and coordinate transformation matrix feedback update in a laser interaction system. It can achieve real-time calibration of touch points under conditions of camera offset, screen movement, environmental vibration, or local point drift. It has the advantages of high calibration efficiency, high touch positioning accuracy, low interference from abnormal points, high degree of visualization of the calibration process, and stable update of the coordinate transformation matrix.

[0005] A real-time touch point calibration simulation system based on digital twin according to an embodiment of the present invention includes the following modules: The touch point data acquisition module collects basic calibration data for the touch points of the laser interaction system. The calibration point filtering module constructs calibration point matching pairs based on the basic data of touch point calibration, and uses the Tukey filtering algorithm to filter the set of valid calibration point pairs from the calibration point matching pairs; The coordinate mapping calibration module, based on the touch point calibration baseline data and the set of effective calibration point pairs, constructs the digital mapping relationship between various coordinate systems and generates the initial coordinate transformation matrix; The twin model construction module utilizes the digital mapping relationship between various coordinate systems and the initial coordinate transformation matrix to construct a digital twin model of the touch point; The touch point deviation twin simulation module, based on the touch point digital twin model, simulates the mapping deviation between the virtual mapping coordinates on the image side and the target virtual calibration point coordinates through Delaunay deviation field simulation, and obtains the touch point deviation simulation results. The calibration strategy collaborative decision-making module determines the target calibration point and the calibration mode of the target calibration point based on the simulation results of the touch point deviation. The visualization and interactive display module performs dual-track visualization rendering of the virtual mapping coordinates on the image side and the coordinates of the target virtual calibration point, displays the simulation results of the touch point deviation, and receives interactive adjustment commands based on the calibration mode of the target calibration point. The feedback update module updates the initial coordinate transformation matrix using the DLT-SVD reconstruction algorithm, generates a calibration coordinate transformation matrix, and feeds the calibration coordinate transformation matrix back to the laser interaction system and the digital twin model of the touch point.

[0006] Optionally, the touch point calibration basic data includes initial image sampling coordinates, initial screen sampling coordinates, camera calibration parameters, screen size parameters, real-time image correction point coordinates, and screen correction point coordinates; The initial image sampling coordinates and real-time image correction point coordinates belong to the camera image coordinate system; the initial screen sampling coordinates and screen correction point coordinates belong to the screen physical coordinate system.

[0007] Optionally, the correction point filtering module specifically includes: The real-time image correction point coordinates and the screen correction point coordinates are used as a set of correction point matching pairs; Initial sampling point pairs are constructed based on the initial image sampling coordinates and the initial screen sampling coordinates, and a reference coordinate transformation matrix is ​​generated based on the initial sampling point pairs; By using a reference coordinate transformation matrix, the coordinates of several real-time image correction points are mapped to the physical coordinate system of the screen to obtain several predicted screen coordinates. The mapping error between the i-th predicted screen coordinates and the i-th screen correction point coordinates is calculated using the L2 norm, and the correction point error is obtained and a correction point error sequence is formed. Calculate the median error of the correction point error sequence; Based on the absolute deviation of the error at each correction point relative to the median error, the robust scaling factor is calculated using the median absolute deviation method. The standardized error of the i-th set of correction point matching pairs is calculated using the error normalization form based on the correction point error, median error, robust scaling factor, Tukey cutoff constant, and non-zero smoothing constant. Based on the standardized error, the robust weight of the i-th set of correction point matching pairs is calculated using the Tukey function. Specifically, if the absolute value of the standardized error is less than 1, the square of the difference between the square of the standardized error and the constant 1 is used as the robust weight; if the absolute value of the standardized error is greater than or equal to 1, the constant 0 is used as the robust weight. Correction point pairs with robust weights greater than 0 are identified as valid correction point pairs, and their corresponding robust weights are used as correction point weights for valid correction point pairs. All valid correction point pairs carrying correction point weights are then aggregated to generate a set of valid correction point pairs.

[0008] Optionally, the coordinate mapping calibration module specifically includes: Based on the screen size parameters, determine the screen origin, screen width, and screen height of the screen physical coordinate system; A virtual calibration interface is generated based on the screen width and screen height. Obtain the display width, display height, and interface origin of the virtual calibration interface. Use the interface origin as the origin of the virtual calibration coordinate system, and use the display width and display height as the horizontal and vertical scales, respectively, to establish the virtual calibration coordinate system. Using the screen origin as the coordinate normalization starting point and the screen width and screen height as the normalization scale, the coordinates of the screen calibration points in the effective calibration point pair set are converted into the coordinates of the target virtual calibration points in the virtual calibration coordinate system, thus generating a mapping relationship from the screen physical coordinate system to the virtual calibration coordinate system. Based on the camera calibration parameters, the coordinates of the real-time image correction points in the effective correction point set are subjected to lens distortion correction, intrinsic parameter normalization and homogenization to obtain the effective homogenous coordinates on the image side. The effective homogeneous coordinates on the image side, the corresponding screen correction point coordinates, and the corresponding correction point weights are used as a set of calibration correction point pairs to obtain a set of calibration correction point pairs. Based on the calibration point pair set, a point pair constraint relationship is constructed from the camera image coordinate system to the screen physical coordinate system; The reference coordinate transformation matrix is ​​calibrated and optimized using point-pair constraint relationships to generate the initial coordinate transformation matrix; Based on the initial coordinate transformation matrix and the mapping relationship from the screen physical coordinate system to the virtual calibration coordinate system, a digital mapping relationship is constructed between the camera image coordinate system, the screen physical coordinate system, and the virtual calibration coordinate system.

[0009] Optionally, the touch point digital twin model is obtained by encapsulating an image sampling point twin layer, a screen target point twin layer, a virtual calibration point twin layer, a coordinate transformation matrix twin layer, and a twin mapping link.

[0010] Optionally, the twin model construction module specifically includes: Parse the numerical mapping relationships and extract the first, second, and third mapping relationships; The first mapping relationship is a mapping relationship from the camera image coordinate system to the screen physical coordinate system; the second mapping relationship is a mapping relationship from the screen physical coordinate system to the virtual calibration coordinate system; and the third mapping relationship is a combined mapping relationship from the camera image coordinate system to the virtual calibration coordinate system. Based on the set of calibration point pairs, a calibration point number is assigned to each set of calibration point pairs; Write the effective homogeneous coordinates of the image side, the coordinates of the real-time image correction points, and the correction point numbers into the twin layer of the image sampling points; Write the screen calibration point coordinates and calibration point numbers into the screen target point twin layer; The first mapping relationship is invoked to convert the effective homogeneous coordinates on the image side into the image side screen mapping coordinates in the screen physical coordinate system, and the image side screen mapping coordinates and correction point numbers are written into the screen target point twin layer. The second mapping relationship is invoked to convert the screen calibration point coordinates into the target virtual calibration point coordinates in the virtual calibration coordinate system, and the target virtual calibration point coordinates and calibration point number are written into the virtual calibration point twin layer; The third mapping relationship is invoked to convert the effective homogeneous coordinates on the image side into the virtual mapping coordinates on the image side in the virtual calibration coordinate system, and the virtual mapping coordinates on the image side and the calibration point number are written into the virtual calibration point twin layer. Write the initial coordinate transformation matrix, matrix version identifier, matrix activation status, and matrix generation time into the coordinate transformation matrix twin layer; According to the calibration point number, the corresponding image-side effective homogeneous coordinates, real-time image calibration point coordinates, screen calibration point coordinates, target virtual calibration point coordinates, image-side virtual mapping coordinates, and initial coordinate transformation matrix of the same set of calibration calibration point pairs are associated and bound to generate calibration point twin association records. Based on the twin association records of the calibration points, a twin mapping link is generated that connects the twin layer of image sampling points, the twin layer of screen target points, the twin layer of virtual calibration points, and the twin layer of coordinate transformation matrix.

[0011] Optionally, the touch point deviation twin simulation module specifically includes: The image-side virtual mapping coordinates corresponding to the same calibration point number are bound to the target virtual calibration point coordinates to generate twin deviation calculation point pairs; Using the coordinates of the target virtual calibration point in each twin deviation calculation point pair as the grid vertices, a Delaunay triangulation grid is constructed, and the virtual calibration coordinate system is divided into multiple Delaunay triangulation simulation elements using the Delaunay triangulation grid. The deviation vector of the j-th twin deviation calculation point pair is calculated based on the coordinate difference between the target virtual calibration point coordinates and the image-side virtual mapping coordinates in the j-th twin deviation calculation point pair. Calculate the L2 norm of the deviation vector to obtain the deviation distance; normalize the deviation vector using the deviation distance to obtain the deviation direction. Read the twin deviation calculation point pairs associated with the three mesh vertices in each Delaunay triangular simulation element, and obtain the deviation vector of the three twin deviation calculation point pairs; Multiple simulation sampling points are set within each Delaunay triangular simulation unit; Based on the centroid coordinates of the m-th simulation sampling point relative to the three vertices of the Delaunay triangular simulation unit, the deviation vectors corresponding to the three vertices are interpolated to obtain the local deviation vector of the m-th simulation sampling point. Calculate the local deviation distance and local deviation direction of the m-th simulation sampling point based on the local deviation vector; The local deviation vector, local deviation distance, and local deviation direction of each simulation sampling point within the same Delaunay triangular simulation unit are used to form a local touch deviation field. The screen diagonal length is calculated based on the screen width and screen height parameters in the screen size parameters, and a first deviation threshold and a second deviation threshold are generated based on the screen diagonal length. When the deviation distance of the j-th twin deviation calculation point pair is greater than the first deviation threshold, the deviation level of the j-th twin deviation calculation point pair is determined to be the coarse adjustment level. When the deviation distance of the j-th twin deviation calculation point pair is less than or equal to the first deviation threshold and greater than the second deviation threshold, the deviation level of the j-th twin deviation calculation point pair is determined as the fine-tuning level. When the deviation distance of the j-th twin deviation calculation point pair is less than or equal to the second deviation threshold, the deviation level of the j-th twin deviation calculation point pair is determined to be the level that does not need adjustment. Based on the deviation level and local touch deviation field, the local drift region identifier of the corresponding Delaunay triangular simulation unit is determined, and the touch point deviation simulation results are generated based on the deviation vector, deviation distance, deviation direction, deviation level and local drift region identifier of the twin deviation calculation point pair.

[0012] Optionally, the calibration strategy collaborative decision-making module specifically includes: Twin deviation calculation point pairs with deviation levels of coarse adjustment or fine adjustment are identified as candidate correction point pairs; The calibration priority is determined based on the deviation level and deviation distance of the candidate calibration point pairs, and the candidate calibration point pairs with the highest calibration priority are determined as the target calibration point pairs; The corresponding calibration point number of the target calibration point is determined as the target calibration point number, and the coordinates of the target virtual calibration point corresponding to the target calibration point are determined as the target calibration point; When the deviation level of the target correction point pair is the coarse adjustment level, the calibration mode of the target correction point is set to coarse adjustment mode; When the deviation level of the target correction point pair is the fine-tuning level, the calibration mode of the target correction point is determined to be the fine-tuning mode; When there are no twin deviation calculation point pairs with deviation levels of coarse or fine, the calibration mode is set to no adjustment required.

[0013] Optionally, the visual interactive display module specifically includes: In the virtual calibration interface, the image-side virtual mapping coordinates and the target virtual calibration point coordinates are displayed on a dual-track overlay according to the same calibration point number, and different display styles are used to distinguish between the image-side virtual mapping coordinates and the target virtual calibration point coordinates. Read the deviation direction, deviation distance, deviation level, and local drift area identifier from the touch point deviation simulation results; In the virtual calibration interface, the deviation direction is displayed by a direction indicator that connects the virtual mapping coordinates on the image side and the coordinates of the target virtual calibration point, the deviation distance is displayed by a numerical label, the deviation level is displayed by a level indicator, and the local drift area is displayed by a region indicator. Based on the target calibration point number, the image-side virtual mapping coordinates associated with the target calibration point and the target virtual calibration point coordinates are activated and displayed in the virtual calibration interface; The calibration mode based on the target calibration point displays the coarse adjustment status, fine adjustment status, or no adjustment status in the virtual calibration interface; When the calibration mode of the target calibration point is coarse adjustment mode or fine adjustment mode, an interactive adjustment command for the target calibration point is received. The interactive adjustment command includes the adjustment direction and the number of adjustment steps.

[0014] Optionally, the feedback update module specifically includes: The calibration step size is determined based on the calibration mode of the target calibration point, and the displacement adjustment amount of the target calibration point is calculated based on the adjustment direction and adjustment step number in the interactive adjustment command. Based on the displacement adjustment, the image-side virtual mapping coordinates corresponding to the target correction point are adjusted to obtain the adjusted image-side virtual mapping coordinates; Transform the adjusted image-side virtual mapping coordinates to the screen physical coordinate system to obtain the adjusted screen mapping coordinates; The effective homogeneous coordinates on the image side corresponding to the target correction point are combined with the adjusted screen mapping coordinates to form an updated correction point pair; Replace the calibration point pairs corresponding to the target calibration point number in the calibration calibration point pair set with the updated calibration point pairs to generate the updated calibration calibration point pair set; Based on the updated set of calibration point pairs, construct a system of linear equations for the DLT. The candidate coordinate transformation matrix is ​​obtained by solving the DLT linear equation system through SVD decomposition. Calculate the weighted projection error after the candidate coordinate transformation matrix is ​​applied to the updated set of calibration correction points; When the weighted projection error meets the matrix update condition, the candidate coordinate transformation matrix is ​​determined as the calibration coordinate transformation matrix; The calibration coordinate transformation matrix is ​​written into the laser interaction system and synchronously updated to the coordinate transformation matrix twin layer in the touch point digital twin model.

[0015] The beneficial effects of this invention are: This invention employs the Tukey filtering algorithm to robustly filter the matching pairs of correction points formed by real-time image correction point coordinates and screen correction point coordinates. This eliminates abnormal correction points caused by reflections, occlusions, accidental touches, and sampling noise. Robust weights are then incorporated into subsequent calibration processes to reduce the amplification of errors caused by abnormal points participating in coordinate matrix solving, thereby improving the reliability of correction point data and the stability of coordinate mapping calibration. Furthermore, a coordinate mapping calibration module constructs a digital mapping relationship between the camera image coordinate system, the screen physical coordinate system, and the virtual calibration coordinate system, generating an initial coordinate transformation matrix. This allows image-side sampling points, screen target points, and virtual calibration points in the laser interaction system to be expressed under a unified mapping link, improving the accuracy and traceability of point conversion between different coordinate systems. During the touch point digital twin model construction stage, the image sampling point twin layer, screen target point twin layer, virtual calibration point twin layer, coordinate transformation matrix twin layer, and twin mapping link are associated and encapsulated to achieve digital expression of touch point acquisition, mapping, deviation, and matrix state. By using Delaunay deviation field simulation to perform regional simulation of the mapping deviation between the virtual mapped coordinates on the image side and the coordinates of the target virtual calibration point, single-point offset and local drift regions can be identified, improving the precision of touch point deviation judgment. Through collaborative decision-making of calibration strategies and dual-track visual interactive display, the deviation direction, deviation distance, deviation level, and local drift region are intuitively presented, and the target correction point and coarse adjustment, fine adjustment, or no adjustment state are determined, improving the operability and efficiency of manual interactive calibration. Finally, the DLT-SVD reconstruction algorithm generates a calibration coordinate transformation matrix based on the interactive adjustment results and feeds it back to the laser interaction system and the digital twin model of the touch point, realizing closed-loop update of the coordinate transformation matrix. Therefore, this invention can improve the real-time performance, accuracy, robustness, and visualization of touch point calibration in laser interaction systems under scenarios such as camera offset, screen movement, environmental vibration, and local point drift. Attached Figure Description

[0016] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings: Figure 1 This is a schematic diagram of a real-time calibration simulation system for touch points based on digital twins proposed in this invention. Figure 2 This is a flowchart of the Tukey screening algorithm in a real-time touch point calibration simulation system based on digital twin proposed in this invention. Figure 3 This is a flowchart of the Delaunay deviation field simulation in a real-time touch point calibration simulation system based on digital twin proposed in this invention. Detailed Implementation

[0017] The present invention will now be described in further detail with reference to the accompanying drawings. These drawings are simplified schematic diagrams, illustrating only the basic structure of the invention, and therefore only show the components relevant to the invention.

[0018] refer to Figures 1-3 A real-time calibration simulation system for touch points based on digital twins includes the following modules: The touch point data acquisition module collects basic calibration data for the touch points of the laser interaction system. The calibration point filtering module constructs calibration point matching pairs based on the basic data of touch point calibration, and uses the Tukey filtering algorithm to filter the set of valid calibration point pairs from the calibration point matching pairs; The coordinate mapping calibration module, based on the touch point calibration baseline data and the set of effective calibration point pairs, constructs the digital mapping relationship between various coordinate systems and generates the initial coordinate transformation matrix; The twin model construction module utilizes the digital mapping relationship between various coordinate systems and the initial coordinate transformation matrix to construct a digital twin model of the touch point; The touch point deviation twin simulation module, based on the touch point digital twin model, simulates the mapping deviation between the virtual mapping coordinates on the image side and the target virtual calibration point coordinates through Delaunay deviation field simulation, and obtains the touch point deviation simulation results. The calibration strategy collaborative decision-making module determines the target calibration point and the calibration mode of the target calibration point based on the simulation results of the touch point deviation. The visualization and interactive display module performs dual-track visualization rendering of the virtual mapping coordinates on the image side and the coordinates of the target virtual calibration point, displays the simulation results of the touch point deviation, and receives interactive adjustment commands based on the calibration mode of the target calibration point. The feedback update module updates the initial coordinate transformation matrix using the DLT-SVD reconstruction algorithm, generates a calibration coordinate transformation matrix, and feeds the calibration coordinate transformation matrix back to the laser interaction system and the digital twin model of the touch point.

[0019] In this embodiment, the basic data for touch point calibration includes initial image sampling coordinates, initial screen sampling coordinates, camera calibration parameters, screen size parameters, real-time image correction point coordinates, and screen correction point coordinates; The initial image sampling coordinates and real-time image correction point coordinates belong to the camera image coordinate system; the initial screen sampling coordinates and screen correction point coordinates belong to the screen physical coordinate system.

[0020] In this embodiment, the calibration point filtering module specifically includes: The real-time image correction point coordinates and the screen correction point coordinates are used as a set of correction point matching pairs; Initial sampling point pairs are constructed based on initial image sampling coordinates and initial screen sampling coordinates, and a reference coordinate transformation matrix is ​​generated based on the initial sampling point pairs. Specifically, each initial image sampling coordinate is paired with its corresponding initial screen sampling coordinate to obtain several sets of initial sampling point pairs; based on these sets of initial sampling point pairs, an initial mapping constraint relationship is established from the camera image coordinate system to the screen physical coordinate system; based on the initial mapping constraint relationship, a reference linear equation system is constructed to solve for the coordinate transformation parameters; the reference linear equation system is solved by minimizing the residuals to obtain the vectorized representation of the coordinate transformation parameters; the vectorized representation of the coordinate transformation parameters is rearranged in 3×3 matrix form to obtain the reference coordinate transformation matrix. By using a reference coordinate transformation matrix, the coordinates of several real-time image correction points are mapped to the physical coordinate system of the screen to obtain several predicted screen coordinates. The mapping error between the i-th predicted screen coordinates and the i-th screen correction point coordinates is calculated using the L2 norm, and the correction point error is obtained and a correction point error sequence is formed. Calculate the median error of the correction point error sequence; Based on the absolute deviation of the error at each correction point relative to the median error, the robust scaling factor is calculated using the median absolute deviation method. The standardized error of the i-th set of correction point matching pairs is calculated using the error normalization form based on the correction point error, median error, robust scaling factor, Tukey cutoff constant, and non-zero smoothing constant. Based on the standardized error, the robust weight of the i-th set of correction point matching pairs is calculated using the Tukey function. Specifically, if the absolute value of the standardized error is less than 1, the square of the difference between the square of the standardized error and the constant 1 is used as the robust weight; if the absolute value of the standardized error is greater than or equal to 1, the constant 0 is used as the robust weight. Correction point pairs with robust weights greater than 0 are identified as valid correction point pairs, and their corresponding robust weights are used as correction point weights for valid correction point pairs. All valid correction point pairs carrying correction point weights are then aggregated to generate a set of valid correction point pairs.

[0021] In this embodiment, the coordinate mapping calibration module specifically includes: Based on the screen size parameters, determine the screen origin, screen width, and screen height of the screen physical coordinate system; A virtual calibration interface is generated based on the screen width and screen height. Obtain the display width, display height, and interface origin of the virtual calibration interface. Use the interface origin as the origin of the virtual calibration coordinate system, and use the display width and display height as the horizontal and vertical scales, respectively, to establish the virtual calibration coordinate system. Using the screen origin as the coordinate normalization starting point and the screen width and screen height as the normalization scale, the coordinates of the screen calibration points in the effective calibration point pair set are converted into the coordinates of the target virtual calibration points in the virtual calibration coordinate system, thus generating a mapping relationship from the screen physical coordinate system to the virtual calibration coordinate system. Based on the camera calibration parameters, the coordinates of the real-time image correction points in the effective correction point set are subjected to lens distortion correction, intrinsic parameter normalization and homogenization to obtain the effective homogenous coordinates on the image side. The effective homogeneous coordinates on the image side, the corresponding screen correction point coordinates, and the corresponding correction point weights are used as a set of calibration correction point pairs to obtain a set of calibration correction point pairs. Based on the calibration point pair set, a point pair constraint relationship is constructed from the camera image coordinate system to the screen physical coordinate system; The reference coordinate transformation matrix is ​​calibrated and optimized using point-pair constraints to generate an initial coordinate transformation matrix. Specifically, the reference coordinate transformation matrix is ​​used as the initial value of the matrix to be optimized. Each effective homogeneous coordinate on the image side is mapped to the screen physical coordinate system using the reference coordinate transformation matrix to obtain the corresponding temporary mapped coordinates. The reprojection error between each temporary mapped coordinate and the corresponding screen correction point coordinates is calculated. The matrix parameters of the reference coordinate transformation matrix are iteratively corrected based on the reprojection error, where the goal of the iterative correction is to reduce the weighted projection error of all effective correction point pairs. When the weighted projection error is less than 0.1, or the error change between two adjacent iterations is less than 0.005, the optimization stops, and the matrix of the current iteration is determined as the initial coordinate transformation matrix. Based on the initial coordinate transformation matrix and the mapping relationship from the screen physical coordinate system to the virtual calibration coordinate system, a digital mapping relationship is constructed between the camera image coordinate system, the screen physical coordinate system, and the virtual calibration coordinate system.

[0022] In this embodiment, the touch point digital twin model is obtained by encapsulating an image sampling point twin layer, a screen target point twin layer, a virtual calibration point twin layer, a coordinate transformation matrix twin layer, and a twin mapping link.

[0023] In this embodiment, the twin model construction module specifically includes: Parse the numerical mapping relationships and extract the first, second, and third mapping relationships; The first mapping relationship is a mapping relationship from the camera image coordinate system to the screen physical coordinate system; the second mapping relationship is a mapping relationship from the screen physical coordinate system to the virtual calibration coordinate system; and the third mapping relationship is a combined mapping relationship from the camera image coordinate system to the virtual calibration coordinate system. Based on the set of calibration point pairs, a calibration point number is assigned to each set of calibration point pairs; Write the effective homogeneous coordinates of the image side, the coordinates of the real-time image correction points, and the correction point numbers into the twin layer of the image sampling points; Write the screen calibration point coordinates and calibration point numbers into the screen target point twin layer; The first mapping relationship is invoked to convert the effective homogeneous coordinates on the image side into the image side screen mapping coordinates in the screen physical coordinate system, and the image side screen mapping coordinates and correction point numbers are written into the screen target point twin layer. The second mapping relationship is invoked to convert the screen calibration point coordinates into the target virtual calibration point coordinates in the virtual calibration coordinate system, and the target virtual calibration point coordinates and calibration point number are written into the virtual calibration point twin layer; The third mapping relationship is invoked to convert the effective homogeneous coordinates on the image side into the virtual mapping coordinates on the image side in the virtual calibration coordinate system, and the virtual mapping coordinates on the image side and the calibration point number are written into the virtual calibration point twin layer. Write the initial coordinate transformation matrix, matrix version identifier, matrix activation status, and matrix generation time into the coordinate transformation matrix twin layer; According to the calibration point number, the corresponding image-side effective homogeneous coordinates, real-time image calibration point coordinates, screen calibration point coordinates, target virtual calibration point coordinates, image-side virtual mapping coordinates, and initial coordinate transformation matrix of the same set of calibration calibration point pairs are associated and bound to generate calibration point twin association records. Based on the twin association records of the calibration points, a twin mapping link is generated that connects the twin layer of image sampling points, the twin layer of screen target points, the twin layer of virtual calibration points, and the twin layer of coordinate transformation matrix.

[0024] In this embodiment, the touch point deviation twin simulation module specifically includes: The image-side virtual mapping coordinates corresponding to the same calibration point number are bound to the target virtual calibration point coordinates to generate twin deviation calculation point pairs; Using the coordinates of the target virtual calibration point in each twin deviation calculation point pair as the grid vertices, a Delaunay triangulation grid is constructed, and the virtual calibration coordinate system is divided into multiple Delaunay triangulation simulation elements using the Delaunay triangulation grid. The deviation vector of the j-th twin deviation calculation point pair is calculated based on the coordinate difference between the target virtual calibration point coordinates and the image-side virtual mapping coordinates in the j-th twin deviation calculation point pair; where j represents the index of the twin deviation calculation point pair. Calculate the L2 norm of the deviation vector to obtain the deviation distance; normalize the deviation vector using the deviation distance to obtain the deviation direction. Read the twin deviation calculation point pairs associated with the three mesh vertices in each Delaunay triangular simulation element, and obtain the deviation vector of the three twin deviation calculation point pairs; Multiple simulation sampling points are set within each Delaunay triangular simulation unit; Based on the centroid coordinates of the m-th simulation sampling point relative to the three vertices of the Delaunay triangular simulation unit, the deviation vectors corresponding to the three vertices are interpolated to obtain the local deviation vector of the m-th simulation sampling point; where m represents the index of the simulation sampling point. Calculate the local deviation distance and local deviation direction of the m-th simulation sampling point based on the local deviation vector; The local deviation vector, local deviation distance, and local deviation direction of each simulation sampling point within the same Delaunay triangular simulation unit are used to form a local touch deviation field. The screen diagonal length is calculated based on the screen width and screen height parameters in the screen size parameters, and a first deviation threshold and a second deviation threshold are generated based on the screen diagonal length. When the deviation distance of the j-th twin deviation calculation point pair is greater than the first deviation threshold, the deviation level of the j-th twin deviation calculation point pair is determined to be the coarse adjustment level. When the deviation distance of the j-th twin deviation calculation point pair is less than or equal to the first deviation threshold and greater than the second deviation threshold, the deviation level of the j-th twin deviation calculation point pair is determined as the fine-tuning level. When the deviation distance of the j-th twin deviation calculation point pair is less than or equal to the second deviation threshold, the deviation level of the j-th twin deviation calculation point pair is determined to be the level that does not need adjustment. Based on the deviation level and local touch deviation field, the local drift region identifier of the corresponding Delaunay triangular simulation unit is determined, and the touch point deviation simulation results are generated based on the deviation vector, deviation distance, deviation direction, deviation level and local drift region identifier of the twin deviation calculation point pair.

[0025] In this embodiment, the calibration strategy collaborative decision-making module specifically includes: Twin deviation calculation point pairs with deviation levels of coarse adjustment or fine adjustment are identified as candidate correction point pairs; The calibration priority is determined based on the deviation level and deviation distance of the candidate calibration point pairs, and the candidate calibration point pairs with the highest calibration priority are determined as the target calibration point pairs; The corresponding calibration point number of the target calibration point is determined as the target calibration point number, and the coordinates of the target virtual calibration point corresponding to the target calibration point are determined as the target calibration point; When the deviation level of the target correction point pair is the coarse adjustment level, the calibration mode of the target correction point is set to coarse adjustment mode; When the deviation level of the target correction point pair is the fine-tuning level, the calibration mode of the target correction point is determined to be the fine-tuning mode; When there are no twin deviation calculation point pairs with deviation levels of coarse or fine, the calibration mode is set to no adjustment required.

[0026] In this embodiment, the visual interactive display module specifically includes: In the virtual calibration interface, the image-side virtual mapping coordinates and the target virtual calibration point coordinates are displayed on a dual-track overlay according to the same calibration point number, and different display styles are used to distinguish between the image-side virtual mapping coordinates and the target virtual calibration point coordinates. Read the deviation direction, deviation distance, deviation level, and local drift area identifier from the touch point deviation simulation results; In the virtual calibration interface, the deviation direction is displayed by a direction indicator that connects the virtual mapping coordinates on the image side and the coordinates of the target virtual calibration point, the deviation distance is displayed by a numerical label, the deviation level is displayed by a level indicator, and the local drift area is displayed by a region indicator. Based on the target calibration point number, the image-side virtual mapping coordinates associated with the target calibration point and the target virtual calibration point coordinates are activated and displayed in the virtual calibration interface; The calibration mode based on the target calibration point displays the coarse adjustment status, fine adjustment status, or no adjustment status in the virtual calibration interface; When the calibration mode of the target calibration point is coarse adjustment mode or fine adjustment mode, an interactive adjustment command for the target calibration point is received. The interactive adjustment command includes the adjustment direction and the number of adjustment steps.

[0027] In this embodiment, the feedback update module specifically includes: The calibration step size is determined based on the calibration mode of the target calibration point, and the displacement adjustment amount of the target calibration point is calculated based on the adjustment direction and adjustment step number in the interactive adjustment command. Based on the displacement adjustment, the image-side virtual mapping coordinates corresponding to the target correction point are adjusted to obtain the adjusted image-side virtual mapping coordinates; Transform the adjusted image-side virtual mapping coordinates to the screen physical coordinate system to obtain the adjusted screen mapping coordinates; The effective homogeneous coordinates on the image side corresponding to the target correction point are combined with the adjusted screen mapping coordinates to form an updated correction point pair; Replace the calibration point pairs corresponding to the target calibration point number in the calibration calibration point pair set with the updated calibration point pairs to generate the updated calibration calibration point pair set; Based on the updated set of calibration point pairs, construct a system of linear equations for the DLT. The candidate coordinate transformation matrix is ​​obtained by solving the DLT linear equation system through SVD decomposition. Calculate the weighted projection error after the candidate coordinate transformation matrix is ​​applied to the updated set of calibration correction points; When the weighted projection error meets the matrix update condition, the candidate coordinate transformation matrix is ​​determined as the calibration coordinate transformation matrix. The matrix update condition is: the weighted projection error corresponding to the candidate coordinate transformation matrix is ​​less than 0.1, or the error change between the weighted projection error corresponding to the candidate coordinate transformation matrix and the weighted projection error corresponding to the currently enabled coordinate transformation matrix is ​​less than 0.005. The currently enabled coordinate transformation matrix is ​​the coordinate transformation matrix whose enabled state is "enabled" in the coordinate transformation matrix twin layer. The calibration coordinate transformation matrix is ​​written into the laser interaction system and synchronously updated to the coordinate transformation matrix twin layer in the touch point digital twin model.

[0028] Example 1: To verify the feasibility of this invention in practice, it was applied to a laser interactive projection system in a smart classroom at a university. The classroom uses a 150-inch projection screen with a physical width of approximately 3.32m and a height of approximately 1.87m. The system consists of a short-throw projector, an infrared laser stylus pen, an external camera, an interactive control host, and visual calibration software. In daily use, the system is primarily used for electronic whiteboard writing, courseware annotation, and collaborative annotation by multiple users. During actual operation, the camera mounting bracket is affected by the opening and closing of doors and windows, the movement of the lectern, and vibrations from equipment heat dissipation. The projected image may also experience scaling or edge shifts due to slight projector displacement, causing a drift in the coordinate transformation relationship between the original image coordinates and the screen's physical coordinates. Traditional one-time calibration methods typically require re-performing full-screen sampling calibration when local deviations occur. Teachers need to pause interactive operations during lectures, and in cases of reflection, obstruction, or accidental touches, abnormal correction points are easily included in the matrix solution, further amplifying subsequent touch point positioning errors.

[0029] In the implementation of this invention, the initial image sampling coordinates, initial screen sampling coordinates, camera calibration parameters, screen size parameters, real-time image correction point coordinates, and screen correction point coordinates are first collected by the touch point data acquisition module. The system sets 25 correction points on the screen, with effective sampling points mainly distributed at the four corners, the midpoint of the edges, and the central area. The correction point filtering module pairs the real-time image correction point coordinates with the screen correction point coordinates, and calculates the standardized error and robustness weight of each pair using the Tukey filtering algorithm. In testing, three sets of abnormal points caused by strong light reflection and hand occlusion were assigned zero weight and excluded, leaving 22 effective correction point pairs for subsequent coordinate mapping calibration. The coordinate mapping calibration module generates a virtual calibration interface based on the screen width and height, establishes a virtual calibration coordinate system, and generates a set of calibration correction point pairs based on the effective homogeneous coordinates on the image side, the screen correction point coordinates, and the correction point weights, obtaining the initial coordinate transformation matrix. Subsequently, the twin model construction module writes the image sampling points, screen target points, virtual calibration points, and coordinate transformation matrix into the corresponding twin layer, forming a touch point digital twin model.

[0030] After the system had been running for 30 minutes, the testers simulated a slight collision with the podium, causing the camera to shift by approximately 1.5mm and resulting in localized touch drift in the upper right area of ​​the projected image. The touch point deviation twin simulation module, using Delaunay deviation field simulation, constructed triangular simulation units with the target virtual calibration point coordinates as mesh vertices and performed regionalized simulation of the deviation between the image-side virtual mapping coordinates and the target virtual calibration point coordinates. The system displayed the image-side virtual mapping coordinates and the target virtual calibration point coordinates in a dual-track format on the virtual calibration interface, along with the deviation direction, deviation distance, deviation level, and local drift area identifier. For areas where the deviation distance exceeded the first deviation threshold, the calibration strategy collaborative decision module classified it as a coarse adjustment level and selected the correction point with the largest deviation as the target correction point; for areas where the deviation distance was between the first and second deviation thresholds, the system classified it as a fine adjustment level. The operator input the adjustment direction and number of steps according to the directional arrows on the interface. The feedback update module used the DLT-SVD reconstruction algorithm to generate a calibration coordinate transformation matrix and wrote it back to the laser interaction system and the touch point digital twin model.

[0031] To compare and analyze the implementation effect of the system of the present invention, it is compared and analyzed with a one-time static whole-screen calibration scheme and a common homography matrix recalculation scheme. The one-time static whole-screen calibration scheme involves manually clicking all preset calibration points to collect image coordinates and screen coordinates at once and solve the coordinate transformation matrix. Subsequent point drift requires recalibrating the whole screen. The common homography matrix recalculation scheme, after detecting touch deviation, directly recalculates the coordinate transformation matrix using the current calibration point, but does not perform abnormal calibration point screening, digital twin modeling, deviation field simulation, or dual-track visual interactive calibration. The comparison results are shown in Table 1.

[0032] Table 1. Comparison of test results for different calibration schemes in the laser interaction system of the smart classroom.

[0033] As shown in Table 1, under conditions of slight camera offset, local screen mapping drift, and interference from abnormal correction points, the one-time static full-screen calibration scheme, while able to complete basic touch positioning, cannot promptly identify local drift areas, resulting in an average touch positioning error of 5.82 mm and a single calibration time approaching 100 seconds, significantly impacting continuous classroom interaction. The ordinary homography matrix recalculation scheme can reduce some positioning errors, but due to the lack of robust screening and deviation field simulation, abnormal correction points may still participate in matrix solving, with the maximum touch positioning error still reaching 7.92 mm. The system of this invention reduces the false participation rate of abnormal points through Tukey screening, identifies local drift areas through digital twin models and Delaunay deviation field simulation, and utilizes dual-track visual interaction to assist operators in quickly confirming the deviation direction and adjustment range. This reduces the average touch positioning error to 1.18 mm, the average reprojection error to 0.92 mm, and the average calibration time to 14.6 seconds. Test results show that the present invention can complete real-time calibration of local touch points without re-performing full-screen calibration, thereby improving the positioning accuracy, calibration efficiency, anti-abnormal interference capability, and continuous use reliability of the laser interactive system in teaching, conference, and display scenarios.

[0034] The above description is only a preferred embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any equivalent substitutions or modifications made by those skilled in the art within the scope of the technology disclosed in the present invention, based on the technical solution and inventive concept of the present invention, should be covered within the scope of protection of the present invention.

Claims

1. A real-time calibration simulation system for touch points based on digital twins, characterized in that, include: The touch point data acquisition module collects basic calibration data for the touch points of the laser interaction system. The calibration point filtering module constructs calibration point matching pairs based on the basic data of touch point calibration, and uses the Tukey filtering algorithm to filter the set of valid calibration point pairs from the calibration point matching pairs; The coordinate mapping calibration module, based on the touch point calibration baseline data and the set of effective calibration point pairs, constructs the digital mapping relationship between various coordinate systems and generates the initial coordinate transformation matrix; The twin model construction module utilizes the digital mapping relationship between various coordinate systems and the initial coordinate transformation matrix to construct a digital twin model of the touch point; The touch point deviation twin simulation module, based on the touch point digital twin model, simulates the mapping deviation between the virtual mapping coordinates on the image side and the target virtual calibration point coordinates through Delaunay deviation field simulation, and obtains the touch point deviation simulation results. The calibration strategy collaborative decision-making module determines the target calibration point and the calibration mode of the target calibration point based on the simulation results of the touch point deviation. The visualization and interactive display module performs dual-track visualization rendering of the virtual mapping coordinates on the image side and the coordinates of the target virtual calibration point, displays the simulation results of the touch point deviation, and receives interactive adjustment commands based on the calibration mode of the target calibration point. The feedback update module updates the initial coordinate transformation matrix using the DLT-SVD reconstruction algorithm, generates a calibration coordinate transformation matrix, and feeds the calibration coordinate transformation matrix back to the laser interaction system and the digital twin model of the touch point.

2. The real-time calibration simulation system for touch points based on digital twins according to claim 1, characterized in that, The basic data for touch point calibration includes initial image sampling coordinates, initial screen sampling coordinates, camera calibration parameters, screen size parameters, real-time image correction point coordinates, and screen correction point coordinates. The initial image sampling coordinates and real-time image correction point coordinates belong to the camera image coordinate system; the initial screen sampling coordinates and screen correction point coordinates belong to the screen physical coordinate system.

3. The real-time calibration simulation system for touch points based on digital twins according to claim 1, characterized in that, The calibration point filtering module specifically includes: The real-time image correction point coordinates and the screen correction point coordinates are used as a set of correction point matching pairs; Initial sampling point pairs are constructed based on the initial image sampling coordinates and the initial screen sampling coordinates, and a reference coordinate transformation matrix is ​​generated based on the initial sampling point pairs; By using a reference coordinate transformation matrix, the coordinates of several real-time image correction points are mapped to the physical coordinate system of the screen to obtain several predicted screen coordinates. The mapping error between the i-th predicted screen coordinates and the i-th screen correction point coordinates is calculated using the L2 norm, and the correction point error is obtained and a correction point error sequence is formed. Calculate the median error of the correction point error sequence; Based on the absolute deviation of the error at each correction point relative to the median error, the robust scaling factor is calculated using the median absolute deviation method. The standardized error of the i-th set of correction point matching pairs is calculated using the error normalization form based on the correction point error, median error, robust scaling factor, Tukey cutoff constant, and non-zero smoothing constant. Based on the standardized error, the robust weight of the i-th set of correction point matching pairs is calculated using the Tukey function. Specifically, if the absolute value of the standardized error is less than 1, the square of the difference between the square of the standardized error and the constant 1 is used as the robust weight; if the absolute value of the standardized error is greater than or equal to 1, the constant 0 is used as the robust weight. Correction point pairs with robust weights greater than 0 are identified as valid correction point pairs, and their corresponding robust weights are used as correction point weights for valid correction point pairs. All valid correction point pairs carrying correction point weights are then aggregated to generate a set of valid correction point pairs.

4. The real-time calibration simulation system for touch points based on digital twins according to claim 1, characterized in that, The coordinate mapping calibration module specifically includes: Based on the screen size parameters, determine the screen origin, screen width, and screen height of the screen physical coordinate system; A virtual calibration interface is generated based on the screen width and screen height. Obtain the display width, display height, and interface origin of the virtual calibration interface. Use the interface origin as the origin of the virtual calibration coordinate system, and use the display width and display height as the horizontal and vertical scales, respectively, to establish the virtual calibration coordinate system. Using the screen origin as the coordinate normalization starting point and the screen width and screen height as the normalization scale, the coordinates of the screen calibration points in the effective calibration point pair set are converted into the coordinates of the target virtual calibration points in the virtual calibration coordinate system, thus generating a mapping relationship from the screen physical coordinate system to the virtual calibration coordinate system. Based on the camera calibration parameters, the coordinates of the real-time image correction points in the effective correction point set are subjected to lens distortion correction, intrinsic parameter normalization and homogenization to obtain the effective homogenous coordinates on the image side. The effective homogeneous coordinates on the image side, the corresponding screen correction point coordinates, and the corresponding correction point weights are used as a set of calibration correction point pairs to obtain a set of calibration correction point pairs. Based on the calibration point pair set, a point pair constraint relationship is constructed from the camera image coordinate system to the screen physical coordinate system; The reference coordinate transformation matrix is ​​calibrated and optimized using point-pair constraint relationships to generate the initial coordinate transformation matrix; Based on the initial coordinate transformation matrix and the mapping relationship from the screen physical coordinate system to the virtual calibration coordinate system, a digital mapping relationship is constructed between the camera image coordinate system, the screen physical coordinate system, and the virtual calibration coordinate system.

5. The real-time calibration simulation system for touch points based on digital twins according to claim 1, characterized in that, The touch point digital twin model is obtained by encapsulating an image sampling point twin layer, a screen target point twin layer, a virtual calibration point twin layer, a coordinate transformation matrix twin layer, and a twin mapping link.

6. The real-time calibration simulation system for touch points based on digital twins according to claim 1, characterized in that, The twin model construction module specifically includes: Parse the numerical mapping relationships and extract the first, second, and third mapping relationships; The first mapping relationship is a mapping relationship from the camera image coordinate system to the screen physical coordinate system; the second mapping relationship is a mapping relationship from the screen physical coordinate system to the virtual calibration coordinate system; and the third mapping relationship is a combined mapping relationship from the camera image coordinate system to the virtual calibration coordinate system. Based on the set of calibration point pairs, a calibration point number is assigned to each set of calibration point pairs; Write the effective homogeneous coordinates of the image side, the coordinates of the real-time image correction points, and the correction point numbers into the twin layer of the image sampling points; Write the screen calibration point coordinates and calibration point numbers into the screen target point twin layer; The first mapping relationship is invoked to convert the effective homogeneous coordinates on the image side into the image side screen mapping coordinates in the screen physical coordinate system, and the image side screen mapping coordinates and correction point numbers are written into the screen target point twin layer. The second mapping relationship is invoked to convert the screen calibration point coordinates into the target virtual calibration point coordinates in the virtual calibration coordinate system, and the target virtual calibration point coordinates and calibration point number are written into the virtual calibration point twin layer; The third mapping relationship is invoked to convert the effective homogeneous coordinates on the image side into the virtual mapping coordinates on the image side in the virtual calibration coordinate system, and the virtual mapping coordinates on the image side and the calibration point number are written into the virtual calibration point twin layer. Write the initial coordinate transformation matrix, matrix version identifier, matrix activation status, and matrix generation time into the coordinate transformation matrix twin layer; According to the calibration point number, the corresponding image-side effective homogeneous coordinates, real-time image calibration point coordinates, screen calibration point coordinates, target virtual calibration point coordinates, image-side virtual mapping coordinates, and initial coordinate transformation matrix of the same set of calibration calibration point pairs are associated and bound to generate calibration point twin association records. Based on the twin association records of the calibration points, a twin mapping link is generated that connects the twin layer of image sampling points, the twin layer of screen target points, the twin layer of virtual calibration points, and the twin layer of coordinate transformation matrix.

7. The real-time calibration simulation system for touch points based on digital twins according to claim 1, characterized in that, The touch point deviation twin simulation module specifically includes: The image-side virtual mapping coordinates corresponding to the same calibration point number are bound to the target virtual calibration point coordinates to generate twin deviation calculation point pairs; Using the coordinates of the target virtual calibration point in each twin deviation calculation point pair as the grid vertices, a Delaunay triangulation grid is constructed, and the virtual calibration coordinate system is divided into multiple Delaunay triangulation simulation elements using the Delaunay triangulation grid. The deviation vector of the j-th twin deviation calculation point pair is calculated based on the coordinate difference between the target virtual calibration point coordinates and the image-side virtual mapping coordinates in the j-th twin deviation calculation point pair. Calculate the L2 norm of the deviation vector to obtain the deviation distance; normalize the deviation vector using the deviation distance to obtain the deviation direction. Read the twin deviation calculation point pairs associated with the three mesh vertices in each Delaunay triangular simulation element, and obtain the deviation vector of the three twin deviation calculation point pairs; Multiple simulation sampling points are set within each Delaunay triangular simulation unit; Based on the centroid coordinates of the m-th simulation sampling point relative to the three vertices of the Delaunay triangular simulation unit, the deviation vectors corresponding to the three vertices are interpolated to obtain the local deviation vector of the m-th simulation sampling point. Calculate the local deviation distance and local deviation direction of the m-th simulation sampling point based on the local deviation vector; The local deviation vector, local deviation distance, and local deviation direction of each simulation sampling point within the same Delaunay triangular simulation unit are used to form a local touch deviation field. The screen diagonal length is calculated based on the screen width and screen height parameters in the screen size parameters, and a first deviation threshold and a second deviation threshold are generated based on the screen diagonal length. When the deviation distance of the j-th twin deviation calculation point pair is greater than the first deviation threshold, the deviation level of the j-th twin deviation calculation point pair is determined to be the coarse adjustment level. When the deviation distance of the j-th twin deviation calculation point pair is less than or equal to the first deviation threshold and greater than the second deviation threshold, the deviation level of the j-th twin deviation calculation point pair is determined as the fine-tuning level. When the deviation distance of the j-th twin deviation calculation point pair is less than or equal to the second deviation threshold, the deviation level of the j-th twin deviation calculation point pair is determined to be the level that does not need adjustment. Based on the deviation level and local touch deviation field, the local drift region identifier of the corresponding Delaunay triangular simulation unit is determined, and the touch point deviation simulation results are generated based on the deviation vector, deviation distance, deviation direction, deviation level and local drift region identifier of the twin deviation calculation point pair.

8. The real-time calibration simulation system for touch points based on digital twins according to claim 1, characterized in that, The calibration strategy collaborative decision-making module specifically includes: Twin deviation calculation point pairs with deviation levels of coarse adjustment or fine adjustment are identified as candidate correction point pairs; The calibration priority is determined based on the deviation level and deviation distance of the candidate calibration point pairs, and the candidate calibration point pairs with the highest calibration priority are determined as the target calibration point pairs; The corresponding calibration point number of the target calibration point is determined as the target calibration point number, and the coordinates of the target virtual calibration point corresponding to the target calibration point are determined as the target calibration point; When the deviation level of the target correction point pair is the coarse adjustment level, the calibration mode of the target correction point is set to coarse adjustment mode; When the deviation level of the target correction point pair is the fine-tuning level, the calibration mode of the target correction point is determined to be the fine-tuning mode; When there are no twin deviation calculation point pairs with deviation levels of coarse or fine, the calibration mode is set to no adjustment required.

9. A real-time touch point calibration simulation system based on digital twin according to claim 1, characterized in that, The visual interactive display module specifically includes: In the virtual calibration interface, the image-side virtual mapping coordinates and the target virtual calibration point coordinates are displayed on a dual-track overlay according to the same calibration point number, and different display styles are used to distinguish between the image-side virtual mapping coordinates and the target virtual calibration point coordinates. Read the deviation direction, deviation distance, deviation level, and local drift area identifier from the touch point deviation simulation results; In the virtual calibration interface, the deviation direction is displayed by a direction indicator that connects the virtual mapping coordinates on the image side and the coordinates of the target virtual calibration point, the deviation distance is displayed by a numerical label, the deviation level is displayed by a level indicator, and the local drift area is displayed by a region indicator. Based on the target calibration point number, the image-side virtual mapping coordinates associated with the target calibration point and the target virtual calibration point coordinates are activated and displayed in the virtual calibration interface; The calibration mode based on the target calibration point displays the coarse adjustment status, fine adjustment status, or no adjustment status in the virtual calibration interface; When the calibration mode of the target calibration point is coarse adjustment mode or fine adjustment mode, an interactive adjustment command for the target calibration point is received. The interactive adjustment command includes the adjustment direction and the number of adjustment steps.

10. A real-time touch point calibration simulation system based on digital twin according to claim 1, characterized in that, The feedback update module specifically includes: The calibration step size is determined based on the calibration mode of the target calibration point, and the displacement adjustment amount of the target calibration point is calculated based on the adjustment direction and adjustment step number in the interactive adjustment command. Based on the displacement adjustment, the image-side virtual mapping coordinates corresponding to the target correction point are adjusted to obtain the adjusted image-side virtual mapping coordinates; Transform the adjusted image-side virtual mapping coordinates to the screen physical coordinate system to obtain the adjusted screen mapping coordinates; The effective homogeneous coordinates on the image side corresponding to the target correction point are combined with the adjusted screen mapping coordinates to form an updated correction point pair; Replace the calibration point pairs corresponding to the target calibration point number in the calibration calibration point pair set with the updated calibration point pairs to generate the updated calibration calibration point pair set; Based on the updated set of calibration point pairs, construct a system of linear equations for the DLT. The candidate coordinate transformation matrix is ​​obtained by solving the DLT linear equation system through SVD decomposition. Calculate the weighted projection error after the candidate coordinate transformation matrix is ​​applied to the updated set of calibration correction points; When the weighted projection error meets the matrix update condition, the candidate coordinate transformation matrix is ​​determined as the calibration coordinate transformation matrix; The calibration coordinate transformation matrix is ​​written into the laser interaction system and synchronously updated to the coordinate transformation matrix twin layer in the touch point digital twin model.

Images